Elasticsearch Efficiently Deleting Snapshots in Elasticsearch

By Opster Team

Updated: Jun 6, 2023

| 2 min read

Introduction

Snapshots are an essential part of managing Elasticsearch clusters, as they provide a way to back up and restore data. However, over time, the number of snapshots can grow, leading to increased storage costs and management complexity.

In this article, we will discuss how to efficiently delete snapshots in Elasticsearch, ensuring that your cluster remains optimized and cost-effective. If you want to learn more about Elasticsearch snapshots in general, check out this guide. You should also take a look at this guide, which contains a detailed explanation on Elasticsearch DELETE.

Prerequisites

Before proceeding, ensure that you have the following:

1. Access to an Elasticsearch cluster with administrator privileges.
2. Familiarity with Elasticsearch snapshots and the snapshot lifecycle management (SLM) feature.

Deleting a Single Snapshot

To delete a single snapshot, you can use the Delete Snapshot API. The syntax for this API is as follows:

DELETE /_snapshot/<repository>/<snapshot>

Replace `<repository>` with the name of the repository containing the snapshot, and `<snapshot>` with the name of the snapshot you want to delete.

For example, to delete a snapshot named `my_snapshot` in a repository named `my_repository`, you would use the following command:

DELETE /_snapshot/my_repository/my_snapshot

Upon successful deletion, Elasticsearch will return a response similar to the following:

{
  "acknowledged": true
}

Deleting Multiple Snapshots

To delete multiple snapshots at once, you can use the Delete Snapshot API with a comma-separated list of snapshot names or wildcard patterns. For example, to delete snapshots `snapshot_1`, `snapshot_2`, and `snapshot_3` from the `my_repository` repository, you would use the following command:

DELETE /_snapshot/my_repository/snapshot_1,snapshot_2,snapshot_3

To delete all snapshots with names starting with `snapshot_` in the `my_repository` repository, you would use the following command:

DELETE /_snapshot/my_repository/snapshot_*

Note that deleting multiple snapshots can take some time, depending on the size and number of snapshots. Be patient and monitor the progress using the Task Management API if necessary.

Automating Snapshot Deletion with Snapshot Lifecycle Management (SLM)

To automate the deletion of snapshots, you can use Elasticsearch’s Snapshot Lifecycle Management (SLM) feature. SLM allows you to define policies that automatically create and delete snapshots based on a schedule and retention rules.

To create an SLM policy, use the Put Snapshot Lifecycle API. The syntax for this API is as follows:

PUT /_slm/policy/<policy_name>
{
  "schedule": "<cron_expression>",
  "name": "<snapshot_name_pattern>",
  "repository": "<repository_name>",
  "config": {
    ...
  },
  "retention": {
    ...
  }
}

Replace `<policy_name>` with a unique name for the policy, `<cron_expression>` with a cron expression defining the snapshot creation schedule, `<snapshot_name_pattern>` with a pattern for naming the snapshots, `<repository_name>` with the name of the repository to store the snapshots, and configure the `config` and `retention` sections as needed.

For example, to create a policy named `my_policy` that creates snapshots daily at midnight and retains them for 30 days, you would use the following command:

PUT /_slm/policy/my_policy
{
  "schedule": "0 0 * * *",
  "name": "daily-snapshot-{now/d}",
  "repository": "my_repository",
  "config": {
    "indices": "*"
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}

This policy will create snapshots with names like `daily-snapshot-2022-01-01`, `daily-snapshot-2022-01-02`, and so on. Snapshots older than 30 days will be automatically deleted, but at least 5 snapshots will always be retained, and no more than 50 snapshots will be kept.

Conclusion

In this article, we have discussed how to efficiently delete snapshots in Elasticsearch using the Delete Snapshot API and the Snapshot Lifecycle Management feature. By properly managing your snapshots, you can optimize your Elasticsearch cluster and reduce storage costs. Remember to monitor your snapshot usage and adjust your deletion strategies as needed to ensure the best performance and cost-effectiveness for your Elasticsearch deployment.