Failed to delete snapshot snapshotId – How to solve this Elasticsearch exception

Opster Team

August-23, Version: 6.8-7.7

Briefly, this error occurs when Elasticsearch fails to delete a snapshot due to reasons like the snapshot is currently in use, the snapshot doesn’t exist, or there are insufficient permissions. To resolve this, ensure the snapshot isn’t being used by any restore operations. If it doesn’t exist, check the snapshot ID for any errors. Lastly, verify that the user has the necessary permissions to delete snapshots.

This guide will help you check for common problems that cause the log ” failed to delete snapshot [” + snapshotId + “] ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: snapshot, blobstore, repositories, delete.

 
 

snapshots

 
 

blobstore

 
 

repositories

Overview

An Elasticsearch snapshot provides a backup mechanism that takes the current state and data in the cluster and saves it to a repository (read snapshot for more information). The backup process requires a repository to be created first. The repository needs to be registered using the _snapshot endpoint, and multiple repositories can be created per cluster. The following repository types are supported: 

Repository types

Repository typeConfiguration type
Shared file systemType: “fs”
S3Type : “s3”
HDFSType :“hdfs”
AzureType: “azure”
Google Cloud StorageType : “gcs”

Examples

To register an “fs” repository:

PUT _snapshot/my_repo_01
{
"type": "fs",
"settings": {
"location": "/mnt/my_repo_dir"
  }
}

Notes and good things to know

  • S3, HDFS, Azure and Google Cloud require a relevant plugin to be installed before it can be used for a snapshot.
  • The setting, path.repo: /mnt/my_repo_dir needs to be added to elasticsearch.yml on all the nodes if you are planning to use the repo type of file system. Otherwise, it will fail.
  • When using remote repositories, the network bandwidth and repository storage throughput should be high enough to complete the snapshot operations normally, otherwise you will end up with partial snapshots.

Log Context

Log “failed to delete snapshot [” + snapshotId + “]” class name is BlobStoreRepository.java. We extracted the following from Elasticsearch source code for those seeking an in-depth context :

 // delete an index that was created by another master node after writing this index-N blob.
 final Map foundIndices = blobStore().blobContainer(indicesPath()).children();
 doDeleteShardSnapshots(snapshotId; repositoryStateId; foundIndices; rootBlobs; repositoryData;
 SnapshotsService.useShardGenerations(repositoryMetaVersion); listener);
 } catch (Exception ex) {
 listener.onFailure(new RepositoryException(metadata.name(); "failed to delete snapshot [" + snapshotId + "]"; ex));
 }
 }
 }  /**

 

 [ratemypost]

Related Articles

Back to all articles