Briefly, this error occurs when Elasticsearch is trying to delete data that has exceeded its retention period. This is a routine process and not necessarily an error, but it can cause issues if the system is unable to delete the data due to insufficient permissions or if the data is locked. To resolve this issue, you can ensure that Elasticsearch has the necessary permissions to delete data. If the data is locked, you may need to unlock it. Additionally, you can adjust the data retention settings if you want to keep the data for a longer period.
To understand why your data has been deleted and to control this action in the future, you should run the Elasticsearch Error Check-Up. 29% of people who ran the Check-Up had this issue and the tool will help you configure your system to ensure optimal settings and performance for your use case.
This guide will help you check for common problems that cause the log “Deleting expired data” to appear. It’s important to understand the issues related to the log, so to get started, read the general overview on common issues and tips related to the Elasticsearch concepts: delete and plugin.
What this error means
This log message is an INFO message letting you know that all job results, model snapshots and forecast data that have exceeded their retention days period have been deleted.
Explanation
The “delete expired data” API comes as part of the X-Pack and is used to delete expired and unused machine learning data.
DELETE _ml/_delete_expired_data
The response is:
{
"deleted": true
}
When the “delete expired data” API is hit, the following log will be generated:
[INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] Deleting expired data [INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] Completed deletion of expired ML data
Log Context
Log “Deleting expired data” classname is TransportDeleteExpiredDataAction.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
protected void doExecute(
Task task;
DeleteExpiredDataAction.Request request;
ActionListener listener
) {
logger.info("Deleting expired data");
Instant timeoutTime = Instant.now(clock)
.plus(
request.getTimeout() == null
? Duration.ofMillis(MlDataRemover.DEFAULT_MAX_DURATION.getMillis())
: Duration.ofMillis(request.getTimeout().millis())
[ratemypost]
