Magento: Error removing old index from Elastic Search.

Question

Hello,

When I start indexing, the old indexes are not deleted. Normally old indexes are deleted to leave shareds.

Is this a configuration problem?

Error logs:

`[2022-08-10 08:05:36] main.CRITICAL: {"error":{"root_cause":[{"type":"snapshot_in_progress_exception","reason":"Cannot delete indices that are being snapshotted: [[magento2_product_4_v12854/gZH6DirdQ2GBwVUVZoTP4w]]. Try again after snapshot finishes or cancel the currently running snapshot."}],"type":"snapshot_in_progress_exception","reason":"Cannot delete indices that are being snapshotted: [[magento2_product_4_v12854/gZH6DirdQ2GBwVUVZoTP4w]]. Try again after snapshot finishes or cancel the currently running snapshot."},"status":400} {"exception":"[object] (Elasticsearch\Common\Exceptions\BadRequest400Exception(code: 400): {"error":{"root_cause":[{"type":"snapshot_in_progress_exception","reason":"Cannot delete indices that are being snapshotted: [[magento2_product_4_v12854/gZH6DirdQ2GBwVUVZoTP4w]]. Try again after snapshot finishes or cancel the currently running snapshot."}],"type":"snapshot_in_progress_exception","reason":"Cannot delete indices that are being snapshotted: [[magento2_product_4_v12854/gZH6DirdQ2GBwVUVZoTP4w]]. Try again after snapshot finishes or cancel the currently running snapshot."},"status":400} at /home/site/irrijardin-ecommerce/www/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:675)"} []`

Technical context :
* Magento 2 ver. 2.4.3-p1

Thank you

Answer

You'll need to update the ElasticSearch indexer to properly handle this error being returned from the AWS ElasticSearch API.  Unfortunately the indexer doesn't properly handle this.  AWS provides a few API's that you can call to check on the status to determine when the snapshot is finished and then you can proceed to properly delete the indice.  If you don't do this; the indice will stick around and then you'll be stuck with unused indices and therefore will start bleeding shards over time per index.  (note: AWS ES/OpenSearch does a snapshot every hour and the only way to change this is to submit a ticket to the AWS ES team to remove that feature - but it has consequences.  Therefore it's best to update the code to properly handle the error.)

Answer

> When I start indexing, the old indexes are not deleted. Normally old indexes are deleted to leave shareds.
This statement is a bit unclear. What's the expectation about index deletion?

Based on the logs you shared, it's clear that there is a snapshot in progress and you cannot delete an index when a snapshot is in progress. Either wait for completion or cancel the snapshot (as suggested in the log as well)

--Syd

Magento: Error removing old index from Elastic Search.

Relevant content