Force merge in Elasticsearch

Question

Hi Everyone,

We use Elasticsearch 7.9 and on monthly bases we create new indexes. Generally the last 4-5 indexes(months) receive large number of updates, so we end up with lots of deleted documents. I read that in order to improve the search performance, we have to do force merge. But from other side its been recommended to do the force merge on indexes that don't receive any write.

Here are my questions.

1- Per (https://aws.amazon.com/premiumsupport/knowledge-center/opensearch-deleted-documents/) merge cause blocking for whole cluster, so how can we periodically do the merging? Wont this cause any prolem for end-user?

2- Since in our case the indexes still receive write, is there any work around to eliminate the deleted documents?

Thanks in advance.

Answer

Hi there, please find the answers inline

1) Yes it is a resource intensive operation and it is recommended to trigger it off business hours or low traffic times. It triggers an asynchronous task that will run in the background until the job is done. You might still be able to perform queries on your cluster while force merge is running if resources are available. But most often all requests are blocked during that time.

2) As of now there is no work around to eliminate the deleted documents. Segment merges remove this deleted documents automatically when it runs. But to avoid cluster running segment merges when indexes are receiving writes, you should execute force merges periodically when indexes don't receive any write.

Hope this helps!

Force merge in Elasticsearch

Relevant content