Index vs Search resource separation on Managed OpenSearch cluster on AWS

Question

Hey,

Is there a way to allocate separate resources (in the form of nodes/threads etc.) for `indexing` vs `searching` on a Managed OpenSearch cluster on AWS?

It seems like this is possible in OpenSearch as we can separately allocate `indexing` and `search` capacity OCUs on OpenSearch Serverless.

Our issue is that we have heavy indexing traffic when we run certain jobs. This currently interferes with the latency of search requests. We are hoping to solve this by separating out the resources.

Appreciate the help!
Thanks

Vinayaka-Gangadhar · Answer

Hello,

As you rightly pointed out OpenSearch serverless allows you to segregate the Search and Indexing workloads. But in OpenSearch service you can't allocate the resources for Search and Indexing requests.

Please follow below guide to improve indexing and search requests:

1. https://repost.aws/knowledge-center/opensearch-indexing-performance
2. https://repost.aws/knowledge-center/opensearch-latency-spikes

If you're running indexing heavy operational analytics workloads such as log analytics, observability, or security analytics, you can benefit from the improved performance and compute efficiency of [OR1 instances](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/or1.html). In addition, the automatic data recovery offered by OR1 instances improves the overall reliability of your domain. For more info, please refer the [blog](https://aws.amazon.com/blogs/big-data/improve-your-amazon-opensearch-service-performance-with-opensearch-optimized-instances/)

Giovanni Lauria · Answer

In AWS Managed OpenSearch Service, you can’t directly allocate separate resources for these tasks.

Instead, you can optimize performance by choosing the right instance types and sizes, optimizing shard allocation, defining index mappings and settings, using parallel execution and scaling, and enabling cross-cluster search and replication.

Index vs Search resource separation on Managed OpenSearch cluster on AWS

Relevant content