AWS OpenSearch Heap Size increase

The heap size limit for Amazon OpenSearch Service is indeed 32 GiB, and this is a hard limit that cannot be increased through a service quota request. This limit is in place regardless of the total memory available on your nodes.

However, there are a few strategies you can consider to address your out-of-memory issues during reindexing:

1. Auto-Tune: Amazon OpenSearch Service offers an Auto-Tune feature that can help optimize your cluster's performance. It can suggest and implement changes to various settings, including JVM heap size (up to the 32 GiB limit) and JVM young generation settings. These optimizations can potentially help with memory management during intensive operations like reindexing.

2. Queue Size Adjustment: Auto-Tune can also adjust queue sizes. By default, the search queue size is 1000 and the write queue size is 10000. Auto-Tune can scale these queues if additional heap is available, which might help manage the load during reindexing.

3. Scaling: If you're consistently running into memory issues, you might need to consider scaling your cluster. This could involve adding more nodes or using larger instance types to distribute the workload.

4. Optimize Reindexing Process: Review your reindexing process to see if it can be optimized. This might involve breaking the reindexing into smaller batches or adjusting your indexing settings.

5. Monitor and Adjust JVM Settings: While you can't exceed the 32 GiB heap limit, you can monitor your cluster's performance and adjust the percentage of RAM allocated to the JVM heap (up to the 32 GiB limit). By default, OpenSearch Service uses 50% of an instance's RAM for the JVM heap.

Remember, increasing the heap size isn't always the best solution for performance issues. Sometimes, other optimizations or architectural changes can be more effective in addressing memory-related problems.
Sources
Amazon OpenSearch Service quotas - Amazon OpenSearch Service
Auto-Tune for Amazon OpenSearch Service - Amazon OpenSearch Service