Bedrock OpenSearch Serverless collection unpredictable Search OCU costs
I have 12 Bedrock knowledge bases that translate to 12 collections storing a total of 9.2 GB of data in vector databases and I'm getting some unpredictable Search OCU allocations.
I'm using test configuration, no-redundancy. Also all of them use the same AWS managed KMS key. According to the documentation this would use a minimum of 0.5 OCU for indexing and 0.5 OCU for search. And the OCUs can scale up if required, for example if needing more RAM.
Now to the unpredictable results. The IndexingOCUs behave quite predictable, they are at 0.5 most of the time and they may spike at some specific times. But the SearchOCUs have gone from 1 -> 4 -> 3 -> 0 -> 3 again. And most times the changes don't correlate with the activity. These collections are being used very infrequently, but still AWS keeps 3 OCUs most of the time, when 2 should be enough to keep all the collections in RAM.
Here's a dashboard monitoring the relevant metrics:
The question is how to make sense of these changes and why is it even using 3 Search OCUs most of the time when 2 is more than enough?
- Newest
- Most votes
- Most comments
When you create OpenSearch Serverless collections, there are some important scaling behaviors to understand regarding OCU allocation.
For OpenSearch Serverless, when you create your first collection, it instantiates a total of four OCUs (two for indexing and two for search). These OCUs always exist, even when there's no indexing or search activity. All subsequent collections can share these OCUs, except for collections with unique AWS KMS keys, which would instantiate their own set of four OCUs.
Since you mentioned all your collections use the same AWS managed KMS key, they should be sharing the same pool of OCUs. The system will automatically scale out and add additional OCUs as your indexing and search usage grows. When traffic decreases, capacity scales back down to the minimum number of OCUs required for your data size, but at most, it will scale down to 2 OCUs for indexing and 2 OCUs for search.
The unpredictable behavior you're seeing with SearchOCUs fluctuating between 0, 1, 3, and 4 is unusual, especially since the minimum should be 2 OCUs for search. The fact that you're seeing it drop to 0 at times suggests there might be an issue with the reporting or a temporary service behavior.
The lack of correlation between OCU allocation and actual search activity is likely because OpenSearch Serverless scales based not just on request volume but also on data size and memory requirements. Even with infrequent usage, if your vector data requires more memory to be readily available for efficient searching, the system may maintain higher OCU levels.
To make sense of these changes:
- Monitor both account-level and collection-level metrics like SearchOCU and SearchRequestLatency
- Look for patterns in when the OCU count changes - it might correlate with behind-the-scenes maintenance or optimization activities
- Consider that the system might be optimizing for performance rather than cost, keeping more data in memory for faster responses
If the OCU usage consistently remains higher than expected, you might want to contact AWS Support to investigate if there's an underlying issue with the service or your specific setup.
Sources
Amazon KnowledgeBase with OpenSearchServerless - minimum price | AWS re:Post
Amazon OpenSearch Serverless monitoring: A CloudWatch setup guide | AWS Big Data Blog
OpenSearch Serverless OCU usage too high | AWS re:Post
answered 9 months ago
Hello,
I understand your concern regarding the unpredictable SearchOCU behavior you're experiencing with your Bedrock Vector Store collections.
The SearchOCU allocation isn't solely based on storage size - it's actually determined by several key factors including the number of shards, CPU utilization, disk utilization, and memory utilization. When you observe fluctuations in SearchOCU (like the changes from 1 -> 4 -> 3 -> 0 -> 3 OCUs), this is the system dynamically responding to these various factors, even when the actual usage appears low.
The system calculates required OCUs based on how your indices are distributed across shards, and how these shards are allocated to workers. Each worker has a maximum capacity for shards, and when the utilization approaches certain thresholds, the system automatically scales to maintain performance. This explains why you might see OCU increases even during periods of low activity - the system is ensuring enough capacity is available to maintain performance standards.
To optimize your setup and potentially reduce unnecessary OCU allocations, I recommend considering these approaches:
First, analyze your indices and consider merging smaller ones. This will reduce the total number of shards and may prevent unnecessary scaling events.
You might also want to review your sharding strategy. While this would require data re-ingestion, it could lead to better resource utilization. However, please note that reducing shards could impact request latencies, so this needs careful consideration.
I hope I was able to offer some clarity over the current situation and answered your questions in a satisfactory manner. However, if you have any other question or concern then please let me know and I will be more than happy to help you.
Thank you!
answered 9 months ago
I still don't understand how it needs 3 OCUs (18 GB) to fit 9 GB of data (no redundancy), even if it needs it all of it in RAM, 2 OCUs should be more than enough. Am I looking at the wrong metrics?
Relevant content
asked 2 years ago
asked 2 years ago
Why would it be 3 OCUs if 2 is enough to fit the entire databases in RAM and there's almost no search activity?