- Newest
- Most votes
- Most comments
Hello
Unfortunately, storing 60 TB of documents in a single OpenSearch Serverless collection isn't feasible. The maximum data limit per collection is 1 TiB (1,024 GB).
vCPU Allocation per OCU:
Implicit Allocation: OpenSearch Serverless doesn't explicitly specify vCPU allocation per OCU. The OCU itself represents a combination of memory (6 GiB) and an associated virtual CPU (vCPU) that scales automatically based on your workload.
Focus on OCUs: For optimal configuration, focus on adjusting the number of OCUs allocated to your collection. This influences both compute power (vCPUs) and available memory for indexing and searching.
Troubleshooting Tips:
Monitor Metrics: Utilize OpenSearch Serverless monitoring tools to track key metrics like indexing and search throughput, resource utilization (CPU, memory), and latency. Identify bottlenecks or potential issues.
Optimize Indexing and Search Operations: Analyze queries and index configurations to optimize performance. Consider indexing strategies, filters, aggregations, and data compression techniques.
Scaling: OpenSearch Serverless automatically scales compute resources (OCUs) based on workload. However, you can also manually adjust OCUs to meet specific requirements.
AOSS supports 10 TB per time-series collection and 1 TB per search or vector collection.
Each OCU represents 6 GB memory and certain number of vCPUs - AFAIK we haven't published how many vCPUs we allocate per OCU. But, from billing standpoint, we charge only based on OCUs consumed.
Minimum no of replicas for search is 2 (one per AZ) but with replica scaling feature, https://aws.amazon.com/blogs/big-data/achieve-higher-query-throughput-auto-scaling-in-amazon-opensearch-serverless-now-supports-shard-replica-scaling/, service can automatically increase the shard replica to handle the increased query throughput.
It is just very hard to believe that AOSS supports ONLY 1 TB per search collection.