Maximum limit for cloudwatch variable using search

Question

I have a cloudwatch dashboard where there is a variable populated by SEARCH.  (In my particular case, I want to show the distinct values of Node for which there is metric value.  I have this metric for multiple Nodes in multiple ProcessPaths.

```
Values.fromSearch(`SEARCH('{DiceDynamicCostUpdateSystem, Context, Operation, Program, ResourceKey, Node, ProcessPath} MetricName="ACCUMULATED_BROKEN_UNIT" (ProcessPath="BIN_PALLET" OR ProcessPath="BIN_LIBRARY" OR ProcessPath="BIN_LIBRARY_DEEP")', 'Average')`, "Node"), 
```

The dropdown does not show all the choices it should.  I am quite confident that I am hitting a limit on the maximum number of results.  I found this for myself by entering that same search into the CloudWatch metrics console. It showed an error message about `` Maximum number of allowed metrics exceeded.``  Adding more terms to the query reduced the number of metrics found.

I edited my dashboard the same way, and now see all the distinct nodes.

My asks are

1. Where is this limit actually documented?
2. Please consider documenting it someplace more easily found

I checked
*   [Values.fromSearch](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cloudwatch.Values.html)
* The page for [search expression syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) 
*  The page for [service quotas](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html)

None of them mention a limit

https://repost.aws/questions/QUG6oH-yt-RDaHq2QRhtdBKQ/cloudwatch-metrics-getmetricdata-error-maximum-number-of-allowed-metrics-exceeded-your-search-may-have-been-limited seems relevant, but it's not obvious to me that SEARCH is the same as GetMetricData.

re:Post Agent · Accepted Answer

You're correct that there is a limit on the number of metrics that can be processed and returned by CloudWatch Metrics Insights queries, which includes SEARCH expressions. This limit is indeed documented, though it may not be immediately obvious where to find it.

The limit you're encountering is that a single query can process no more than 10,000 metrics. If your SEARCH expression matches more than 10,000 metrics, only the first 10,000 that it finds will be processed.

Additionally, there's another relevant limit: a single query can return no more than 500 time series. This means that even if your query processes up to 10,000 metrics, the results will be limited to 500 time series.

These limits apply to CloudWatch Metrics Insights queries, which include SEARCH expressions used in dashboards and other CloudWatch features.

Regarding your specific asks:

1. The limit is documented in the CloudWatch documentation, specifically in the section about CloudWatch Metrics Insights limits.

2. Your suggestion to make this information more easily discoverable is valid. It would indeed be helpful if these limits were more prominently displayed or cross-referenced in related documentation, such as the pages you mentioned (Values.fromSearch, search expression syntax, and service quotas).

To work around these limits in your dashboard, you might consider:

- Narrowing your search criteria to reduce the number of matching metrics
- Using multiple widgets with more specific searches if you need to display more than 500 time series
- If possible, restructuring your metrics to reduce the total number while still capturing the necessary information

It's worth noting that these limits are in place to ensure performance and prevent excessive resource consumption, as processing and returning large numbers of metrics can be computationally expensive.  
**Sources**  
[Metrics Insights limits - Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-metrics-insights-limits.html)  
[CloudWatch observability solutions - Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Monitoring-Solutions.html)  
[Using the metric query option to query CloudWatch Metrics Insights data - Amazon Managed Grafana](https://docs.aws.amazon.com/grafana/latest/userguide/CloudWatch-using-the-metric-query.html)

Maximum limit for cloudwatch variable using search

Add your answer

Relevant content