How can I manage and optimize CloudWatch custom metrics to control costs?

Short Description

CloudWatch charges for custom metrics, including from CloudWatch agent or Container Insights. For information on the pricing structure, see Amazon CloudWatch Pricing.

Note: Basic monitoring metrics for AWS services under AWS namespaces are included at no additional charge. However, if you turn on detailed monitoring for services such as Amazon Elastic Compute Cloud, (Amazon EC2), then you incur additional costs.

Resolution

To optimize CloudWatch custom metrics, you must first identify the sources of those metrics. 

Identify sources of custom metrics

To identify the source of your custom metrics, complete the following steps:

1. Go to AWS Cost Explorer
2. In the Report Parameters pane, select Group by. For Dimension, select API operation.
3. Under Filters, for Service, select CloudWatch. For Usage type, select CW:MetricMonitorUsage (Metrics).

The API operation shows the source of the following custom metrics:

• MetricStorage: Custom metrics pushed from your applications.
• MetricStorage:AWS/{Service}: Detailed monitoring turned on for specific AWS services.
• MetricStorage:AWS/Logs-EMF: Logs embedded metrics from Container Insights.
• MetricStorage:AWS/CloudWatchLogs: CloudWatch Logs metric filters.

How to optimize custom metric usage

Review and reduce Container Insights metrics:

If you use Container Insights, then these metrics can contribute significantly to costs. Contact your Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS) team to review and potentially reduce the metrics used.

Optimize custom application metrics:

• Batch multiple data points locally and send aggregated statistics at lower frequencies.
• Use Amazon CloudWatch Embedded Metric Format (EMF) to batch multiple metrics in a single request. CloudWatch EMF is more cost-effective than individual PutMetricData calls.
• Implement custom metrics filtering to only send essential metrics and set appropriate sampling rates.

Review detailed monitoring for AWS services:

• Turn off detailed monitoring for services where it's not required.

Optimize CloudWatch agent configuration:

• Adjust your collection intervals.
• Use selective metric collection.
• Configure appropriate aggregation intervals.

Batch API requests:

• When you use PutMetricDataCommand, batch multiple metrics into single calls to reduce API request costs.

Optimize metric retrieval:

• Review and optimize GetMetricData API usage.
• Consider caching frequently accessed metric data.
• Reduce the frequency of metric retrievals where possible.

Best practices for cost management

Regularly audit your metrics usage:

• Review and remove unnecessary metrics.
• Adjust sampling rates and aggregation intervals as needed.

Use appropriate metric resolution:

• Standard resolution (60-second intervals) is sufficient for most use cases and is more cost-effective than high resolution.

Use CloudWatch Logs for cost-effective logging:

• Use CloudWatch Logs metric filters to generate metrics from log data. This can be more cost-effective than custom metrics in some cases.

Monitor CloudWatch costs:

• Set up Cost Explorer to track your CloudWatch spending patterns.
• Create budget alerts to notify you of unexpected increases in CloudWatch costs.

Consider using AWS Cost Categories:

• Organize your CloudWatch costs by different projects, teams, or environments for better cost allocation and management.

Related information

Amazon CloudWatch pricing

Metrics in Amazon CloudWatch

Analyzing, optimizing, and reducing CloudWatch costs

Embedding metrics within logs

Collect metrics, logs, and traces using the CloudWatch agent