Cloudwatch Fees from Metric Storage Issue

0

I've just been doing very basic cloudwatch queries over the past several years using the web interface to monitor things like ram utilization of my server.

I'm noticing cloudwatch costs and metric storage costs, which I think is related to cloudwatch, have been increasing and I would like to reduce them.

In my research it seems I should be able to set metric retention using log groups. However, I don't seem to have any log groups. I created a temporary one and used it with insights, but obviously insights didn't find anything since that isn't really the log group I'm using. Also I shouldn't be charged for cloudwatch metrics based on how few I'm using, but I am so I suspect something is running cloudwatch metrics, but I can't figure out what is doing that.

I would be fine completely removing all my cloudwatch data and restarting fresh with cloudwatch, but I can't even figure out a way to do that.

Thanks for any suggestions! I've done a few hours of searching and just can't seem to figure out how this should work so I'd really appreciate any help you can provide.

Enbar
asked 4 years ago3185 views
6 Answers
0

Hi Enbar,

When it comes to CloudWatch Metrics, you will be charged for two things:

Custom metrics (Any metrics with a namespace other than AWS namespaces) that you are pushing to CloudWatch Service using either:

#-- > CLI\API
#-- > CloudWatch Agent

below API call requests:

#-- > GetMetricData
#-- > GetInsightRuleReport #-- > GetMetricWidgetImage
#-- > GetMetricStatistics
#-- > ListMetrics
#-- > PutMetricData
#-- > GetDashboard
#-- > ListDashboards
#-- > PutDashboard
#-- > DeleteDashboards

As for metric retention, it is only a way where AWS CloudWatch service aggregate your metrics raw data points for long-term storage. CloudWatch retains metric data as follows:
#. Data points with a period of less than 60 seconds are available for 3 hours. These data points are high-resolution custom metrics.
#. Data points with a period of 60 seconds (1 minute) are available for 15 days
#. Data points with a period of 300 seconds (5 minute) are available for 63 days
#. Data points with a period of 3600 seconds (1 hour) are available for 455 days (15 months)

Metrics cannot be deleted, but they automatically expire after 15 months if no new data is published to them in the last 2 weeks.

Data points older than 15 months expire on a rolling basis; as new data points come in, data older than 15 months is dropped.

I suggest to look at your CloudWatch custom metrics (Any metrics that is not in AWS namespaces) and try to analyze which application\Scrip\Code is publishing those metrics and how frequent.

The AWS namespaces typically use the following naming convention: AWS/service. For example, Amazon EC2 uses the AWS/EC2 namespace.

Shuai
answered 4 years ago
0

ShuaibA-AWS, thank for the reply.

I think something is misconfigured with my could watch because I have tried all the basic stuff over the past several months and nothing seems to help. But, it is possible I am missing something so hopefully you can help with this a little more.

I've looked into the detailed cost and you are right that those fees are related to custom metrics.
For example below are the important columns from my cost and usage spread sheet.

AmazonCloudWatch -- CW:MetricMonitorUsage -- MetricStorage -- $0.30 per metric-month for the first 10,000 metrics -- Custom Metrics

At https://console.aws.amazon.com/cloudwatch I have tried the following things for over a 1 month period:

  • Removed all custom dashboards
  • Made sure none of the Metrics were graphed under Metrics>Graphed metrics.
  • Made sure I had no logs or events

I do have 30 custom metrics under CWAgent. I can't even remember how I got all those added years ago. A few are useful like CPU utilization and ram utilization. However, most I've never used other than in a few experiments a few years ago. My understanding is they are supposed to be removed after 2 weeks of not being uses, but they have never been removed. This combine with the charges I see makes me think something is using these metrics, but I don't know how to stop using them. Is there some other way I can stop using them?

I have also https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html to look directly at my amazon-cloudwatch-agent.json file and make sure the configuration makes sense. In the past I even modified it to have no metrics in it when doing the steps I mentioned didn't help, but right now it has a few that match what my custom dashboard is.

The documents for cloudwatch say that 50 metrics are free. It isn't completely clear if that includes any custom metrics, but I think it does. Even if it doesn't in the past I've tried to use no custom metrics and I was still getting charged for them.

Do you have a suggestion on how to make sure I'm not using an custom metrics?

Thanks again for your help!

Enbar
answered 4 years ago
0

Thanks for your reply. It has helped refresh my memory that I setup a metricAgent on my instance several years ago.

Is the best way to modify my metric agent to change the amazon-cloudwatch-agent.json file? I have put my current file at the end of the document. I can certainly increase the interval and remove a few things from this file. I see 9 metrics with 30 second intervals.

I'm a little confused how things are priced for this, but I am now guessing this is why I am getting charged. I'd be fine paying the $0.30 for 10K metrics, but I'm not sure how those are counted. If I made my metric just report back every hour instead of every 30 seconds then with my current number of metrics I would expect to stay under 10K metrics for the month so just $0.30. Is that right, or am I counting this incorrectly?

I did try modifying amazon-cloudwatch-agent.json in the past and it didn't seem to help, but I'm willing to try that again the experts here think that should help.

{
"metrics": {
"append_dimensions": {
"AutoScalingGroupName": "${aws:AutoScalingGroupName}",
"ImageId": "${aws:ImageId}",
"InstanceId": "${aws:InstanceId}",
"InstanceType": "${aws:InstanceType}"
},
"metrics_collected": {
"cpu": {
"measurement": [
"cpu_usage_idle",
"cpu_usage_iowait",
"cpu_usage_user",
"cpu_usage_system"
],
"metrics_collection_interval": 30,
"resources": [
""
],
"totalcpu": false
},
"disk": {
"measurement": [
"used_percent",
"inodes_free"
],
"metrics_collection_interval": 30,
"resources": [
"
"
]
},
"diskio": {
"measurement": [
"io_time"
],
"metrics_collection_interval": 30,
"resources": [
"*"
]
},
"mem": {
"measurement": [
"mem_used_percent"
],
"metrics_collection_interval": 30
},
"swap": {
"measurement": [
"swap_used_percent"
],
"metrics_collection_interval": 30
}
}
}
}

Enbar
answered 4 years ago
0

Hello,

As a CloudWatch user monitoring servers in the AWS console, I will guess you are monitoring EC2 instances.

You are likely to incur charges in two areas monitoring EC2: Custom Metrics and Detail Monitoring metrics.

Basic EC2 metrics are sent to CloudWatch on a 5-minute interval for free.

If you are monitoring RAM on your server, then you likely have the EC2 CloudWatch Agent installed and running on your EC2 instances. CloudWatch Agent sends additional EC2 instance metrics to CloudWatch as Custom Metrics, for example mem_total and mem_free.

Also, when you configure your EC2 instances, you can choose to enable Detail Monitoring to emit EC2 metrics at a faster 1-minute interval.

Here are some resources to help you understand which metrics are available and their pricing:

CloudWatch metrics for your instances: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/viewing_metrics_with_cloudwatch.html

Metrics Collected by the CloudWatch Agent: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/metrics-collected-by-CloudWatch-agent.html

Enable or disable Detailed Monitoring for your instances: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html

CloudWatch Pricing Page: https://aws.amazon.com/cloudwatch/pricing/

Best,
Kevin

AWS
answered 4 years ago
0

Hello,

Custom Metrics are charged $0.30 per metric per month. They are pro-rated by the hour and you are only charged for a metric during an hour in which that metric receives a datapoint. As long as a Custom Metric receives a datapoint at-least once per hour, it will be $0.30 per month.

CloudWatch Agent is a software service you installed on your EC2. While it is running, it will send Custom Metrics to CloudWatch and put them in CWAgent.

If you no-longer wish to receive Custom Metrics from your EC2, your best course is to stop and/or uninstall the CloudWatch Agent running on your EC2.

Instructions for stopping CloudWatch Agent are here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/troubleshooting-CloudWatch-Agent.html#CloudWatch-Agent-troubleshooting-stopping-restarting

Best,
Kevin

AWS
answered 4 years ago
0

I am pretty convinced it was this cloudwatch agent and I have uninstalled that to verify it solves my issue. Thanks for all the help!

Enbar
answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions