Cloudwatch alarms created via CLI (CWAgent) fail "Unchecked: Initial alarm"

0

I am having issues with the cloudwatch API/CLI. Below I have included 3 API/CLI calls for "put-metric-alarm" (with describe-alarms to show values) with unit=Percent and unit=None and also no usage of unit. In the Wizard/Console, I don't select unit. Through the wizard, the alarm works, via API/CLI - all three ways, the alarm never passes "INSUFFICIENT_DATA" - "Unchecked: Initial alarm creation".

What is the issue and how can I fix it?

Thank you.
NOTE: "XXXXX" is used to obfuscate identifying items

aws cloudwatch put-metric-alarm --region us-west-2 --alarm-name "HighMemPercentUsedAlarm EC2 - API - unit:Percent" --alarm-description "Alarm when Memory Percent Usage exceeds 95 percent" --metric-name mem_used_percent --namespace CWAgent --datapoints-to-alarm 2 --statistic Average --period 900 --threshold 95.0 --comparison-operator GreaterThanThreshold --dimensions "Name=InstanceId,Value=i-XXXXX" --evaluation-periods 2 --alarm-actions arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic --treat-missing-data "missing" --unit Percent
aws cloudwatch describe-alarms --alarm-names "HighMemPercentUsedAlarm EC2 - API - unit:Percent"

aws cloudwatch describe-alarms --alarm-names "HighMemPercentUsedAlarm EC2 - API - unit:Percent"
{
"MetricAlarms": [
{
"Dimensions": [
{
"Name": "InstanceId",
"Value": "i-XXXXX"
}
],
"Namespace": "CWAgent",
"DatapointsToAlarm": 2,
"ActionsEnabled": true,
"MetricName": "mem_used_percent",
"EvaluationPeriods": 2,
"StateValue": "INSUFFICIENT_DATA",
"StateUpdatedTimestamp": "2020-06-05T16:44:16.620Z",
"AlarmConfigurationUpdatedTimestamp": "2020-06-05T16:44:16.620Z",
"AlarmActions": [
"arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic"
],
"InsufficientDataActions": [],
"AlarmArn": "arn:aws:cloudwatch:us-west-2:XXXXX:alarm:HighMemPercentUsedAlarm EC2 - API - unit:Percent",
"Threshold": 95.0,
"StateReason": "Unchecked: Initial alarm creation",
"OKActions": [],
"AlarmDescription": "Alarm when Memory Percent Usage exceeds 95 percent",
"Period": 900,
"ComparisonOperator": "GreaterThanThreshold",
"AlarmName": "HighMemPercentUsedAlarm EC2 - API - unit:Percent",
"Statistic": "Average",
"TreatMissingData": "missing",
"Unit": "Percent"
}
]
}

================================================================================================
aws cloudwatch put-metric-alarm --region us-west-2 --alarm-name "HighMemPercentUsedAlarm EC2 - API - unit:None" --alarm-description "Alarm when Memory Percent Usage exceeds 95 percent" --metric-name mem_used_percent --namespace CWAgent --datapoints-to-alarm 2 --statistic Average --period 900 --threshold 95.0 --comparison-operator GreaterThanThreshold --dimensions "Name=InstanceId,Value=i-XXXXX" --evaluation-periods 2 --alarm-actions arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic --treat-missing-data "missing" --unit None
aws cloudwatch describe-alarms --alarm-names "HighMemPercentUsedAlarm EC2 - API - unit:None"

aws cloudwatch describe-alarms --alarm-names "HighMemPercentUsedAlarm EC2 - API - unit:None"
{
"MetricAlarms": [
{
"Dimensions": [
{
"Name": "InstanceId",
"Value": "i-XXXXX"
}
],
"Namespace": "CWAgent",
"DatapointsToAlarm": 2,
"ActionsEnabled": true,
"MetricName": "mem_used_percent",
"EvaluationPeriods": 2,
"StateValue": "INSUFFICIENT_DATA",
"StateUpdatedTimestamp": "2020-06-05T16:44:27.434Z",
"AlarmConfigurationUpdatedTimestamp": "2020-06-05T16:44:27.434Z",
"AlarmActions": [
"arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic"
],
"InsufficientDataActions": [],
"AlarmArn": "arn:aws:cloudwatch:us-west-2:XXXXX:alarm:HighMemPercentUsedAlarm EC2 - API - unit:None",
"Threshold": 95.0,
"StateReason": "Unchecked: Initial alarm creation",
"OKActions": [],
"AlarmDescription": "Alarm when Memory Percent Usage exceeds 95 percent",
"Period": 900,
"ComparisonOperator": "GreaterThanThreshold",
"AlarmName": "HighMemPercentUsedAlarm EC2 - API - unit:None",
"Statistic": "Average",
"TreatMissingData": "missing",
"Unit": "None"
}
]
}

================================================================================================
aws cloudwatch put-metric-alarm --region us-west-2 --alarm-name "HighMemPercentUsedAlarm EC2 - API" --alarm-description "Alarm when Memory Percent Usage exceeds 95 percent" --metric-name mem_used_percent --namespace CWAgent --datapoints-to-alarm 2 --statistic Average --period 900 --threshold 95.0 --comparison-operator GreaterThanThreshold --dimensions "Name=InstanceId,Value=i-XXXXX" --evaluation-periods 2 --alarm-actions arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic --treat-missing-data "missing"

aws cloudwatch describe-alarms --alarm-names "HighMemPercentUsedAlarm EC2 - API"
{
"MetricAlarms": [
{
"EvaluationPeriods": 2,
"TreatMissingData": "missing",
"AlarmArn": "arn:aws:cloudwatch:us-west-2:XXXXX:alarm:HighMemPercentUsedAlarm EC2 - API",
"StateUpdatedTimestamp": "2020-06-05T16:44:34.574Z",
"AlarmConfigurationUpdatedTimestamp": "2020-06-05T16:44:34.574Z",
"ComparisonOperator": "GreaterThanThreshold",
"AlarmActions": [
"arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic"
],
"AlarmDescription": "Alarm when Memory Percent Usage exceeds 95 percent",
"Namespace": "CWAgent",
"Period": 900,
"StateValue": "INSUFFICIENT_DATA",
"Threshold": 95.0,
"AlarmName": "HighMemPercentUsedAlarm EC2 - API",
"Dimensions": [
{
"Name": "InstanceId",
"Value": "i-XXXXX"
}
],
"DatapointsToAlarm": 2,
"Statistic": "Average",
"StateReason": "Unchecked: Initial alarm creation",
"InsufficientDataActions": [],
"OKActions": [],
"ActionsEnabled": true,
"MetricName": "mem_used_percent"
}
]
}

================================================================================================
Same via the "Wizard/Console"
aws cloudwatch describe-alarms --alarm-names "HighMemPercentUsedAlarm EC2 - Wizard"
{
"MetricAlarms": [
{
"Dimensions": [
{
"Name": "InstanceId",
"Value": "i-XXXXX"
},
{
"Name": "ImageId",
"Value": "ami-XXXXX"
},
{
"Name": "InstanceType",
"Value": "m5.2xlarge"
}
],
"Namespace": "CWAgent",
"DatapointsToAlarm": 2,
"ActionsEnabled": true,
"MetricName": "mem_used_percent",
"EvaluationPeriods": 2,
"StateValue": "OK",
"StateUpdatedTimestamp": "2020-06-05T16:43:06.797Z",
"AlarmConfigurationUpdatedTimestamp": "2020-06-05T16:41:52.619Z",
"AlarmActions": [
"arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic"
],
"InsufficientDataActions": [],
"AlarmArn": "arn:aws:cloudwatch:us-west-2:XXXXX:alarm:HighMemPercentUsedAlarm EC2 - Wizard",
"StateReasonData": "{"version":"1.0","queryDate":"2020-06-05T16:43:06.795_0000","startDate":"2020-06-05T16:13:00.000_0000","statistic":"Average","period":900,"recentDatapoints":http://14.36740102848503,15.074661280217848,"threshold":95.0}",
"Threshold": 95.0,
"StateReason": "Threshold Crossed: 2 out of the last 2 datapoints 15.074661280217848 (05/06/20 16:28:00), 14.36740102848503 (05/06/20 16:13:00) were not greater than the threshold (95.0) (minimum 1 datapoint for ALARM -> OK transition).",
"OKActions": [],
"AlarmDescription": "Alarm when Memory Percent Usage exceeds 95 percent",
"Period": 900,
"ComparisonOperator": "GreaterThanThreshold",
"AlarmName": "HighMemPercentUsedAlarm EC2 - Wizard",
"Statistic": "Average",
"TreatMissingData": "missing"
}
]
}

asked 4 years ago736 views
2 Answers
0

Hi Mark,
Metric uniqueness is a combination of metric name and its dimensions.

Notice the alarm created by the wizard includes all of the dimensions for the metric, and the alarm created by the CLI declares only the InstanceId dimension. Most likely, you've configured metric agent to add ec2 dimensions (ImageId, InstanceId, InstanceType, AutoScalingGroupName), therefore mem_used_percent is not a per-instance metric.

You must include all the dimensions for a metric when you create an alarm.

Best,
Kevin

AWS
answered 4 years ago
0

Hi Kevin,
Thanks for answering. I did try with using all the other dimensions (except AutoScalingGroupName) and it too failed to create. With regards to AutoScalingGroupName, I don't see it in any of the results and we aren't using them explicitly.

However, with your "hint" and in fiddling around with the dimensions syntax, it seems I worked it out.
In the documentation "example", they have '--dimensions "Name=InstanceId,Value=i-12345678"' with double quotes around the actual dimensions. At the bottom of the documentation, for multiple dimensions, they have '--dimensions Name=key1,Value=value1 Name=key2,Value=value2' with no double quotes. That was the difference. And I didn't need the --unit.

Thanks for the help,
Mark

example:
aws cloudwatch put-metric-alarm --region us-west-2 --alarm-name "HighMemPercentUsedAlarm EC2 - CLI" --alarm-description "Alarm when Memory Percent Usage exceeds 'n' percent" --metric-name mem_used_percent --namespace CWAgent --datapoints-to-alarm 2 --statistic Average --period 900 --threshold 95.0 --comparison-operator GreaterThanThreshold --dimensions Name=InstanceId,Value=i-XXXXX Name=ImageId,Value=ami-XXXXX Name=InstanceType,Value=m5.2xlarge --evaluation-periods 2 --alarm-actions arn:aws:sns:us-west-2:XXXXX:Default_CloudWatch_Alarms_Topic --treat-missing-data "missing"

Edited by: markwilhelm on Jun 9, 2020 7:22 AM

answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions