Questions tagged with Amazon CloudWatch

Hi All, Under cloudwatch matrix FreeStorageSpace for few RDS instances shows more GB compare to actual allocated. E.g. : Actual allocated space : 100 GB FreeStorageSpace : 103 GB while for other RDS instances it was referring properly i.e. FreeStorageSpace less then actual allocated space. Please advise . Regards, Nikhil Shah

We are using SES free version for sending out regular emails to our 130 volunteers and would like to create a log file (or table) with all email related events (sent, delivered, bounced), similar to what we had when sending with mailjet.com smtp relay. Is that possible, i.e. in Cloudwatch, within a free account or would we need a paid account for 1000 plus USD per month?

I work in SM Studio, and I do not understand why CPU and memory usage do not appear in the notebook toolbar. These metrics should be there, at least given this description: https://docs.amazonaws.cn/en_us/sagemaker/latest/dg/notebooks-menu.html When I open a notebook in SM Studio, I see the same toolbar but without CPU and memory usage listed. Moreover, I see 'cluster' before the kernel's name in my toolbar. Has anyone experienced sth similar? I assume an alternative for me would be to use CloudWatch.

Hi Team, I have configured dynamic scaling for elastic cache for 80% memory usage, adding shards. Configurations : 1- 3 primary 2- 2 replica each 3. cache.r6g.large When it's triggering up, AWS elastic cache modifying the cluster is taking up. Sometimes more than an hour for 14GB of instance. The impact is when modifying timeout is happening in the application. Is there any way to avoid this ?

I need to do this to charge my clients for data usage, so for each request I would need to get the request header with the output data transfer response and save it to s3

Hi, I want to ask about how to get CloudWatch metrics from another (shared) account in the organization, when I'm working in the monitoring account. I want to collect CloudWatch metrics from every account in the organization, and then process them in a central place in the monitoring account. Since we create and delete AWS accounts frequently, it isn't feasible to create a fixed dashboard to collect all metrics in the console. I have tried to share CloudWatch metrics from an account to a central monitoring account. Although I can access the metrics in the console, it seems to fail when I call `cloudwatch_client.get_metric_data` in the Python code. I have set up the cross-account metrics sharing config, but the code (using credentials from the monitoring account) is not authorized to perform `cloudwatch:GetMetricData` on the shared account's instances. Is there anything wrong that I have done? If so, how can I use CloudWatch Python client to read metrics of a shared account in my monitoring account.

I need to save this information as downloaded and uploaded headers and data and save to dynamodb or s3

Recently I worked with a project that needs some EC2 servers and RDS for the web applications. After creating all infrastructure resources, I have to create CloudWatch alarms for all EC2 and RDS instances, then I found I have to create hundreds of alarms in CloudWatch which is a nightmare for me! In order to make life easier, I developed a solution for this situation, for details please refer to the following link: https://github.com/jayhebe/cloudwatch_alarm_generator Hope this is helpful and I know there are a lot of limitations and as a beginner of programming, the code is not perfect as well, but I am still working on that iteratively to make it better. Let me know if you have any question :) Br Jay

Looking at the CloudWatch metric AppRunner > Instance metrics > CPU Utilization, I see that my instance is reaching up to 1000 of an unknown unit. Well, the question here is simple - what is the unit for CPU Utilization? Here are my theories so far: - The official [docs page](https://docs.aws.amazon.com/apprunner/latest/dg/monitor-cw.html) on AppRunner metrics is not very helpful; their description is "The average CPU usage during one-minute periods." - The most obvious is percentage of CPU, but that should stop at 100%, so 1000% doesn't make much sense. - According to some other AWS products though, 100% = 1vCPU. This SageMaker [question](https://aws.amazon.com/premiumsupport/knowledge-center/sagemaker-cpu-gpu-utilization-100/) says 400% = 4 vCPUs. So 1000 would be 10 vCPUs. - But my instances are configured to use 1vCPU & 2 GB. So I'd still expect a max of 100, not 1000. Am I allowed to go above the max for brief amounts of time? Will I be charged extra if so? - Another theory is that this might be something to do with less than 1vCPU - like AWS Lambda's CPU throttling which lets you use less than 1vCPU. ie, 1000 = 1vCPU. - Finally, I might be doing something wrong. I'd be happy to share some details of my setup. I'm using maximum for aggregation statistic in Cloud Watch & only 1 instance is running. But to sum up - does anyone know what the unit is for AppRunner's CPU Utilization?

I have a lightsail instance that I want to reboot if cloudwatch alarm on it goes off. I don't see in cloudwatch how to restart lightsail instances. Please let me know if someone knows a way to do this.

Hi All, I have setup the DLM for daily & weekly backup of EBS volumes. I have setup the cloud matrics to get the total count of the Snapshot completed & failed. The sum of data is based on the period filter not the duration i.e., 1) If the period =1day and Duration : 1 months, it shows sum= 1 2) If the period=7days and Duration : 1 Months, it shows sum = 7 3) If the period=30days and Duration : 1 Months, it shows sum = 30 I want that when I select the duration, it should show total count of completed & failed snapshot. Please suggest. Regards, Nikhil Shah

Customer would like to know if there is a CloudWatch metric, or other mechanism, to report on the number of requests against their AWS API Gateway per API Key. The goal is to distinguish how many requests are coming from each API Key.

Hello, I tried using [Inf1 EC2 instance](https://aws.amazon.com/ec2/instance-types/inf1/) for deploying my ML model. I need to monitor the GPU usage of the ML model. I could find the CPU usage in the aws console, but not gpu usage. Already tried: 1. https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-gpu-monitoring-gpumon.html This didn't work. It threw this error ``` (python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ python3 gpumon.py Traceback (most recent call last): File "gpumon.py", line 146, in <module> nvmlInit() File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1450, in nvmlInit nvmlInitWithFlags(0) File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1440, in nvmlInitWithFlags _nvmlCheckReturn(ret) File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 765, in _nvmlCheckReturn raise NVMLError(ret) pynvml.nvml.NVMLError_DriverNotLoaded: Driver Not Loaded ``` Also, `nvidia-smi` didn't work ``` (python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ nvidia-smi NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running. ``` Kindly provide some help to monitor GPU usage.

SOLVED! There was a syntax problem in the runbook, that is not detected when manually remediating. In the content of the remediation doc (that was created using Cloudformation), I used a parameter declaration: parameters: InstanceID: type: 'AWS::EC2::Instance::Id' It should be: parameters: InstanceID: type: String ===================================================================================== I have a remediation runbook that creates Cloudwatch alarms for the metric 'CPUUtilization' for any EC2 instances that have none defined. The runbook is configured as a remediation document for a config rule that checks for the absence of such alarms. When I configure the remediation on the rule as manual, all goes well. When I configure the remediation with the exact same runbook as automatic, the remediation fails with this error (snippet): "StepDetails": [ { "Name": "Initialization", "State": "FAILED", "ErrorMessage": "Invalid Automation document content for Create-CloudWatch-Alarm-EC2-CPUUtilization", "StartTime": "2022-05-09T17:30:02.361000+02:00", "StopTime": "2022-05-09T17:30:02.361000+02:00" } ], This is the remediation configuration for the automatic remediation. The only difference with the manual remediation configuration is obviously the value for key "Automatic" being "false" { "RemediationConfigurations": [ { "ConfigRuleName": "rul-ensure-cloudwatch-alarm-ec2-cpuutilization-exists", "TargetType": "SSM_DOCUMENT", "TargetId": "Create-CloudWatch-Alarm-EC2-CPUUtilization", "TargetVersion": "$DEFAULT", "Parameters": { "AutomationAssumeRole": { "StaticValue": { "Values": [ "arn:aws:iam::123456789012:role/rol_ssm_full_access_to_cloudwatch" ] } }, "ComparisonOperator": { "StaticValue": { "Values": [ "GreaterThanThreshold" ] } }, "InstanceID": { "ResourceValue": { "Value": "RESOURCE_ID" } }, "Period": { "StaticValue": { "Values": [ "300" ] } }, "Statistic": { "StaticValue": { "Values": [ "Average" ] } }, "Threshold": { "StaticValue": { "Values": [ "10" ] } } }, "Automatic": true, "MaximumAutomaticAttempts": 5, "RetryAttemptSeconds": 60, "Arn": "arn:aws:config:eu-west-2:123456789012:remediation-configuration/rul-ensure-cloudwatch-alarm-ec2-cpuutilization-exists/5e3a81a7-fc55-4cbe-ad75-6b27be8da79a" } ] } The error message is rather cryptic, I can't find documentation on possible root causes. Any suggestions would be very welcome! Thanks!

Hi, Actually i have alarms for some errors and performance monitoring with Lambdas, but since last days i have a doubt, its possible get a notification when, for example, a variable or a portion of the code change in lambda? like a Audit monitoring of the code ? Thanks for guidance in advance,

Hi, do you know when might Cross-account cross-Region CloudWatch metrics and alarms creation be available with cloudformation? https://aws.amazon.com/about-aws/whats-new/2021/08/announcing-amazon-cloudwatch-cross-account-alarms/

Hi! I'm trying to do a bit of clean up in CloudWatch, but I run into some problem. I'm still a newbie at Clouwatch so I'm hoping there are some easy answers to my questions. Having said this, I'm quite surprised why what I try to achieve seems to be hard. **1. Is it not possible to edit an alarm name?** When I go into edit mode of an already created alarm, I can edit the Metric name that the alarm is based on, but not the actual alarm name itself (step 3: Add name and description). That input box for the name is greyed out, i.e. readonly. Since all alarms appear in a list and there is no possibility as far as I know to have tree view hierarchy in this list, you do want to group alarm names in order to get some kind of overview and hierachy to find an alarm you are loking for. To me, it seems like the choice has been made under the CloudWatch hood to make the name of the alarm act as the programming identifier, i.e. not editable. Why? **So my question boils down to: is it only possible to edit the alarm name by deleting the alarm and recreating it?** **2. Some metrics seem impossible to delete** When I was new to CloudWatch I accidentally created several different namespaces for the same namespace (mistyped). So I need to clean up and move some metrics from one namespace to another. To move a metric from one namespace to another does not seem possible, so I recreated the metric with the correct namespace name. But now I find that it is impossible (?) to delete the old metric with the old incorrect namespace. If I use the menu option "Metrics -> All metrics" I'll get an overview of all namespaces under the browse tab. But there is no way I can see to delete neither the metric, nor the faulty namespace they reside in in this view. How do I achieve this? If I go into the main LogGroups view, there is a column "Metric Filters" in the grid that shows a link to the metrics attached to each LogGroup. But the metrics with faulty namespaces do not appear anywhere which may be because the old log group they were based on may have been deleted. If so I guess they were deattached from the parent LogGroup and no longer accessible? So why were they not deleted when the LogGroup was deleted (cascading delete) and why is it not possible (?) to delete a parentless LogGroup if so?

Error: Could not load data. The current role cannot perform cloudwatch:GetMetricData. My iam policie: ` "Effect": "Allow", "Action": [ "cloudwatch:GetDashboard", "cloudwatch:GetMetricData", "cloudwatch:DescribeAlarms", "ec2:*" ],`

I am logged in as root user but I don't see where to terminate these services.

I need to transfer files between regions AWS. Bucket South America to Bucket in North Virginia with DataSync. My files in South America region (source), are in S3 Deep Archive, and I'm going to keep them that way, I'm just going to change regions to reduce storage costs. In the AWS documentation, I understood the costs of DataSync, but about the Deep Archive I was in doubt... I have many files with less than 180 days (300TB), I will pay pro rata for taking them out of the bucket before this deadline or for I keep in the same storage class just transfer the files to another location, no? Because I want to make this change but if you have a pro-rata payment, it's a very high cost.

I sending CloudTrail API event for event source ec2 to CW Events. Following is the Event Pattern configured in EventBridge ``` { "detail": { "eventName": ["RunInstances"] "eventSource": ["ec2.amazonaws.com"] }, "detail-type": ["AWS API Call via CloudTrail"] "source": ["aws.ec2"] } ``` I am not getting complete event but the following message ``` { "omitted": true, "originalSize": 109435, "reason": "responseElements too large" } ``` If I check Quotas in CloudTrail user guide, it is mentioned that events can have a max size of 256 KB. The original size mentioned above is less than that, could anyone help me with understanding why I am not getting the complete event? Thanks!

We have a bunch of CloudWatch alarms based on the metric values exceeding thresholds. We can associate extra properties with metrics to store additional identifiers (for example failed Herd workflow ids) inside the metrics via, for example @Prop annotations. But when CloudWatch processes alarm actions and automatically creates tickets, it includes only high level alarm details into the ticket description and no information about metric details. Thus, even if I associate additional ids with metrics, CloudWatch is not able to post these ids into the ticket. Is there a way to extract metric details and add into autocut tickets?

We need to use OKTA as the entry point for users to gain access to CloudWatch dashboards, since we do not want to create new AWS accounts to allow users to use them

We are trying to trigger a lambda when certain AWS Config rules are breached. Currently, we have linked AWS Config with AWS Cloudwatch event bridge and this triggers the lambda on any rule breach. This works great for a single region, but for multiple regions, we will have to set up AWS config rules in that region and an event bridge integration, and a lambda handler in that region as well. This is a lot of additional setups to extend this trigger to multiple regions. So, we set up an AWS Config aggregator which collects AWS config rule breaches from all regions and accounts and consolidates them in one place. If we can set up an AWS Cloudwatch event bridge trigger on the AWS Config Aggregator, it will solve our problem. I could not find anything regarding this. Also, if there is any other way to solve this problem, please let us know. Any input is greatly appreciated. Thanks.

Hi I'm currently experiencing issues when trying to use the "isValidIp" function for filtering logs in CloudWatch ``` fields @timestamp, StatusCode, RequestPath, @@m, isValidIp(Host),!(isValidIp(Host)) | filter !(isValidIp(Host)) ``` I expect the query above to filter all logs where "Host" is not a valid IP address, but nothing is being filtered. At the same time, I render `isValidIp(Host)` and `!(isValidIp(Host))` and see that these can clearly render the correct values for the function. Is there some limitation or is my syntax incorrect? Here is a link to a screenshot of the query and results: https://ibb.co/NSM8qM3

I created a simple Lambda@Edge function like below. ``` 'use strict'; exports.handler = async function(event, context, callback) { const cf = event.Records[0].cf; console.log('Record: ', JSON.stringify(cf, null, 2)); console.log('Context: ', JSON.stringify(context, null, 2)); console.log('Request: ', JSON.stringify(cf.request, null, 2)); callback(null, cf.request); } ``` And I deployed it using AWS CDKv2 `experimental EdgeFunction like below ``` const edgeFunction = new cloudfront.experimental.EdgeFunction(this, 'EdgeFunction', { runtime: Runtime.NODEJS_14_X, handler: 'index.handler', code: Code.fromAsset(path.join(__dirname, '../../../../lambda/ssr2')), }); ``` and also I set it up as edge function for a Distribution ``` const distribution = new Distribution(this, 'Distribution', { defaultBehavior: { origin, cachePolicy: CachePolicy.CACHING_DISABLED, viewerProtocolPolicy: ViewerProtocolPolicy.REDIRECT_TO_HTTPS, edgeLambdas: [ { functionVersion: edgeFunction.currentVersion, eventType: LambdaEdgeEventType.VIEWER_REQUEST, } ] }, ``` But when I tried sending the request to the Distribution, the log didn't show up anything. I checked the permission, the role already has permission ``` Allow: logs:CreateLogGroup Allow: logs:CreateLogStream Allow: logs:PutLogEvents ``` I expect the function write logs to the CloudWatch. What did I miss? **UPDATE 1** Below is the role document, ``` { "sdkResponseMetadata": null, "sdkHttpMetadata": null, "partial": false, "permissionsBoundary": null, "policies": [ { "arn": "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole", "document": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "*" } ] }, "id": "ANPAJNCQGXC425412345", "name": "AWSLambdaBasicExecutionRole", "type": "managed" } ], "resources": { "logs": { "service": { "icon": "data:image/svg+xml;base64,PHN2ZyB2aWV3Qm94PSIwIDAgNjQgNjQiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CiAgPGcgdHJhbnNmb3JtPSJzY2FsZSguOCkiPgogICAgPGRlZnM+CiAgICAgIDxsaW5lYXJHcmFkaWVudCB4MT0iMCUiIHkxPSIxMDAlIiB4Mj0iMTAwJSIgeTI9IjAlIiBpZD0iYSI+CiAgICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iI0IwMDg0RCIgb2Zmc2V0PSIwJSIvPgogICAgICAgIDxzdG9wIHN0b3AtY29sb3I9IiNGRjRGOEIiIG9mZnNldD0iMTAwJSIvPgogICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgPC9kZWZzPgogICAgPGcgZmlsbD0ibm9uZSIgZmlsbC1ydWxlPSJldmVub2RkIj4KICAgICAgPHBhdGggZD0iTTAgMGg4MHY4MEgweiIgZmlsbD0idXJsKCNhKSIvPgogICAgICA8cGF0aCBkPSJNNTUuMDYgNDYuNzc3YzAtMy45MDktMy4yMDItNy4wOS03LjEzOC03LjA5LTMuOTM1IDAtNy4xMzYgMy4xODEtNy4xMzYgNy4wOSAwIDMuOTEgMy4yIDcuMDkgNy4xMzYgNy4wOXM3LjEzNy0zLjE4IDcuMTM3LTcuMDltMi4wMSAwYzAgNS4wMTEtNC4xMDMgOS4wODctOS4xNDcgOS4wODctNS4wNDMgMC05LjE0Ny00LjA3Ni05LjE0Ny05LjA4NyAwLTUuMDEgNC4xMDQtOS4wODYgOS4xNDctOS4wODYgNS4wNDQgMCA5LjE0OCA0LjA3NiA5LjE0OCA5LjA4Nm04LjQ0IDEzLjY5N0w1OC41IDU0LjIwM2ExMy4wMzMgMTMuMDMzIDAgMDEtMS45NDcgMi4xNmw2Ljk5OCA2LjI3YTEuNDc0IDEuNDc0IDAgMDAyLjA2Ni0uMTA3IDEuNDUzIDEuNDUzIDAgMDAtLjEwOC0yLjA1Mm0tMTcuNTg4LTIuODEyYzYuMDQzIDAgMTAuOTU4LTQuODgzIDEwLjk1OC0xMC44ODVzLTQuOTE1LTEwLjg4NC0xMC45NTgtMTAuODg0Yy02LjA0MSAwLTEwLjk1NyA0Ljg4Mi0xMC45NTcgMTAuODg0IDAgNi4wMDIgNC45MTYgMTAuODg1IDEwLjk1NyAxMC44ODVtMTkuMTkgNi4yQTMuNDgzIDMuNDgzIDAgMDE2NC41MjkgNjVhMy40NzUgMy40NzUgMCAwMS0yLjMyMi0uODgzTDU0LjkzMSA1Ny42YTEyLjkzNSAxMi45MzUgMCAwMS03LjAwOSAyLjA2Yy03LjE1IDAtMTIuOTY3LTUuNzc5LTEyLjk2Ny0xMi44ODIgMC03LjEwMiA1LjgxNy0xMi44ODEgMTIuOTY3LTEyLjg4MSA3LjE1MSAwIDEyLjk2OSA1Ljc3OSAxMi45NjkgMTIuODgxIDAgMi4wMzgtLjQ5MiAzLjk2LTEuMzQ0IDUuNjc0bDcuMzA5IDYuNTRhMy40NDQgMy40NDQgMCAwMS4yNTYgNC44NzJNMjEuMjggMjkuMzkzYzAgLjUxOS4wMzIgMS4wMzYuMDk0IDEuNTM2YS45OTQuOTk0IDAgMDEtLjgyMyAxLjEwNmMtMi40NzIuNjM0LTYuNTQgMi41NTMtNi41NCA4LjMxIDAgNC4zNDggMi40MTMgNi43NDggNC40MzkgNy45OTYuNjkxLjQzMyAxLjUxLjY2NCAyLjM3My42NzNsMTIuMTIyLjAxMS0uMDAyIDEuOTk3LTEyLjEzMS0uMDFjLTEuMjQ2LS4wMTQtMi40MjgtLjM1MS0zLjQyOC0uOTc3QzE1LjM3NyA0OC43OTcgMTIgNDUuODkgMTIgNDAuMzQ1YzAtNi42ODMgNC42LTkuMTUzIDcuMy0xMC4wMjYtLjAyLS4zMDctLjAzLS42MTctLjAzLS45MjYgMC01LjQ2IDMuNzI4LTExLjEyMyA4LjY3Mi0xMy4xNzEgNS43ODItMi40MDcgMTEuOTA4LTEuMjE0IDE2LjM4NCAzLjE4OSAxLjM4OCAxLjM2NCAyLjUyOSAzLjAyIDMuNDA0IDQuOTM3YTYuNTA5IDYuNTA5IDAgMDE0LjE1NC0xLjUwMmMzLjAwMiAwIDYuMzgyIDIuMjY0IDYuOTg0IDcuMjE1IDIuODEyLjY0NCA4Ljc1MyAyLjg5NCA4Ljc1MyAxMC4zNjIgMCAyLjk4MS0uOTQxIDUuNDQ0LTIuNzk4IDcuMzE5bC0xLjQzMy0xLjQwMWMxLjQ3My0xLjQ4OCAyLjIyLTMuNDc5IDIuMjItNS45MTggMC02LjUzMi01LjUwNC04LjE1Ny03Ljg3My04LjU1MWExLjAwMiAxLjAwMiAwIDAxLS44MjMtMS4xNTdjLS4zMjktNC4wNTUtMi43NTMtNS44NzItNS4wMy01Ljg3Mi0xLjQzNyAwLTIuNzg0LjY5NS0zLjY5NyAxLjkwN2ExLjAwNiAxLjAwNiAwIDAxLTEuNzUtLjI1OGMtLjgyMy0yLjI2Ni0yLjAxLTQuMTcxLTMuNTI1LTUuNjYxLTMuODgtMy44MTYtOS4xODQtNC44NS0xNC4xOTUtMi43NjYtNC4xNyAxLjcyNy03LjQzNyA2LjcwMi03LjQzNyAxMS4zMjgiIGZpbGw9IiNGRkYiLz4KICAgIDwvZz4KICA8L2c+Cjwvc3ZnPgo=", "name": "Amazon CloudWatch Logs" }, "statements": [ { "action": "logs:CreateLogGroup", "effect": "Allow", "resource": "*", "service": "logs", "source": { "index": "0", "policyName": "AWSLambdaBasicExecutionRole", "policyType": "managed" } }, { "action": "logs:CreateLogStream", "effect": "Allow", "resource": "*", "service": "logs", "source": { "index": "0", "policyName": "AWSLambdaBasicExecutionRole", "policyType": "managed" } }, { "action": "logs:PutLogEvents", "effect": "Allow", "resource": "*", "service": "logs", "source": { "index": "0", "policyName": "AWSLambdaBasicExecutionRole", "policyType": "managed" } } ] } }, "roleName": "MyProject-EdgeFunctionFnServiceRoleC7B72E4-1DV3AZXP558ZS", "trustedEntities": [ "lambda.amazonaws.com", "edgelambda.amazonaws.com" ] } ``` I just tried using the Test in the Lambda Panel. All the tests send logs to the CloudWatch. However when I send request to the CloudFront, it didn't send anything. **UPDATE 2** I just found out from StackOverflows that the log is being stored not centrally but distributed to regions. Something like below ``` /aws/lambda/us-east-1.MyProject-EdgeFunctionFn44308ADF-loJeFwXXzTOm ``` So instead of opening it from Lambda panel, I need to open it in the CloudFront panel. Somewhat I couldn't find it in any AWS documentations. **References** https://aws.amazon.com/id/blogs/networking-and-content-delivery/aggregating-lambdaedge-logs/ https://stackoverflow.com/questions/66949758/serverless-aws-lambdaedge-how-to-debug#:~:text=Go%20to%20CloudWatch%20and%20search,%2D%3E%20Lambda%40Edge%20Errors%20.

I got error: "ec2tagger: Unable to describe ec2 tags for initial retrieval: AuthFailure: AWS was not able to validate the provided access credentials" in cloudwatch log agent on an ec2 instance that has: 1. CloudWatchAgentServerRole -- this is default AWS managed role attached to the instance, this default role already allow ""ec2:DescribeTags"," in its policy. <---- NOTE this 2. Its NACL allowed all outbound and allowed all vpc's CIDR network range inbound 3. Cloudwatch log agent config file's region is correct 4. telnet ec2.us-east-2.amazonaws.com 443 or telnet monitoring.us-east-2.amazonaws.com 443 or telnet logs.us-east-2.amazonaws.com 443 under the ec2 instance all return successful connection (Connected <..> Escape character is '^]') I also create three vpc endpoints: logs (com.amazonaws.us-east-2.logs), monitoring (com.amazonaws.us-east-2.monitoring), ec2 (com.amazonaws.us-east-2.ec2) interface endpoints. They have SG that allowed all VPC's CIDR network range inbound. The idea is to expose metrics to cloudwatch via vpc endpoints. Despite all above setup, I can't make cloudwatch agent to work and it keeps echo above error complain about credentials is not valid even though the REGION in config file is correct and traffic between instance and cloudwatch is allowed.

Is there a way to create SSO in subaccount for CloudWatch dashboard sharing so that it uses our existing Active Directory SSO from master account?

Hi, how do I retrieve multiple metrics of ec2 instance in a single aws cli call? I want to retrieve cpu, writeops, readops etc for an ec2 instance, but instead of of issuing multiple aws cloudwatch get-metric-statistics calls for each metric, is there a way to get data for all metrics in one call? ``` aws cloudwatch get-metric-statistics --metric-name EBSWriteOps --start-time "2022-04-18" --end-time "2022-04-19" --period 60 --namespace AWS/EC2 --statistics Average --dimensions Name=InstanceId,Value=$id --output text --query 'avg(Datapoints[*].Average)' aws cloudwatch get-metric-statistics --metric-name CPUUtilization --start-time "2022-04-18" --end-time "2022-04-19" --period 60 --namespace AWS/EC2 --statistics Average --dimensions Name=InstanceId,Value=$id --output text --query 'avg(Datapoints[*].Average)' ``` Can above 2 calls be combined as one call?

I need a cloudwatch alarm, to warn if any resource was created without TAG. I couldn't find any service that would make this work or any guidance. Has anyone done this here?

I am creating a solution to send SMS from a client application via AWS SNS. I am using the NodeJS sdk and able to send sms via SNS publish api. I can see the delivery status in the SNS dashboard and the log in the SMS log in the SNS dashboard. I would like to integrate the feedback loop back to my client application to receive the SMS delivery feedback. Is there any means of providing a callback Webhook to the publish api OR any integration via SNS where the SMS status feedback triggers a configured Webhook endpoint? I even looked through AWS pinpoint. So, any solution via SNS/AWS pinpoint/CloudWatch or any other AWS component is welcome.

We are developing a custom model monitoring container to be used to interact with Sagemaker's model monitoring API, as we require additional custom metrics. One of our requirements is for these metrics to populate and visualize appropriately in Studio visualization tabs, e.g. Data Quality, Model Quality, and Explainability. **None of the built-in containers are an option for us, and we are trying to interop with the Monitoring APIs as seamlessly as possible.** The docs specify the container contracts for output, specifically *statistics.json*, *constraints.json*, and *constraint_violations.json*. However, the docs are not clear on what JSON files to emit for specifically *custom Model Quality metrics*. There does not appear to be a provided schema. Things I have tried: - Emitting CloudWatch metrics to a different namespace, *sagemaker/Endpoint/model-quality-metrics* - Attempting to retrofit the statistics.json file to also include Model Quality metrics. Is there a separate JSON file I should write to */opt/ml/processing/resultdata* for our custom model quality metrics to populate the Studio visualization tab? Specifically, where do the base containers look for this JSON report: https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality-metrics.html

Hello, We are using GreenGrass v2 and would like to have logs pushed as frequently as possible. We understand the limitation for certain components (eg telemetry) but our application logs should be sent as near real time as we can. Currently, regardless of the configurations set our logs are only sent intermittently to CloudWatch. Logs are seen every few hours or even few days. Could anyone please help us understand whats happening? We are using the following configuration: ``` 'aws.greengrass.Nucleus': { componentVersion: '2.4.0', configurationUpdate: { merge: `{ "logging" : { "level" : "DEBUG", "format" : "JSON" } }`, }, }, 'aws.greengrass.LogManager': { componentVersion: '2.2.1', configurationUpdate: { merge: `{ "logsUploaderConfiguration": { "systemLogsConfiguration": { "uploadToCloudWatch": "true", "minimumLogLevel": "DEBUG", "diskSpaceLimit": "2", "diskSpaceLimitUnit": "KB", "deleteLogFileAfterCloudUpload": "true" }, "componentLogsConfigurationMap": { "com.component1": { "minimumLogLevel": "DEBUG", "diskSpaceLimit": "2", "logFileDirectoryPath": "/greengrass/v2/logs/", "logFileRegex": "com.component1\\\\w*.log", "diskSpaceLimitUnit": "KB", "deleteLogFileAfterCloudUpload": "true" }, "com.component2": { "minimumLogLevel": "DEBUG", "diskSpaceLimit": "2", "logFileDirectoryPath": "/greengrass/v2/logs/", "logFileRegex": "com.component2\\\\w*.log", "diskSpaceLimitUnit": "KB", "deleteLogFileAfterCloudUpload": "true" }, "aws.greengrass.SageMakerEdgeManager": { "minimumLogLevel": "DEBUG", "logFileDirectoryPath": "/greengrass/v2/logs/", "logFileRegex": "aws.greengrass.SageMakerEdgeManager\\\\w*.log", "diskSpaceLimit": "2", "diskSpaceLimitUnit": "KB", "deleteLogFileAfterCloudUpload": "true" }, "aws.greengrass.SecureTunneling": { "minimumLogLevel": "DEBUG", "logFileDirectoryPath": "/greengrass/v2/logs/", "logFileRegex": "aws.greengrass.SecureTunneling\\\\w*.log", "diskSpaceLimit": "2", "diskSpaceLimitUnit": "KB", "deleteLogFileAfterCloudUpload": "true" } } }, "periodicUploadIntervalSec": "10" }`, }, }, ```

Hi Everyone, please help me to understand the following situation I'm looking at my usage report and I can't understand why I'm being charged 0.10 for the 100.294 metrics considering, that my number of metrics is below 10,000 (1800 metrics), where does 100.294 come from? Thanks! | cloudwatch | metric | cost | | ---| ---| ---| | $0.10 per metric-month for the next 240,000 metrics | 100.294 Metrics | $10.03 | | $0.30 per metric-month for the first 10,000 metrics | 1800 Metrics | $540.0 |

Hi, I'm making some experiments regarding the impact of JAR size on Lambda Cold Starts and response times. I have created 3 JARs of different sizes. | JAR Size | Response time| | --- | --- | | 10mb| 1s| | 64mb| 6s| | 100mb| 12s| In Cludwatch logs I can see that init durations for all these Lambas is on average ~620ms, but for every lambda the response time is different and I don't know why. The code is the same(there is no logic -> all of them are returning a String), differences in size are due to different maven dependencies that I added. All of them are from AWS SDK. Is this normal?

The monitoring types dictates how an AWS Service sends metrics to CloudWatch and they are of two types: Basic (every 5 minutes) and Detailed (every minute). On the other hand, each metric is one of the following: Standard resolution (with data having a one-minute granularity) or High resolution, (with data at a granularity of one second). My questions: 1. If I'm getting it right, the resolution is the frequency at which metrics are generated and the monitoring type is the frequency at which a service publishes the metric to CloudWatch right? 2. If So, why have a "high resolution" metric being able to generate data each 1 second if CloudWatch can only ingest metrics each minute (based on the detailed monitoring type)?

I want to find out how can we audit AWS events/logs to see when lucidscale (a third party application) makes a connection. I already looked into AWS CloudTrail, CloudWatch but I didn't find anything there. Here is the link that shows how I imported AWS data to LucidScale for diagramming purpose: https://lucidscale.zendesk.com/hc/en-us/articles/4407993351188#import-aws-accounts-and-create-models-via-cross-account-role (Import AWS Accounts and Create Models via Cross-Account Role) Thanks for your help.

I am looking for the solution of exporting Cloudwatch logs to S3 continuously where there is log updates from application. I am looking for a automated log export from cloudwatch to S3.

Hey everyone, I am doing a general cleanup on Lambda functions. As I don't want to ruin the production environment, I want to make sure that Lambda is not used at all before deleting it. I am using Lambda monitor tab > CloudWatch metrics -> Invocation metric Can we say that when invocation count is 0 the Lambda is not used at all for the time interval I provide? Thank you all in advance for your time and help!

I have a Development account which only needs monitoring during business hours. Currently, notifications are being routed to OpsGenie via an SNS topic. There are options to silence the alerts at the OpsGenie level, but they are still generated, which skews alert metrics. Is there any way to prevent alerts from being sent to the topic outside of 9-5 business hours? Thanks!

I have 5 crons which have separate destination lambdas. All lambdas do same work except they just pass a different parameter to a common downstream. Currently there is no way to pass additional parameter alongside cron so that I can have 1 lambda as destination for 5 crons instead of 5 different ones. Can you add this feature?

Is it possible to setup cloudwatch Insight JSON rules on log groups that are in other AWS accounts? We have our monitoring stack setup in another account, we'd like for our service accounts to talk to monitoring accounts to pull out cloudwatch insight graphs on a dashboard in service accounts. Any help will be appreciated. I've tried following instructions in this article with no luck getting insight rules to display data - https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html But since cloudwatch insight rules work with log group names and not ARNs, wondering if its even possible?

I'm seeing a change that really just started yesterday afternoon, Wednesday April 6th 2022 approximately 14:00 GMT. I used to be able to trigger a Lambda function (in my case I'm using API Gateway to trigger it) and I could normally see the relevant log entries show up in the AWS CloudWatch Console usually within a few seconds. As of yesterday, the logs are taking up 30 minutes to show up and I'm wondering if anyone else has experienced anything similar or could point me in the right direction to check things out. Thank you in advance!

I would like to collect Redis metris i.e both default and custom metrics. I have installed and configured CloudWatch agent on EC2 and managed to capture host-level metrics. The Redis cluster is managed by sentinel. Is there a way to collect from the Redis cluster?

Hello community, I somehow can't find the right information. I have following simple task to solve: create a lambda that checks if a dead letter queue has messages and if it has, read how many. Before I did that I had an alarm set on an SQS metric. I chose the 'ApproximateNumberOfMessagesVisible' metric since 'NumberOfMessagesSent' (which was my first choice) does not work for DLQueues. I have read this article: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html. >The NumberOfMessagesSent and NumberOfMessagesReceived for a dead-letter queue don't match > If you send a message to a dead-letter queue manually, it is captured by the NumberOfMessagesSent metric. However, if a message is sent to a dead-letter queue as a result of a failed processing attempt, it isn't captured by this metric. Thus, it is possible for the values of **NumberOfMessagesSent** and NumberOfMessagesReceived to be different. That is nice to know, but I was missing the information: which metric shall I use if **NumberOfMessagesSent** won't work? I was being pragmatic here so I created an error, a message was sent to the DLQ as a result of a failed processing attempt. Now I looked at the queue in the AWS console under the monitoring-tab and I checked which metric spiked. It was **ApproximateNumberOfMessagesVisible**, which sounded suitable, so I used it. Now I wanted to get alerted more often so I chose to build a lambda function that checks how many messages are in the DLQueue. I use Javascript / Typescript so I found this: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_GetQueueAttributes.html. Code looked something like this: ``` const params = { QueueUrl: url, AttributeNames: ['ApproximateNumberOfMessagesVisible'] } const resp = SQS.getQueueAttributes(params).promise() ``` It was kind of a bummer that the attribute I wanted was not in there, or better: it was not valid. > Valid Values: All | Policy | VisibilityTimeout | MaximumMessageSize | MessageRetentionPeriod | ApproximateNumberOfMessages | ApproximateNumberOfMessagesNotVisible | CreatedTimestamp | LastModifiedTimestamp | QueueArn | ApproximateNumberOfMessagesDelayed | DelaySeconds | ReceiveMessageWaitTimeSeconds | RedrivePolicy | FifoQueue | ContentBasedDeduplication | ... My first attempt was to use CloudWatch metrics. So I tried this: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/cloudwatch-examples-getting-metrics.html ``` var params = { Dimensions: [ { Name: 'LogGroupName', /* required */ }, ], MetricName: 'IncomingLogEvents', Namespace: 'AWS/Logs' }; cw.listMetrics(params, function(err, data) { if (err) { console.log("Error", err); } else { console.log("Metrics", JSON.stringify(data.Metrics)); } }); ``` but I could not get this working since I did not know what to add to Dimensions / Name to make this working. Please note that I am not working very long with AWS (only 6 months). Maybe I am on a total wrong track. Summarized: I want to achieve that my lambda gets the number of messages in a DLQ. I hope someone can help me Cheers Aleks

I set my treshold to scale-up when cpu usage is 80% and scale-in when there is below 70% of usage. And the problem is that (AFAIK) for autoscaling group the average value is taken. Why its a problem? Example situation: 1. There is one node, i make 100% cpu load 2. Alarm is triggered, another instance is created 3. Now metric is divided by 2 so `(100% + 0%) / 2 = 50%` which is below 70% -> scale-in alarm is triggered and even though one node is still loaded with 100%, one node is being destroyed. Ideally for scale down i would use not average but SUMMARY of all loads on the nodes. There is `AWS::CloudWatch::Alarm/Properties/Statistic` settings with average or sum values but these are for Evaluation periods, not for ammount of factors in given dimension? https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cw-alarm.html#cfn-cloudwatch-alarms-statistic my template ``` { "AWSTemplateFormatVersion":"2010-09-09", "Description" : "Creates Autoscaling group. Used securitygroup ids and subnets ids are hardcoded.", "Parameters" : { "myprojectAmiId": { "Description": "New AMI ID which will be used to create/update autoscaling group", "Type": "AWS::EC2::Image::Id" }, "myprojectNodesDefaultQuantity":{ "Type": "Number", "MinValue" : "1" } }, "Resources" : { "myprojectLaunchTemplate":{ "Type":"AWS::EC2::LaunchTemplate", "Properties":{ "LaunchTemplateData":{ "IamInstanceProfile":{ "Arn": "arn:aws:iam::censored6:instance-profile/myproject-ec2" }, "ImageId": { "Ref":"myprojectAmiId" }, "InstanceType" : "t3a.small", "KeyName" : "my-ssh-key", "SecurityGroupIds" : [ "sg-censored", "sg-censored", "sg-censored5", "sg-censored" ] } } }, "myprojectAutoScalingGroup": { "Type":"AWS::AutoScaling::AutoScalingGroup", "UpdatePolicy" : { "AutoScalingRollingUpdate" : { "MaxBatchSize" : "1", "MinInstancesInService" : "1", "PauseTime" : "PT5M", "WaitOnResourceSignals": "true" } }, "Properties": { "MinSize":{ "Ref":"myprojectNodesDefaultQuantity" }, "MaxSize":"3", "HealthCheckGracePeriod":300, "LaunchTemplate": { "LaunchTemplateId": { "Ref":"myprojectLaunchTemplate" }, "Version":{ "Fn::GetAtt":[ "myprojectLaunchTemplate", "LatestVersionNumber" ] } }, "VPCZoneIdentifier" : [ "subnet-censored", "subnet-0censoredc" ], "TargetGroupARNs" : [ "arn:aws:elasticloadbalancing:us-west-2:censored:targetgroup/autoscaling-tests-targetgroup/censored" ], "Tags" : [ {"Key" : "Name", "Value" : "myproject-cloudformation-ascaling-tests", "PropagateAtLaunch" : true}, {"Key" : "Stack", "Value" : "dev-staging","PropagateAtLaunch" : true}, {"Key" : "CreatedBy", "Value" : "cloudformation", "PropagateAtLaunch" : true} ] } }, "myprojectScaleUpPolicy":{ "Type" : "AWS::AutoScaling::ScalingPolicy", "Properties" : { "AdjustmentType" : "ChangeInCapacity", "AutoScalingGroupName" : { "Ref" : "myprojectAutoScalingGroup" }, "Cooldown" : "60", "ScalingAdjustment" : 1 } }, "myprojectScaleDownPolicy":{ "Type" : "AWS::AutoScaling::ScalingPolicy", "Properties" : { "AdjustmentType" : "ChangeInCapacity", "AutoScalingGroupName" : { "Ref" : "myprojectAutoScalingGroup" }, "Cooldown" : "60", "ScalingAdjustment" : -1 } }, "myprojectCPUAlarmHigh": { "Type" : "AWS::CloudWatch::Alarm", "Properties" : { "AlarmActions" : [ { "Ref" : "myprojectScaleUpPolicy" } ], "AlarmDescription" : "Scale-up if CPU > 80% for 5 minutes", "ComparisonOperator" : "GreaterThanThreshold", "Dimensions" : [ { "Name": "AutoScalingGroupName", "Value": { "Ref" : "myprojectAutoScalingGroup" }} ], "EvaluationPeriods" : 2, "MetricName" : "CPUUtilization", "Namespace" : "AWS/EC2", "Period" : 30, "Statistic" : "Average", "Threshold" : 80 } }, "myprojectCPUAlarmLow": { "Type" : "AWS::CloudWatch::Alarm", "Properties" : { "AlarmActions" : [ { "Ref" : "myprojectScaleDownPolicy" } ], "AlarmDescription" : "Scale-down if CPU < 70% for 10 minutes", "ComparisonOperator" : "LessThanThreshold", "Dimensions" : [ { "Name": "AutoScalingGroupName", "Value": { "Ref" : "myprojectAutoScalingGroup" }} ], "EvaluationPeriods" : 2, "MetricName" : "CPUUtilization", "Namespace" : "AWS/EC2", "Period" : 600, "Statistic" : "Average", "Threshold" : 70 } } } } ```

We have a relatively small EFS of about 20G in burst mode, it was setup about 2 months ago and there were not much performance issue, utilization are always under 2% even under our max load usage (only for a very short period of time) And yesterday, we suddenly noticed that our site are not responding, but our server have very minimal CPU loads. Then we saw that the utilization of the EFS suddenly went up to 100%, digging deeper, it seems that we had been slowing and consistently consuming the original 2.3T BurstCreditBalance for the past few weeks, and it went to zero yesterday. Problems 1. The EFS monitoring tab provided completely useless information and does NOT even include the report of BurstCreditBalance, we had to find it in CloudWatch ourselves. 2. The Throughput utilization is misleading that we are actually slowly using up the credits, but there are no indications of such 3. We had since switched to Provisioned mode at 10MBps in the meantime as we're not really sure how to get the correct throughput number we need for our system. CloudWatch is showing 1s average max value of MeteredIOBytes 7.3k, DataReadIOBytes 770k, DataWriteIOBytes 780k. 4. we're seeing BurstCreditBalance build up much quicker (w 10MBps Provisioned) than we had used previously (in Burst). However, when we switched to 2MBps Provisioned, our system is visibly throttled even though there are 1T BurstCreditBalance, why? Main questions 1. How to properly define a Provisioned rate that is not too excessive, but not limiting our system when it needs to use it based on the CloudWatch metrics? 2. Ideally, we'd like to use Burst as that fits better, but with just 20GB, we don't seem to accumulate any BurstCreditBalance

My t3a.small instance credit report appears to be broken. Setting the CPU credit balance graph to 30s resolution reveals strange phenomenon with lines going from about 0.36 credits each 5 minutes to 576 credits one minute later: https://snipboard.io/YpX8tK.jpg This also affects our own CloudWatch credit check implementation rendered in Zabbix: https://snipboard.io/45GR9i.jpg I've already tried to reboot the instance, but the issue persists. Thanks for help.

Hi AWS, Please help me change Name in Namespace on CloudWatch All Metrics. I have config CloudWatch agent push metrics from EC2. This is an evidence image: https://postimg.cc/HJ25Zfz1 This is my configuration ``` { "agent": { "metrics_collection_interval": 60, "run_as_user": "root" }, "metrics": { "namespace": "DEV/EC2", "append_dimensions": { "AutoScalingGroupName": "${aws:AutoScalingGroupName}", "InstanceId": "${aws:InstanceId}", "InstanceType":"${aws:InstanceType}" }, "metrics_collected": { "disk": { "measurement": [ "used_percent" ], "metrics_collection_interval": 60, "resources": [ "/" ] }, "mem": { "measurement": [ "mem_used_percent" ], "metrics_collection_interval": 60 } } } } ``` Thank All. Best regard.

Is it possible to include metadata about the source of a CloudWatch alarm in onward notification to SNS? I'm working on a multi-account configuration hosting various services and plan to integrate with a third party notification and ops toolset. It would be very helpful to let operational teams receiving notifications know more about the alarm (e.g. source account, service, environment). The only fields I can see are name and description, these could be used but are unstructured so may be prone to error. Another solution would be to have multiple SNS topics, but ideally operational teams would manage any categorisation in their ops tool rather than making changes to infrastructure. Any suggestions appreciated. Thanks.

I have setup the following workflow: - private REST API with sources `/POST/event` and `/POST/process` - a `VPCLink` to an `NLB` (which points to an `ALB` pointing to a microservice running on `EKS`) - a `VPC endpoint` with DNS name `vpce-<id>-<id>.execute-api.eu-central-1.vpce.amazonaws.com` with `Private DNS enabled` - an EventBridge `EventBus` with a rule that has two targets: 1 `API Destination` for debugging/testing and 1 `AWS Service` which points to my private REST Api on the source `/POST/process` - all required `Resource Policies` and `Roles` - all resources are defined within the same AWS Account The **designed** worflow is as follows: - invoke `POST/event` on the VPC endpoint (any other invocation is prohibited by the `Resource Policy`) with an `event` payload - the API puts the `event` payload to the `EventBus` - the `EventBusRule` is triggered and sends the `event` payload to the `POST/process` endpoint on the private REST API - the `POST/process` endpoint proxies the payload to a microservice running on EKS (via `VPCLink` > `NLB` > `ALB`> `k8s Service`) **What does work** so far: - invoking `POST/event` on the VPC endpoint - putting the `event` payload to the `EventBus` - forwarding the `event` payload to the `API Destination` set up for testing/debugging (it's a temporary endpoint on https://webhook.site) - testing the `POST/event` and `POST/process` integration in the AWS Console (the latter is verified by checking that the `event` payload reaches the microservice on EKS successfully) That is all single steps in the workflow seem to work, and all permissions seem to be set properly. **Whad does not work **is invoking the `POST/process` endpoint from the `EventBusRule`, i.e. invoking `POST/event` does not invoke `POST/process` via the `EventBus`, _although_ the `EventBusRule` was triggered. So my **question** is: **How to invoke a private REST API endpoint from an EventBusRule?** **What I have already tried:** - change the order of the `EventBusRule targets` - create a Route 53 record pointing to the `VPC endpoint` and treat it as an (external) `API Destination` - allow access from _anywhere_ by _anyone_ to the REST API (temporarily only, of course) **Remark on the design:** I create _two_ endpoints (one for receiving an `event`, one for processing it) with an EventBus in between because - I have to expect a delay of several minutes between the `Event Creation/Notification` and the successful `Event Processing` - I expect several hundred `event sources`, which are different AWS and Azure accounts - I want to keep track of all events that _reach_ our API and of their successful _processing_ in one central EventBus and _not_ inside each AWS account where the event stems from - I want to keep track each _failed_ event processing in the same central EventBus with only one central DeadLetterQueue

How can I check CloudWatch agent and shell script use which apis. I wanna confirm billing charge of CloudWatch agent but I'm not sure which apis is used in CloudWatch agent and shell script. So I wanna ask for it.

I stopped and removed all my ec2 instances, IPs, images, snapshots, etc. I have ZERO active services this month, yet I am being billed for Cloudwatch. How do I turn it off?

According to [this documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Synthetics_Canaries_Library_Python.html) I should be able to set the user agent of my CloudWatch Canary using: add_user_agent(user_agent_str), ex: "synthetics_webdriver.add_user_agent('MyApp-1.0')" I have tried this, and my code that attempts this looks like the following: ``` from aws_synthetics.selenium import synthetics_webdriver as webdriver . . userAgentString = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36" . . webdriver.add_user_agent(userAgentString) driver = webdriver.Chrome() ``` The Canary passes the test, but the user agent is never set (it defaults to the standard user agent). Has anyone had any success changing the user agent for a Canary, and if so, how? Thank you!

How to trigger cloudwatch event from lambda upon creation of a record in database. My use case is I want to run a cloudwatch event that is scheduled for 5 mins for one time. As soon as a record is created in DynamoDB by Lambda, this cronjob should be triggered by same lambda and which will invoke target function after 5 mins as scheduled in cronjob.

Hi, I find results in log insights using the following query: `fields @timestamp, @message | filter log like "GET /espace-client/selfcare" | sort @timestamp desc | limit 20` The results I find are like: `172.17.82.30 - - [29/Mar/2022:17:08:54 +0200] "GET /espace-client/selfcare/contract/<redacted> HTTP/1.1" 200 4891 "<redacted referrer>" "Mozilla/5.0 (Linux; Android 11; SM-A515F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.88 Mobile Safari/537.36"` The pattern I use in my metric is: `[remote_addr, user, remote_user, timestamp, location="GET /espace-client/selfcare/contract/*", response_code=2*, body_bytes_sent, referrer, user_agent]` When I use the Test pattern feature in the metric filter creation, the above log line is matched correctly yet, when I try to create a line graph widget (metric value = 1 and default value = 0, no use of dimensions). The graph is definitely flat on a value of zero. I can tell that I have hundreds of match in log insights during one hour though. I looked for previous questions but to no avail. What is wrong with what I am doing? Thank you for your help.

We want to run containers on an ec2 fargate cluster that will run jobs of an arbitrary duration (30 minutes on average). We want to scale out based on active TCP connection metrics via the AWS loadbalancer (1 tcp connection contains stream that needs to be handled by 1 container). This seems possible with cloudwatch metrics. The question is how to **scale in** once the job is done. We do not want to stop a container when it's still processing an active connection. It could take hours for the job to finish (and that is OK). Is there a way we can remove a container once it is done processing (aka no active tcp connection anymore?) If the process inside the container stops, is it removed from the desired active container total?

I'm working on Centralized Logging of my environnement AWS presented in this guide:https://docs.aws.amazon.com/solutions/latest/centralized-logging/overview.html I'm using the same architecture as presented in AWS Doc. I'm searching the possibility to send metrics from CloudWatch of each account to OpenSearch in centralized account. I was thinking of a Lambda function that retrieves each X minutes the data with get_metric_data ? I'm collecting the metrics with CloudWatch Agent Is there an another method to do it ? Thank you

For some reason, one of my Lambda's logs are no longer appearing on CloudWatch. I see output from a Test run in the Lambda's screen but the CloudWatch list of logs is persistently empty. After some flailing around, I decided to try creating a new Lambda with the same code and configuration. Test ran. But when I hit "Click here to view the corresponding CloudWatch log group.", It opened CloudWatch looking at the expected log group name -- with a big red warning that this group did not exist. Clicking "(Logs)" at the top of the test output gave the same behavior. This is surprising; I thought I remembered that creating a Lambda created its log groups automagically...? I tried creating the group manually, but now I'm back where I was -- the lambda runs, I get local log output, but the CloudWatch Log Group for my lambda still shows no Log Streams. I checked the CloudWatch configuration, and it does list both the old and new Lambda's ARNs as being allowed to create and write to log streams... My oldest lambda (the one my Alexa skill uses directly) is still apparently writing to CloudWatch successfully. I am very confused. I'm a relatively new user, and I'm willing to believe this is user error -- but I have no idea what that error might be. Any advice folks can offer on fixing this would be tremendously appreciated, especially since my Skill just went live and the failing lambda is the one that is triggered by an EventBridge cron job to update the database in the background. It does seem to be running OK for now -- but I need logs if I'm ever going to have to debug it again, and I need to understand why a new copy of that Lambda is having the same problem. Programmer's mantra: "If it was easy, they wouldn't need _us_..."

Alarms and dashboards can be created in one monitoring account based on metrics from one or more source accounts using CloudWatch's [cross account cross region ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html) capability. Can metric filters be created in a similar way - deployed to the monitoring account but based on source accounts? I assume not given AWS::Logs::MetricFilter has property "LogGroupName" rather than ARN, but hope to be wrong. This would be great for use cases where alarms are centralised but there's no requirement for, or even security or governance constraints against centralising logs.

I am using the sagemaker python SDK to train a bespoke model. I have defined my `metric_definition` regexes and passed them to the estimator like: ```python num_re = "([0-9\\.]+)(e-?[[01][0-9])?" metrics = [ {"Name": "learning-rate", "Regex": f"lr: {num_re}"}, {"Name": "training:loss", "Regex": f"loss: {num_re}"}, # ... ] estimator = Estimator( image_uri=training_image_uri, # ... metric_definitions=metrics, enable_sagemaker_metrics=True, ) ``` When I run training, these metrics are visible in my logs and I can also see them in SageMaker Studio in `Trial Components > Metrics (tab)` as a grid of numbers like: > Name | Minimum | Maximum | Standard Deviation | Average | Count | Final value > learning-rate | 8.889 | 8.907 | 0.010392304845413657 | 8.898 | 4 |8.907 > ... Which suggests that the regexes are correctly matching on the logs However, I am not able to visualise any graphs for my metrics. I have tried all of: - `Sagemaker Studio > Trial components > charts`. It is only possible to plot things like `learning-rate_min` (i.e. a point value not a time-series metric) - `SageMaker aws console > training > training jobs > <select job> > Scroll to Monitor section`. Here I can see metrics like CPUUtilization over time but for my metrics there is just an empty graph for each metric that I have defined that says 'No data available' - `SageMaker aws console > training > training jobs > <select job> > Scroll to Monitor section > View algorithm metrics (opens in CloudWatch) > Browse > select metric (e.g. learning-rate and 'Add to Graph' `. I filter by the correct time period and go the `Graphed metrics (1) tab`, even after updating the period to `1 second` I am not able to see anything on the graph. I'm not sure what the issue is here but any help would be much appreciated

I've set up the CloudWatch agent on a Linux EC2 instance, but I'm completely stumped on how to access the metrics it produces in the CloudWatch console (in this case, some items from the procstat plug-in) . The documentation makes reference to selecting the CWAgent namespace, but I'm not seeing any way to specify this namespace when browsing metrics. All I see from this instance is the standard set of metrics like CPU utilization, etc. Any advice?

I am not able to see Disk read ops and Disk Write ops on all my instance how do I get that?

I am trying to automate Fargate AWS Batch jobs by means of AWS Cloudwatch Events. So far, so good. I am trying to run the same job definition with different configurations. I am able to set the batch job as a cloudwatch event target. I have learned how to use the Constant (JSON text) configuration to set a parameter of the job. Thus, I can set the name parameter successfully and the job runs. However, I am not able to also set the memory and cpu settings in the Cloudwatch event. I would like to use a larger machine for a a bigger port such as Singapore, without changing the job definition. After all, at the moment it still uses the default vpcu and memory settings of the job definition. ``` { "Parameters": {"name":"wilhelmshaven"}, "ContainerOverrides": { "Command": ["upload_to_day.py", "-port_name","Ref::name"], "resourceRequirements": [ {"type": "MEMORY", "value": "4096"}, {"type": "VCPU", "value": "2"} ] } } ``` Does any one know how to set the Constant (JSON text) configuration or input transformer correctly? Edit: If I try the same thing using the AWS CLI, I can achieve what I would like to do. ``` aws batch submit-job \ --job-name "run-wilhelmshaven" \ --job-queue "arn:aws:batch:eu-central-1:123666072061:job-queue/upload-raw-to-day-vtexplorer" \ --job-definition "arn:aws:batch:eu-central-1:123666072061:job-definition/upload-to-day:2" \ --container-overrides '{"command": ["upload_to_day.py", "-port_name","wilhelmshaven"], "resourceRequirements": [{"value": "2", "type": "VCPU"}, {"value": "4096", "type": "MEMORY"}]}' ```

how do we extract tomcat8 jvm memory usage metric and send it to cloudwatch?

I have configured CloudWatch Agent to send metrics to Cloudwatch for ABC instance for every 60 Seconds but **I haven't enable detailed monitoring** in ABC instance does AWS charge in this case?

Hi, Are there ways to collect GPU metrics for EC2 Windows instances, and send those metrics to Amazon CloudWatch for monitoring? NVIDIA GPU performance metrics from Amazon Elastic Compute Cloud (Amazon EC2) accelerated computing instances running Linux are available, but could not find clear documentation for instances running Windows.

We are trying to identify the better math metric logic for the alarm threshold where we build threshold on the request count that is if we receive 100 requests and 10 percent of these return http error code 5XX errors trigger an alarm. Than it must create cloud watch alarm.

I run Cloud Watch Agent on my ubuntu on ec2 instance to get matrix of mem_used_percent. But unintentionally it started to collect matrix of disk and I want Cloud Watch Agent to stop send information of disks. Is there any solution to stop just part of collection of matrix or should I just unistall it and set up again ?

I send data to front-end client by APIGateway WebSocket with a Lambda in JS, triggered by a DynamoDB Stream : The JS Code to POST a message to the front-end is : ``` const params = { ConnectionId: clientId, Data: JSON.stringify(message) // message envoyé au client }; await client.postToConnection(params).promise() ``` When I use this code in a Lambda which is used by some routes in APIGateway it works perfectly. When I use an other lambda which is activate by a dynamoDB Stream, the lambda is executed when I insert new data and the post event appears on CloudWatch for the lambda but don't appears on CloudWatch for the APIGateway Socket and of course on the front-end ??? If I insert again new different data in dynamoDB, CloudWatch shows the Post of the first inserted data and API Gateway send it twice ??? And again and again there is always an offset in the message reception in the front-end. I can't understand why ???

I'm getting an error in CloudWatch that is: ConsumedWriteCapacityUnits The threshold says: ``` ConsumedWriteCapacityUnits < 30 for 15 datapoints within 15 minutes ``` I'm new to DynamoDB. My website looks up content in this table. It is very small, both the website and the table, so it is hard to believe I'm over doing anything? I'm guessing I need to make more "WriteCapacityUnits"? But I'm not sure how to do that? Any tips or advice is welcome!

We added the S3 trigger to Lambda. How is the failure handled between S3 trigger and Lambda? What will happen if the lambda throw error ? Will it retry ? What metrics we should monitor to capture this failure to alert for manual intervention ? Can we configure retries similar to SQS and monitor DLQ ?

TL;DR; ---- How to change the color of Log-Insights Visualization? What is desired ------ Consider the following log insight query: ``` stats count(*) AS CHECKOUTS group by status ``` It will provide the logs counted by groups of status. When going to the tab of visualization I can view, for example, a Pie chart with the log information. For instance: [![Dashboard][1]][1] This chart can be added to a dashboard, just like the metrics, but selecting the color for display is very important, for instance if I have a status "FAILED" I'd like it to show red, whilst if I have a status "SUCCEDED" I'd like to render in green. How can this be achieved? [1]: https://i.stack.imgur.com/mtwBV.png

TL;DR; ---- What is the proper way to create a metric so that it generates reliable information about the log insights? What is desired ------ The current Log insights can be seen similar to the following [![AWS Log insights][1]][1] However, it becomes easier to analyse these logs using the metrics (mostly because you can have multiple sources of data in the same plot and even perform math operations between them). Solution according to docs ----- Allegedly, a log can be converted to a metric filter following a guide like [this][2]. However, this approach does not seem to work entirely right (I guess because of the time frames that have to be imposed in the metric plots), providing incorrect information, for example: [![Dashboard][3]][3] Issue with solution ----- In the previous image I've created a dashboard containing the metric count (the number 7), corresponding to the sum of events each 5 minutes. Also I've added a preview of the log insight corresponding to the information used to create the event. However, as it can be seen, the number of logs is 4, but the event count displays 7. Changing the time frame in the metric generates other types of issues (e.g., selecting a very small time frame like 1 sec won't retrieve any data, or a slightly smaller time frame will now provide another wrong number: 3, when there are 4 logs, for example). P.S. ----- I've also tried converting the log insights to metrics using [this lambda function][4] as suggested by [Danil Smirnov][5] to no avail, as it seems to generate the same issues. [1]: https://i.stack.imgur.com/0pPdp.png [2]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CountingLogEventsExample.html [3]: https://i.stack.imgur.com/Dy5td.png [4]: https://serverlessrepo.aws.amazon.com/#!/applications/arn:aws:serverlessrepo:us-east-1:085576722239:applications~logs-insights-to-metric [5]: https://blog.smirnov.la/cloudwatch-logs-insights-to-metrics-a2d197aac379

Hello, I got charged this month and I didn't know why. It seems it is from CloudWatch. Today I read about CloudWatch for the first time. It seems I did something which no I do not know how to stop. The support could not help me, and they said "If you do not want to be billed make less requests". [Image1](https://ibb.co/dfQxvhz) [Image2](https://ibb.co/SwJ7Kym) I removed all roles. Added MFA to root user. I used to drop DataDog logs using their integration (which was linked to my root user), I deleted that integration. My root user does not have credentials. Could anyone help me figure out what I did in order to stop getting billed?

I setup an IoT greengrass core device with the aws.greengrass.LogManager plugin but its not pushing logs to cloudwatch. I created the GreengrassV2TokenExchangeRole + policy with cloudformation (since i want to manage the role and policy myself and deploy it to multiple accounts) and passed that to the greengrass installer as an argument. The policy is of the same type (customer managed) and has the same permission scope as when the installer creates it, Im not sure if this is actually causing issues for the component but I really dont know what else it could be. Questions: - Is it possible to manage the IAM policy and role in cloudformation and have AWS greengrass nucleus use it? - Should i even be doing it like this? - if no then Is there a different/better approach to achieve this? Side notes - There are no errors about the logmanager not being able to push logs to cloudwatch in the greengrass.log file. - The access adviser of the policy i see that it has never been accessed throughout the tracking period. - Having Greengrass create the roles and policies itself works.

When I run a CodeDeploy deployment there is a script called "scripts.log" that is added to the code agent output on the target server that contains logs related to the lifecycle hook scripts. Is it possible to either change the log level on these or switch them off? thanks

I have a dashboard that is made up of alarms from different subaccounts, and when attempting to share it throws error on the "Accept Policy and Generate Shareable Link": page: "An error happened trying to share your dashboard" Cloudtrail shows this error: }, "eventTime": "2022-02-25T15:16:14Z", "eventSource": "iam.amazonaws.com", "eventName": "CreateRole", "awsRegion": "us-east-1", "sourceIPAddress": "x.x.x.x.x", "userAgent": "Coral/Netty4", ** "errorCode": "InternalFailure", "errorMessage": "CreateRole operation failed",** "requestParameters": null, "responseElements": null, "requestID": "e25fb6da-ffca-4d9f-b1d2-0e6c2xxxxxx", "eventID": "301acbbf-fcf2-479b-a6fd-7b70axxxxxx", "readOnly": false, "eventType": "AwsApiCall", "managementEvent": true, "recipientAccountId": "12345678910", "eventCategory": "Management" }

Hello everybody, we've successfully deployed the Content Localization service with the CloudFormation template. When we upload a video in the Content Localization Web App, the following error is raised: ``` { "error": "ValueError", "cause": { "errorMessage": "Exception: 'Stage Translate encountered and error during execution, aborting the workflow'", "errorType": "ValueError", "stackTrace": [ " File \"/var/task/app.py\", line 315, in complete_stage_execution_lambda\n return complete_stage_execution(\"lambda\", event[\"Name\"], event[\"Status\"], event[\"Outputs\"], event[\"WorkflowExecutionId\"])\n", " File \"/var/task/app.py\", line 460, in complete_stage_execution\n raise ValueError(\n" ] } } ``` The error is raised at the end of the translation. GraphInspector shows the stage "Complete Stage Translate" in red color. CloudWatch logs show the same error and don't provide additional info. The marked comments in the code shortly before the ValueError is raised indicate that might be a known issue (lines 437-460 in `/var/task/app.py`): ``` ########### SEE THIS COMMENT: # Start the next stage for execution # FIXME - try always completing stage # status == awsmie.STAGE_STATUS_COMPLETE: ############ KNOWN ISSUE? workflow_execution = start_next_stage_execution( "Workflow", stage_name, workflow_execution) if status == awsmie.STAGE_STATUS_ERROR: raise Exception("Stage {} encountered and error during execution, aborting the workflow".format(stage_name)) except Exception as e: logger.info("Exception {}".format(e)) # Need a try/catch here? Try to save the status execution_table.put_item(Item=workflow_execution) update_workflow_execution_status(workflow_execution["Id"], awsmie.WORKFLOW_STATUS_ERROR, "Exception while rolling up stage status {}".format(e)) logger.info("Exception {}".format(e)) raise ValueError( "Exception: '%s'" % e) ``` Do you have any idea on how to solve this issue? Thank you very much in advance! Best regards

I signed up an AWS 1GB LightSail plan less than two months ago with WordPress installed, did a test run for a few weeks, and everything seems fine. So I moved my WordPress website there about two weeks ago. Ever since then the instance just keeps down every two or three days. https://imgur.com/ewaGlJU This graph shows my CPU usage and remaining burst capacity. The CPU usage stays inside the sustainable zone most of the time, the remaining burst capacity stays at 100%, and the dips are caused by rebooting. When the instance is down (The site is not accessible, and I can't SSH to it via putty or web either), I have to stop and start the instance, then the remaining capacity drops to 20% and slowly climbs to 100% and stays there, till the next cycle. https://imgur.com/TnHTNun This is the last 24 hours memory usage chart. During the normal time, the memory usage is below 40%. The high sparks are caused by me when I remotely connected to it via Visual Studio Code. When my site was down (reported down time at 5:50am CST), the memory usage dropped to 27% (at the end of chart around 12:00UTC). https://imgur.com/OVwRieu This morning after I received the site down message (around 5:55am CST), I tried to connect to the instance. The remaining burst capacity stayed at 100%, CPU usage was around 1%, and memory usage was around 27%. Around 6:40am, (the CPU usage is still 1%, burst capacity 100%, memory usage 27% at the time), I gave up, so I stopped and started the instance. When the instance was up around 6:45am, the CPU usage was around 2%, burst capacity dropped to 20%, memory usage jumped to 35%. Can anyone help me to figure out what's going on here? and what are other ways to troubleshot this problem? Thanks

I am trying to setup a canary on our API. Our API requires a Oauth2 token. Keys and secrets to generate this token are kept in secrets manager. I used a Python script to run system tests, that uses the 'Oauth2lib' to generate this token, and this system works fine. I tried to use the same technique in a custom canary script, but it refuses to find the library, just get module not found, "ModuleNotFoundError: No module named 'oauthlib'" I've built this locally, all the packages are at root, the python file in in the 'python' direction (not consistent with other lambdas I might add). I built and uploaded to s3, created a canary and added the file via s3. After resolving the handler entry point, it finds the Python script fine, but refuses to find the library. I found some instructions here, https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Synthetics_Canaries_WritingCanary_Python.html but don't feel they are complete, at least not for me. So can you tell me how I get this working please? Thanks Leona.

I have multiple dedicated IPs in my SES setup and sending works good so far. I'd like to analyse the Microsoft SNDS reputation data that should be available according to the Developer Guide (https://docs.aws.amazon.com/ses/latest/dg/snds-metrics-dedicated-ips.html). Looking into CloudWatch under **Metrics > All Metrics** and selecting SES only gives me the option to select **Account Metrics**. I guess this is where the **IP Metrics** mentioned in the guide should be available. Is there any additional setup needed to enable the IP Metrics in CloudWatch for SES?

Hi, I created a graph on CloudWatch using a query. If the graph is set to 3h or less, it's displaying. But when it's set to anything greater than 3h, its displaying a `Max time window exceeded for SQL query` error. Is there any limit I should adjust to make my query work? This is the query I'm using ``` SELECT AVG("4XXError") FROM SCHEMA("AWS/ApiGateway") WHERE ApiName = 'apiname' AND Stage = 'stage' ``` TIA

I'm using CloudWatch to monitor cpu_usage_system metric from CWagent. I'm plotting data that is more then 24h old. When using the regular CloudWatch browsing tab to view the data I see data points, when I do the same with CloudWatch SQL I do not.

Does Step Functions have metrics for how many times it retried to execute a state?

Hello. I have 12 lambda functions. This month I received an AWS Free Tier limit alert. I would like to make a report, on which lambda consumes more times per GB and/or made many requests. For example, Lambda 1 operated 5000 requests and consumed 350000 GB-second. Using the day-by-day reports I can see anomalies and made decisions to solve them. Tell me please is it possible? How?

I have an alarm on my SES Bounce Reputation that sends an SNS message when the threshold is breached. However, this action only happens once and if I happen to miss that notification then I'm in danger of having my email processing impacted. In the past we've used other notification systems that allow you to configure repeating alerts. If you don't respond to the alert within some time period (ie. 6 hours) it triggers another notification. Is this type of thing possible with CloudWatch Alarms? How do you handle missed alert notifications? On a related note; is it possible to trigger a CloudWatch alarm notification at a specific time? For example, at 9am every morning, alert me if my SES bounce reputation is > 5%. This would also solve the problem of missing a notification. At least you would get another one the next day.

Can anyone point me in the direction of an example on setting up elastic beanstalk memory usage? Before I had an .ebextension file which downloaded the `CloudWatchMonitoringScripts` zip and it worked fine. However now we are using AL2 I cannot work out how to get the cloudwatch agent to also send the memory usage metrics as well. I'm guessing this has to be done with a new config file and passed to the agent from a platform hook? Thanks

In older version of AWS Cloudwatch agent, there was a parameter for initial_position. But I am not able to find it in the new Cloudwatch agent. Older version of CloudWatch ``` initial_position = start_of_file ``` But I am not able to find the same anywhere in the new documentation. Please help in pointing to the parameter for this. ``` collect_list :[{ "file_path": "/var/log/nginx/access.log", "log_group_name": "GroupName", "log_stream_name": "nginxAccessLogStream", "timezone": "UTC" }] ``` Edit: Found Logs Input Plugin. Field: from_beginning https://github.com/aws/amazon-cloudwatch-agent/tree/f870550f94a822f3eceaea0ecb2efdeabe961e5c/plugins/inputs/logfile But adding it is not running the plugin.

I would like to automate the process of Enabling cross-account functionality in CloudWatch for each new account so that there is no need to go through this [link](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html) tutorial on each new account. I have an account dedicated to monitoring that should have CloudWatch access by default in every new and existing account in the organisation. Ideally, it would be possible to set something like this up via CDK, but any alternative would be welcome.