By using AWS re:Post, you agree to the Terms of Use

Questions tagged with Amazon CloudWatch

Sort by most recent
  • 1
  • 12 / page

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Issue with pushing an EC2 instance's Docker container logs into CloudWatch

I have a working EC2 instance in free tier, with a responding **java-based** grpc server in a docker container inside the instance.\ I'd like to send the logs of the container into the CloudWatch.\ I created the suggested policy, the EC2 role, and the role is attached to the instance.\ The container is started from the bash of the linux instance with this command:\ `docker run -d -p 9092:9092 -t <<my-container-name>> --log-driver=awslogs --log-opt awslogs-region=us-east-1 --log-opt awslogs-group="gRPC-POC" --log-opt awslogs-stream="gRPC-POC-log" --log-opt awslogs-create-group=true --log-opt awslogs-create-stream=true` \ I tried to run the container with different users, with different options of the log-driver, omitting parts and almost everything.\ The policy I created to use the CloudWatch looks like this:\ ``` { "Version": "2012-10-17", "Statement": [ { "Action": [ "logs:CreateLogStream", "logs:CreateLogGroup", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Effect": "Allow", "Resource": "arn:aws:logs:us-east-1:<<my-account-number>>:log-group:*:*" } ] } ``` So far,no sign of the gathered logs in CloudWatch even if I create a log-group and/or log-stream or I don't.\ Maybe I'm missing a step or a vital information somewhere?\ Do You have any suggestions, please? #EDIT The command `aws sts get-caller-identity` gives this result: ![Enter image description here](/media/postImages/original/IM2OUiCy6OTyi-RAGhLS-C1g) The command was used from the bash of the running instance. (This is what You meant, @Roberto? Anyways, thanks.)\ It looks like the instance has the proper right, 'GrpcPocAccessLogs'.
2
answers
0
votes
45
views
asked 5 days ago

CloudWatch Unified Agent custom namespace not showing up on CloudWatch Metrics

We have previously succeeded in running CWUA on our Auto Scaling Group Ubuntu EC2 servers with custom configuration from SSM Parameter store. The parameter value looks like ``` { "agent": { "metrics_collection_interval": 60, "run_as_user": "root" }, "metrics": { "namespace": "cdk-sample-asg-ASG1-asg-exp", "append_dimensions": { "AutoScalingGroupName": "${aws:AutoScalingGroupName}", "InstanceId": "${aws:InstanceId}", "InstanceType": "${aws:InstanceType}" }, "aggregation_dimensions": [ [ "AutoScalingGroupName" ] ], "metrics_collected": { "mem": { "measurement": [ { "name": "mem_used_percent", "unit": "Percent" } ], "metrics_collection_interval": 60 } } } } ``` and the user data script in the launch template to pick it up looks like ``` wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/arm64/latest/amazon-cloudwatch-agent.deb -O /tmp/amazon-cloudwatch-agent.deb dpkg -i /tmp/amazon-cloudwatch-agent.deb /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:AmazonCloudWatch-cdk-sample-asg-ASG1-asg-exp ``` All these configuration and deployment were achieved with CDK. Now we're generalising into more reusable constructs with the same setup, but the test deployment for some reason does not show the custom ASG namespace in CloudWatch Metrics. Initially I thought it might be because Detailed monitoring wasn't enabled but changing that did not improve the situation. What other obstacles can prevent the custom namespace from showing up? The system log shows CWUA successfully installed and configured. ``` [ 54.811101] cloud-init[1274]: 2022-08-12 09:59:17 (6.15 MB/s) - ‘/tmp/amazon-cloudwatch-agent.deb’ saved [57412840/57412840] [ 54.836742] cloud-init[1274]: Selecting previously unselected package amazon-cloudwatch-agent. [ 54.864478] cloud-init[1274]: (Reading database ... 65339 files and directories currently installed.) [ 54.867059] cloud-init[1274]: Preparing to unpack .../amazon-cloudwatch-agent.deb ... [ 54.896405] cloud-init[1274]: create group cwagent, result: 0 [ 54.922558] cloud-init[1274]: create user cwagent, result: 0 [ 54.940742] cloud-init[1274]: create group aoc, result: 0 [ 54.960169] cloud-init[1274]: create user aoc, result: 0 [ 54.961249] cloud-init[1274]: Unpacking amazon-cloudwatch-agent (1.247354.0b251981-1) ... [ 56.464911] cloud-init[1274]: Setting up amazon-cloudwatch-agent (1.247354.0b251981-1) ... [ 56.534388] cloud-init[1274]: ****** processing amazon-cloudwatch-agent ****** [ 56.534654] cloud-init[1274]: /opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source ssm:AmazonCloudWatch-cdk-sample-asg-ASG1-asg-exp --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default [ 56.540078] cloud-init[1274]: I! Trying to detect region from ec2 [ 56.540893] cloud-init[1274]: D! [EC2] Found active network interface [ 56.544437] cloud-init[1274]: Region: ap-southeast-1 [ 56.544580] cloud-init[1274]: credsConfig: map[] [ 56.605107] cloud-init[1274]: Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-cdk-sample-asg-ASG1-asg-exp.tmp [ 56.608867] cloud-init[1274]: Start configuration validation... [ 56.609027] cloud-init[1274]: /opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default [ 56.618549] cloud-init[1274]: 2022/08/12 09:59:19 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-cdk-sample-asg-ASG1-asg-exp.tmp ... [ 56.621567] cloud-init[1274]: 2022/08/12 09:59:19 I! Valid Json input schema. [ 56.622296] cloud-init[1274]: I! Detecting run_as_user... [ 56.622471] cloud-init[1274]: I! Trying to detect region from ec2 [ 56.622597] cloud-init[1274]: D! [EC2] Found active network interface [ 56.626058] cloud-init[1274]: No csm configuration found. [ 56.626169] cloud-init[1274]: No log configuration found. [ 56.626318] cloud-init[1274]: Configuration validation first phase succeeded [ 56.628994] cloud-init[1274]: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml [ 56.681402] cloud-init[1274]: Configuration validation second phase succeeded [ 56.681576] cloud-init[1274]: Configuration validation succeeded [ 56.695308] cloud-init[1274]: amazon-cloudwatch-agent has already been stopped [ 56.950624] cloud-init[1274]: Created symlink /etc/systemd/system/multi-user.target.wants/amazon-cloudwatch-agent.service → /etc/systemd/system/amazon-cloudwatch-agent.service. [ OK ] Started Amazon CloudWatch Agent. ``` UPDATE Comparing to the instances that have worked, I notice some extra actions missing in the older version (circa April 2022). ``` [ 30.714094] cloud-init[851]: 2022-04-27 17:02:49 (4.98 MB/s) - ‘/tmp/amazon-cloudwatch-agent.deb’ saved [54613346/54613346] [ 32.203984] cloud-init[851]: (Reading database ... 110768 files and directories currently installed.) [ 32.208150] cloud-init[851]: Preparing to unpack .../amazon-cloudwatch-agent.deb ... [ 32.256865] cloud-init[851]: ****** processing cwagent-otel-collector ****** [ 32.261143] cloud-init[851]: cwagent-otel-collector has already been stopped [ 32.600172] cloud-init[851]: ****** processing amazon-cloudwatch-agent ****** Stopping Amazon CloudWatch Agent... [ OK ] Stopped Amazon CloudWatch Agent. [ 33.518263] cloud-init[851]: Unpacking amazon-cloudwatch-agent (1.247350.0b251780-1) over (1.247350.0b251780-1) ... [ 35.821820] cloud-init[851]: Setting up amazon-cloudwatch-agent (1.247350.0b251780-1) ... [ 35.892578] cloud-init[851]: ****** processing amazon-cloudwatch-agent ****** [ 35.893649] cloud-init[851]: /opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source ssm:AmazonCloudWatch-Original-app-asg-dev --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default [ 35.902414] cloud-init[851]: Region: ap-southeast-1 [ 35.903189] cloud-init[851]: credsConfig: map[] [ 36.010690] cloud-init[851]: Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-Original-app-asg-dev.tmp [ 36.015002] cloud-init[851]: Start configuration validation... [ 36.016156] cloud-init[851]: /opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default [ 36.022624] cloud-init[851]: 2022/04/27 17:02:54 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm_AmazonCloudWatch-Original-app-asg-dev.tmp ... [ 36.025894] cloud-init[851]: Valid Json input schema. [ 36.027002] cloud-init[851]: I! Detecting run_as_user... [ 36.030131] cloud-init[851]: No csm configuration found. [ 36.031144] cloud-init[851]: No log configuration found. [ 36.032190] cloud-init[851]: Configuration validation first phase succeeded [ 36.033524] cloud-init[851]: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml [ 36.065638] cloud-init[851]: Configuration validation second phase succeeded [ 36.066681] cloud-init[851]: Configuration validation succeeded [ 36.076898] cloud-init[851]: amazon-cloudwatch-agent has already been stopped [ OK ] Started Amazon CloudWatch Agent. ``` There are no additional users and groups created, and no final symlink created between the service files. However I am not experienced enough with Linux to properly grasp the significance of these differences and how they might cause report failure.
1
answers
0
votes
8
views
asked 6 days ago
  • 1
  • 12 / page