Hi,
I am trying to set up CloudWatch log streaming for custom logs for a multi-container application running on elastic beanstalk.
The issue that I am facing is when the log stream is sent over from an instance instead of being the instance id such as i-123456789a1234567
it is sending the hostname which is like ip-00-0-000-000.eu-west-1.compute.internal
Following the documentation here https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html#AWSHowTo.cloudwatchlogs.loggroups under Custom log file streaming
and the following configuration block (found on the elastic beanstalk samples - https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/logs-streamtocloudwatch-linux.config)
packages:
yum:
awslogs: []
files:
"/etc/awslogs/awscli.conf" :
mode: "000600"
owner: root
group: root
content: |
[plugins]
cwlogs = cwlogs
[default]
region = `{"Ref":"AWS::Region"}`
"/etc/awslogs/awslogs.conf" :
mode: "000600"
owner: root
group: root
content: |
[general]
state_file = /var/lib/awslogs/agent-state
"/etc/awslogs/config/logs.conf" :
mode: "000600"
owner: root
group: root
content: |
[/var/log/messages]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/messages"]]}`
log_stream_name = {instance_id}
file = /var/log/messages
[/var/log/dmesg]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/dmesg"]]}`
log_stream_name = {instance_id}
file = /var/log/dmesg
commands:
"01":
command: systemctl enable awslogsd.service
"02":
command: systemctl restart awslogsd
Any ideas on how I can get this to stream through the reference of the EC2 instance rather than the hostname?
Thanks
Thanks very much for your response!
I think you are right that for some reason it is trying to use the old CW Logs Agent, I tried updating the configuration of the yum install to match the newer package name.
This didn't work however but it did match the latest release on the Github release for the package https://github.com/aws/amazon-cloudwatch-agent/releases/
I then followed this example from the AWSDocs Github
which used a container command to start the Cloudwatch agent in a different way
I think the next steps are to try following these instructions from the link you provided https://repost.aws/knowledge-center/cloudwatch-push-metrics-unified-agent on a single EC2 instance running inside Elastic Beanstalk, if it works then try and automate it into the
.ebextensions