Docker aws log driver from on-premise server

Hi Enno

The error message indicates that Docker is unable to find the necessary credentials to create a CloudWatch log stream.

• instead of relying on IMDS, explicitly provide the AWS credentials in your .env file:
• Ensure these keys have the necessary permissions to create CloudWatch log streams (e.g., logs:CreateLogStream).

Use IAM Roles for Containers (Recommended):

• Create an IAM role with the logs:CreateLogStream permission and attach it to an EC2 Instance Profile.
• Assign this Instance Profile to the EC2 instance where your Docker containers run.
• In your docker-compose configuration, set the awslogs-skip-credential-verification option to true:

Docker awslogs driver documentation: https://docs.docker.com/config/containers/logging/awslogs/

Troubleshooting IAM roles for containers: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security-ecs-iam-role-overview.html

Hi,

I think that you should read this blog post in details: https://aws.amazon.com/blogs/containers/deploy-applications-on-amazon-ecs-using-docker-compose/

It proposes a solution to send logs to CloudWatch from docker containers managed with compose.

Best,

Didier

Hello,

Please try this solution,

To send logs to AWS CloudWatch from your on-premises server using Docker Compose, make sure your AWS credentials are correctly set in your environment variables and verified with the AWS CLI. Update your docker-compose.yaml file to include logging options with your AWS region, log group, and log stream.

version: '3.8'

services:
  your-service:
    image: your-image
    env_file:
      - .env
    logging:
      driver: awslogs
      options:
        awslogs-region: us-east-2
        awslogs-group: test-group
        awslogs-stream: test-stream

your .env file contains the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Restart your Docker Compose setup with docker-compose down followed by docker-compose up to apply the changes.

if you want to more information please go through the AWS Document link.

https://aws.amazon.com/blogs/containers/deploy-applications-on-amazon-ecs-using-docker-compose/

https://docs.docker.com/config/containers/logging/awslogs/

Hey there After many tests i was able to send logs to cloudwatch from an on-premise docker container notice that credentials must be set at docker daemon level

• Edit your daemon configuration
  • in my case i am running an ubuntu server /lib/systemd/system/docker.service
  • add these 4 env vars under [Service]

[Service]
Environment="AWS_EC2_METADATA_DISABLED=true"
Environment="AWS_EC2_METADATA_V1_DISABLED=true"
Environment="AWS_ACCESS_KEY_ID=*******"
Environment="AWS_SECRET_ACCESS_KEY=*********"

• systemctl daemon-reload
• systemctl status docker.service
• Run your container

docker run \
  --rm \
  --log-driver=awslogs \
  --log-opt awslogs-region=us-west-2 \
  --log-opt awslogs-group=browserless-local \
......

hope you find this helpful :)