Automating CodeDeploy Agent Debug Logging to CloudWatch Logs for Auto Scaling Environments

Background

When using Auto Scaling Groups (ASG) with CodeDeploy, a deployment failure can cause the ASG to terminate the unhealthy instance automatically. This makes it difficult to investigate the root cause, as the logs on the instance are lost. A common error seen in this scenario is:

CodeDeploy agent was not able to receive the lifecycle event. 
Check the CodeDeploy agent logs on your host and make sure the agent is running and can connect to the CodeDeploy server.

By configuring the Launch Template user-data to install both the CodeDeploy agent and the CloudWatch agent, all relevant logs are persisted in CloudWatch Logs — even after the instance is terminated.

Collected Logs

Log Group	File Path	Purpose
/codedeploy/agent-log	/var/log/aws/codedeploy-agent/codedeploy-agent.log	Agent startup and communication failures
/codedeploy/deployment-log	/opt/codedeploy-agent/deployment-root/deployment-logs/codedeploy-agent-deployments.log	Lifecycle event script execution failures
/ec2/cloud-init-output	/var/log/cloud-init-output.log	User-data execution failures

Prerequisites

• An existing Auto Scaling Group with a CodeDeploy Deployment Group attached
• IAM Instance Profile with CloudWatch Logs permissions (see IAM Permissions section below)

Steps

1. Create the Log Groups

Create the log groups with a short retention (3 days) since these are for debugging purposes.

REGION="<REGION>"

for LOG_GROUP in /codedeploy/agent-log /codedeploy/deployment-log /ec2/cloud-init-output; do
  aws logs create-log-group \
    --log-group-name "$LOG_GROUP" \
    --region "$REGION" 2>/dev/null || true
  aws logs put-retention-policy \
    --log-group-name "$LOG_GROUP" \
    --retention-in-days 3 \
    --region "$REGION"
done

2. Add the user-data script to the Launch Template

Amazon Linux 2023 based. Replace <REGION> with your region (e.g., us-east-1).

#!/bin/bash
set -e

REGION="<REGION>"

# 1. Install CodeDeploy Agent
dnf install -y ruby wget
wget "https://aws-codedeploy-${REGION}.s3.${REGION}.amazonaws.com/latest/install" -O /tmp/cd-install
chmod +x /tmp/cd-install
/tmp/cd-install auto

# 2. Enable debug logging
sed -i 's/:log_level:.*/:log_level: debug/' /etc/codedeploy-agent/conf/codedeployagent.yml
systemctl restart codedeploy-agent

# 3. Install CloudWatch Agent and configure log delivery
dnf install -y amazon-cloudwatch-agent

cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json << 'EOF'
{
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/aws/codedeploy-agent/codedeploy-agent.log",
            "log_group_name": "/codedeploy/agent-log",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/opt/codedeploy-agent/deployment-root/deployment-logs/codedeploy-agent-deployments.log",
            "log_group_name": "/codedeploy/deployment-log",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/var/log/cloud-init-output.log",
            "log_group_name": "/ec2/cloud-init-output",
            "log_stream_name": "{instance_id}"
          }
        ]
      }
    }
  }
}
EOF

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config -m ec2 \
  -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s

3. Verify the setup

After a new instance launches, confirm logs are being delivered:

aws logs describe-log-streams \
  --log-group-name /codedeploy/agent-log \
  --order-by LastEventTime --descending \
  --limit 5 --region <REGION>

IAM Permissions

The instance's IAM Role must include the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogStreams"
      ],
      "Resource": [
        "arn:aws:logs:*:*:log-group:/codedeploy/*",
        "arn:aws:logs:*:*:log-group:/ec2/*"
      ]
    }
  ]
}

Example Logs

The following are real log outputs captured from a test instance using this setup.

/codedeploy/agent-log — Agent startup and polling

2026-03-18T13:56:58 INFO  [codedeploy-agent(26018)]: On Premises config file does not exist or not readable
2026-03-18T13:56:58 INFO  [codedeploy-agent(26018)]: Creating client url from IMDS region and domain
2026-03-18T13:56:58 INFO  [codedeploy-agent(26018)]: CodeDeploy endpoint: https://codedeploy-commands.us-east-1.amazonaws.com
2026-03-18T13:56:58 INFO  [codedeploy-agent(26018)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandExecutor: Archives to retain is: 5}
2026-03-18T13:56:58 INFO  [codedeploy-agent(26018)]: Version file found in /opt/codedeploy-agent/.version with agent version OFFICIAL_1.8.1-26_rpm.
2026-03-18T13:56:57 INFO  [codedeploy-agent(26016)]: master 26016: Spawned child 1/1
2026-03-18T13:56:58 INFO  [codedeploy-agent(26016)]: Started master 26016 with 1 children
2026-03-18T13:57:43 INFO  [codedeploy-agent(26018)]: [Aws::CodeDeployCommand::Client 200 45.391192 0 retries] poll_host_command(host_identifier:"arn:aws:ec2:us-east-1:<ACCOUNT_ID>:instance/<INSTANCE_ID>")

/codedeploy/deployment-log — Deployment lifecycle event execution

# Logfile created on 2026-03-18 05:49:57 +0000 by logger.rb/41954
[2026-03-18 05:49:57.592] [d-O8UNSJFIH]LifecycleEvent - ApplicationStart
[2026-03-18 05:49:57.592] [d-O8UNSJFIH]Script - scripts/start.sh
[2026-03-18 05:49:57.623] [d-O8UNSJFIH][stdout]Application started successfully

/ec2/cloud-init-output — User-data execution (CodeDeploy + CloudWatch agent install)

Cloud-init v. 22.2.2 running 'modules:final' at Wed, 18 Mar 2026 13:55:32 +0000. Up 6.50 seconds.

Saving to: '/tmp/cd-install'
2026-03-18 13:55:56 (190 MB/s) - '/tmp/cd-install' saved [19045/19045]

amazon-cloudwatch-agent-1.300064.1-1.amzn2023.x86_64
Install  1 Package
Total download size: 67 M

****** processing amazon-cloudwatch-agent ******
Starting config-downloader, this will map back to a call to amazon-cloudwatch-agent
I! Trying to detect region from ec2
Start configuration validation...
Configuration validation first phase succeeded
Configuration validation second phase succeeded
Configuration validation succeeded

Cloud-init v. 22.2.2 finished at Wed, 18 Mar 2026 13:57:06 +0000. Datasource DataSourceEc2.  Up 100.04 seconds

Important Notes

• {instance_id} in the CloudWatch agent config is a built-in variable — do not replace it. The CloudWatch agent automatically resolves it to the EC2 instance ID at runtime.
• Retention is set to 3 days. Delete the log groups or adjust retention after debugging is complete.

Related Resources

• Integrating CodeDeploy with Amazon EC2 Auto Scaling
• Troubleshoot Amazon EC2 Auto Scaling issues
• Install the CodeDeploy agent (Amazon Linux)
• CodeDeploy agent configuration reference
• Collect metrics and logs with the CloudWatch agent