Skip to content

CD pipeline fails due to failed EC2 Deploy action on ARM64 EC2 instance running Ubuntu 24.04

0

I have an ARM64 EC2 instance (based on AWS Graviton) running Ubuntu 24.04 & I use AWS CodePipeline to build & deploy code to my EC2 instance. I am using the newly introduced 'EC2 Deploy' action (Deploy -> Amazon EC2) in CodePipeline to invoke my custom deployment scripts on EC2 & automatically deploy my application. It was working properly for the last few weeks. On June 16 2025, at around 6:45 PM GMT, our pipeline run started & failed shortly. The error was caused by EC2 Deploy action failing during the 'DOWNLOAD' event.

The log of the Deploy action is as follows:-

[2025/06/16 18:52:21.934] Deploy event BLOCK_TRAFFIC succeeded on instances: i-[REDACTED].

[2025/06/16 18:52:21.954] Processing deploy event DOWNLOAD on instances: i-[REDACTED].

[2025/06/16 18:52:22.118] Executing commands on instances i-[REDACTED], SSM command id [REDACTED], commands: mkdir -p /opt/codepipeline if [ $(uname -m | grep arm) ]; then export DE_URL=[REDACTED]; else export DE_URL=[REDACTED]; fi curl "$DE_URL" -o /opt/codepipeline/deploy || wget -O /opt/codepipeline/deploy "$DE_URL" chmod +x /opt/codepipeline/deploy /opt/codepipeline/deploy -event Download -executionId [REDACTED] -bucket codepipeline-ap-southeast-2-[REDACTED] -key [REDACTED]/BuildArtif/KBdQEFJ -accountId [REDACTED]

[2025/06/16 18:52:52.800] Instance i-[REDACTED] is FAILED on event DOWNLOAD, message: SSM Command Failed, please check SSM Command for details.

[2025/06/16 18:52:52.862] Found 1 instances failed deploy. Max error 1

[2025/06/16 18:52:52.887] Deployment FAILED

In this 'DOWNLOAD' event, a Unix shell script is invoked via SSM commands. When I examined the details of the failure in AWS Systems Manager, I see that AWS downloads an executable called 'deploy' into the EC2's /opt/codepipeline directory from a CloudFront URL after determining that the EC2 architecture is ARM64. It then tries to execute the downloaded executable, but the execution fails with an error 'Exec format error'. Because the executable is compatible with x86-64, not ARM64. Yes, Amazon has erroneously uploaded an x86-64 executable at the ARM64 download URL. I've confirmed this by running 'file /opt/codepipeline/deploy' and this is what I get:-

deploy: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=[REDACTED], with debug_info, not stripped

Currently, all our automatic deployments are failing. Kindly fix it. It is URGENT.

2 Answers
0
Accepted Answer

I can confirm that Amazon has fixed this issue. The correct 'deploy' executable is now downloaded to the ARM64 based EC2 instance in /opt/codepipeline directory and the deployment happens correctly.

answered 10 months ago
0

It appears you're experiencing an issue with the EC2 Deploy action in CodePipeline where the deployment agent executable being downloaded is incompatible with your ARM64 architecture. The error occurs because the system is downloading an x86-64 executable instead of an ARM64-compatible one, despite correctly detecting your ARM64 architecture.

This is a known issue with the native EC2 deployment support that was introduced in February 2025. While the feature is designed to simplify deployments to EC2 instances without requiring CodeDeploy resources, there appears to be a problem with the architecture detection or executable distribution for ARM64 instances.

As a temporary workaround, you have a few options:

  1. Retry the failed stage in your pipeline using the AWS CLI command:
aws codepipeline retry-stage-execution --pipeline-name YourPipelineName --stage-name Deploy --pipeline-execution-id YourExecutionId --retry-mode FAILED_ACTIONS

This might work if the issue was temporary.

  1. Consider reverting to using AWS CodeDeploy for your deployments. Note that while CodeDeploy does support ARM64 instances running Amazon Linux 2 and Amazon Linux 2023, there are known limitations with Ubuntu 24.04 on ARM64.

  2. Contact AWS Support to report this specific issue with the EC2 Deploy action providing incorrect executables for ARM64 architecture.

Since this appears to be an issue with AWS's deployment mechanism rather than your configuration, it will likely require AWS to fix the problem by ensuring the correct ARM64-compatible executable is provided at the ARM64 download URL.
Sources
AWS CodeDeploy Host Agent not available for ARM64 EC2 instance running Ubuntu 24.04.1 LTS | AWS re:Post
Configuring stage retry for a failed stage or failed actions - AWS CodePipeline
AWS CodePipeline adds native Amazon EC2 deployment support - AWS

answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.