Get Real Time CloudWatch Logs for EC2 Image Builder Pipeline Execution in Shell Script Using AWS CLI

Question

I am using Shell Script in my GitHub action. In that I am using AWS CLI to execute EC2 Image Builder Pipeline and want to get the real time logs from Cloud Watch and show it on the GitHub Output Console. I am using the following commands. But I can't get it working. Please help.

```
exec_out=$(aws imagebuilder start-image-pipeline-execution --image-pipeline-arn $pipeline_arn)
imgBldr_arn=$(echo "${exec_out}" | jq --raw-output '.imageBuildVersionArn')
log_group_name="/aws/imagebuilder/my-img-pipeline"
log_stream_name_prefix="1.0.0/1"
 while true
      do
        echo Running: aws imagebuilder get-image --image-build-version-arn "${imgBldr_arn}"  
        arn_out=$(aws imagebuilder get-image --image-build-version-arn "${imgBldr_arn}")
        status=$(echo "${arn_out}" | jq --raw-output '.image.state.status')
        echo "Current Status: ${status}"
        if  [[ "${status}" -ne "AVAILABLE" ]]; then
          aws logs tail ${log_group_name} --follow  --log-stream-name-prefix  ${log_stream_name_prefix} 
          continue
        elif [[ "${status}" -eq "AVAILABLE" ]]; then
          echo "------------- Finished successfully ---------------"
          break
        fi
 done
```

Answer

There are a couple issues with the provided script that are likely causing the problem displaying the logs:

The aws logs tail command doesn't output the log content by default. You need to specify --output text to print the log events.
The --follow parameter won't work within the loop, as it will just keep waiting for new log data. You need to tail without follow initially, then add follow after the loop.

```
exec_out=$(aws imagebuilder start-image-pipeline-execution --image-pipeline-arn $pipeline_arn)
imgBldr_arn=$(echo "${exec_out}" | jq -r '.imageBuildVersionArn')

log_group_name="/aws/imagebuilder/my-img-pipeline" 
log_stream_name_prefix="1.0.0/1"

while true; do

echo "Checking status..."
  
  arn_out=$(aws imagebuilder get-image --image-build-version-arn "${imgBldr_arn}")
  status=$(echo "${arn_out}" | jq -r '.image.state.status')

echo "Current status: ${status}"

# Print logs
  aws logs tail ${log_group_name} --log-stream-name-prefix ${log_stream_name_prefix} --output text

if [ "${status}" == "AVAILABLE" ]; then
      echo "Build completed successfully!"  
      break
  fi

sleep 30
done

# Follow logs for any trailing output
aws logs tail ${log_group_name} --log-stream-name-prefix ${log_stream_name_prefix} --follow --output text
```

Get Real Time CloudWatch Logs for EC2 Image Builder Pipeline Execution in Shell Script Using AWS CLI

Relevant content