By using AWS re:Post, you agree to the Terms of Use
/ECS Exec error TargetNotConnectedException/

ECS Exec error TargetNotConnectedException

0

I'm getting this error when I try to connect to my Fargate ECS container using ECS Exec feature:

An error occurred (TargetNotConnectedException) when calling the ExecuteCommand operation: The execute command failed due to an internal error. Try again later.

The task's role was missing the SSM permissions, I added them, but the error persists. Below is the output of the amazon-ecs-exec-checker tool:

Prerequisites for check-ecs-exec.sh v0.7
-------------------------------------------------------------
  jq      | OK (/usr/bin/jq)
  AWS CLI | OK (/usr/local/bin/aws)

-------------------------------------------------------------
Prerequisites for the AWS CLI to use ECS Exec
-------------------------------------------------------------
  AWS CLI Version        | OK (aws-cli/2.2.1 Python/3.8.8 Linux/5.13.0-39-generic exe/x86_64.ubuntu.20 prompt/off)
  Session Manager Plugin | OK (1.2.295.0)

-------------------------------------------------------------
Checks on ECS task and other resources
-------------------------------------------------------------
Region : us-west-2
Cluster: removed
Task   :  removed
-------------------------------------------------------------
  Cluster Configuration  | Audit Logging Not Configured
  Can I ExecuteCommand?  | arn:aws:iam::removed
     ecs:ExecuteCommand: allowed
     ssm:StartSession denied?: allowed
  Task Status            | RUNNING
  Launch Type            | Fargate
  Platform Version       | 1.4.0
  Exec Enabled for Task  | OK
  Container-Level Checks | 
    ----------
      Managed Agent Status
    ----------
         1. RUNNING for "api-staging"
    ----------
      Init Process Enabled (api-staging:14)
    ----------
         1. Enabled - "api-staging"
    ----------
      Read-Only Root Filesystem (api-staging:14)
    ----------
         1. Disabled - "api-staging"
  Task Role Permissions  | arn:aws:iam::removed
     ssmmessages:CreateControlChannel: allowed
     ssmmessages:CreateDataChannel: allowed
     ssmmessages:OpenControlChannel: allowed
     ssmmessages:OpenDataChannel: allowed
  VPC Endpoints          | 
    Found existing endpoints for vpc-removed:
      - com.amazonaws.us-west-2.s3
    SSM PrivateLink "com.amazonaws.us-west-2.ssmmessages" not found. You must ensure your task has proper outbound internet connectivity.  Environment Variables  | (api-staging:14)
       1. container "api-staging"
       - AWS_ACCESS_KEY: not defined
       - AWS_ACCESS_KEY_ID: defined
       - AWS_SECRET_ACCESS_KEY: defined

The output has only green and yellow markers, no red ones, which means Exec should work. But it doesn't. Any ideas why?

1 Answers
1

Ah, I think I know why it doesn't work. AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env vars are defined and it's likely they don't have the SSM permissions. It looks like Exec will use the env vars instead of the IAM role. I'll update my post a bit later when I fix the env vars issue.

answered 2 months ago
  • Yes, adding SSM permissions to the user behind AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env vars fixed the problem for me.

  • I had this issue also but had to add ecs:ExecuteCommand permissions to the AWS_ACCESS_KEY_ID account in addition to the SSM permissions.

  • Granting SSM permissions to the user account associated with the env var KEY_ID worked for me. Was something changed recently in Fargate so that this credential controls over the IAM role of the task? About a month ago, I was able to connect without SSM permissions attached to the internally-referenced user account.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions