ECS: Unable to start task from within a private subnet without enabling public IP

0

I am trying to start a simple ECS service using the nginx image. Just a single task with no LB. I launched it from a VPC with a subnet (1 of 7). The VPC is connected to a internet gateway. I configured the task to have only private IP. Oh, the run time is Fargate.

The result is that the task get stuck in "Pending" state for ever.

If I enable the automatically assigned public IP, the task runs without any problem.

What could be the problem?

asked 2 years ago4360 views
2 Answers
1

Look at step 5 here: https://docs.aws.amazon.com/AmazonECS/latest/userguide/service-configure-network.html

You need to have network access from the ECS task to ECR to retrieve the container image.

You can achieve it by assigning a public IP address to the task or having a NAT gateway in your VPC.

Alternatively, if you don't want to pull ECR image across Internet, you can also use a VPC endpoint (https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html#ecr-vpc-endpoint-considerations)

answered 2 years ago
  • Thanks for the reply. I though having an internet gateway is enough? As in, igw does what NAT Gateway does in this case?

  • Internet gateway alone is not enough, the ECS task needs a public IP address to connect to the internet.

1

As Richard said, it's just down to your networking configuration. Either you need to

  • A) run the tasks in a subnet that use the InternetGateway as the route to "all" (0.0.0.0/0) and enable EIP auto-assign on your tasks or

  • B) run containers which use a NAT Gateway to 0.0.0.0/0 as the route to 0.0.0.0/0 (container -> nat -> internet -> ECR) or

  • C) run the container in subnet(s) that have a VPC Endpoint to ECR (you need two, one for api, one for ECR) in which case you get, container -> endpoint -> ECR

If you are going to have big workloads (i.e, download images constantly, large ones in the mix) then using the VPC endpoint with ECR can reduce your bill significantly over using a NAT Gateway.

If your image is in DockerHub/Quay.io, then you must either go with A or B, but C won't work for these images, unless you create a repository in ECR and enable cachethrough, but that's "even more involved"

If you are starting with AWS and ECS, you could something simple such as, this docker-compose file, i.e. docker-compose.yaml

version: 3.8
networks:
  public:
    x-vpc: PublicSubnets

services:
  nginx:
    image: nginx # or URL to your ECR repo
    x-network:
      AssignPublicIp: True
      Ingress:
        ExtSources:
          - IPv4: 0.0.0.0/0
            Description: ANY
    networks:
      public: {}
    ports:
      - published: 80
        target: 80
        protocol: tcp

and then run (on Linux/MacOS. Not sure about the source compose-x/bin/activate on Windows)

python3 -m venv compose-x
source compose-x/bin/activate
pip install pip -U; pip install ecs-compose-x;
ecs-compose-x init
ecs-compose-x up -n my-first-ecs-app -d templates/ -f docker-compose.yaml

This will generate all the CFN templates you need to create a new VPC (without a NAT Gateway, saving some $$ for a PoC), create the service and put it in the public subnets, with AssignPublicIp set to True to get an IP address.

Hope this helps your journey. See this for more details on the above.

profile picture
answered 2 years ago
  • thanks. The subnet that the task is running in does have a default route to the igw.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions