By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Does ECS have a limitation where only one task can run on the same EC2 instance?

0

Hello, I have a question regarding ECS. I've noticed that while using ECS, even if an EC2 instance has sufficient CPU and memory, it doesn't allow a second task to run on it. For example:

Task A is defined with CPU: 1024 and memory: 2048. Launching EC2 instance type: t2.2xlarge Availability Zones: 1a, 1c

Initially, running 1 task A consumes CPU and memory as shown in the Capacity Providers interface, with ample resources remaining. Updating the service, adjusting Desired tasks from 1 to 3.

Through the above operations, I observed that Capacity Providers deploy 3 instances to run 3 tasks, instead of running 3 tasks on 1 instance or 2 instances. Does this mean that even with sufficient resources, only one task can run on one EC2 instance? Therefore, should we adjust the size of the EC2 instance precisely to avoid wasting excess resources?

Topics
Containers
Tags
Amazon Elastic Container Service
Language
English
kerokero
asked 13 days ago115 views
2 Answers
2
Accepted Answer

Hi Kerokero,

Answering your question, by default, for tasks running as part of an Amazon ECS service, the task placement strategy is spread using the attribute:ecs.availability-zone. Which means that, if you have 1 task and would like to run 3 instead, the ECS will place the new 2 tasks into the other available AZ's configured on the service on best effort basis. This is following the well architect framework recommendation.

If you are looking for run tasks in the same instance as much as you can, you would need to change the placement strategy to binpack instead. On binpack, tasks are placed on container instances so as to leave the least amount of unused CPU or memory. This strategy minimizes the number of container instances in use.

When this strategy is used and a scale-in action is taken, Amazon ECS terminates tasks. It does this based on the amount of resources that are left on the container instance after the task is terminated. The container instance that has the most available resources left after task termination has that task terminated.

Please, take a read on my comment about ECS Fargate as well since it seems that you didn't have the right information while considering ECS EC2.

profile pictureAWS
Henrique Santana
answered 13 days ago
profile picture
EXPERT
Oleksii Bebych
reviewed 9 hours ago
  • kerokero
    13 days ago

    Hi Henrique,

    Thank you for your response and comments. Regarding the task placement strategy you mentioned, I will test it to see if it aligns with my expectations.

    Additionally, in response to your comments:

    1. To my knowledge, Fargate Graviton belongs to the ARM64 architecture, which may not fall within the scope of consideration as the application is expected to run on x86/x64 architecture.
    2. My understanding is that Fargate operates within a serverless framework, thus causing the machine and program to stop when not in use. Therefore, when in use, it's akin to restarting the application, leading to a reconnection to the database. Is this understanding correct?
    3. If operating under the awsvpc network mode, both EC2 and Fargate indeed require the addition of a NAT gateway in the public subnet. However, EC2 has the option to choose other network modes, such as bridge. In this case, placing EC2 in the public subnet alone allows for external connectivity.

    Please correct me if there are any mistakes.

  • Henrique Santana
    13 days ago

    Quickly answering your points:

    1. You are correct. The application should support ARM64. If so, you can upload multi-architecture images or images that have ARM64 in your image manifest. More info on the Announcing AWS Graviton2 Support for AWS Fargate – Get up to 40% Better Price-Performance for Your Serverless Containers blog post.
    2. This is not true for Fargate. You described how AWS Lambda works. For Fargate, if you are running the tasks as part of a service, it will keep running the desired number of task all the time.
    3. Yes, it is true that you can have different networking mode. But, if you can place the EC2 to a subnet with internet access, you can do the same for the tasks running on Fargate. Regardless if EC2 or Fargate, my recommendation is to run the workload in a private subnet with NATGW. More about this recommendation on our ECS Best Practice guide
  • kerokero
    13 days ago

    Thank you for your prompt response.

    Regarding the second point, should I understand that if using Fargate, it consistently maintains the service running, thereby avoiding disconnection from the database and ensuring a continuous connection, similar to running on EC2?

  • Henrique Santana
    13 days ago

    Exactly the same as ECS with EC2. The difference is that you don't need to manage the underlying OS. There are some considerations that you also should be aware of. But, none of them are about disconnecting or terminating the task if there is no ongoing connection.

  • kerokero
    10 days ago

    Hi Henrique,

    After several rounds of experimentation, it appears that adjusting the task placement strategy to Binpack indeed enables the execution of the same task on the same EC2 instance. However, I've come across some observations that have left me somewhat puzzled. Specifically, when I set the task count to 5, it triggers the initiation of 3 EC2 instances, despite one EC2 instance having adequate resources to accommodate all 5 tasks. This discrepancy has raised doubts in my mind regarding the Binpack strategy. Could there be certain limitations inherent in Binpack leading to such behavior?

    Expected Outcome: If one resource is sufficient, then all 5 tasks should be placed on the same EC2 instance: EC2 A -> 5 tasks

    Actual Outcome: It results in the activation of 2 additional EC2 instances, distributing the 5 tasks evenly: EC2 A -> 2 tasks EC2 B -> 2 tasks EC2 C -> 1 task

1

Hi,

Why don't you use the AWS-managed version of ECS named Fargate ? It will manage required capacity for you. You will pay only what you use and won't have excess capacity to pay.

See https://aws.amazon.com/fargate/

Best,

Didier

profile pictureAWS
EXPERT
Didier_Durand
answered 13 days ago
  • kerokero
    13 days ago

    Hi, Didier I've decided not to use Fargate due to the following considerations:

    1. In scenarios involving heavy usage, I anticipate that Fargate costs would exceed those of EC2.
    2. I prefer not to have to reconnect to the database every time, as I understand that Fargate might establish and terminate connections to the database dynamically.
    3. Fargate seems to only support the awsvpc network mode. From what I gather, using the awsvpc network mode with Fargate may require additional NAT gateways if connectivity to external services is needed.

    These are the points I've considered based on official reference documents and online articles. Please feel free to correct me if I'm mistaken.

  • Henrique Santana
    13 days ago

    Hi Kerokero,

    1. It really depends. How heavy you are thinking? Also, did you know that you have Fargate Graviton which reduces the cost up to 40% with better performance? There is also the SPOT option if it is not a critical application or tolerate down time.
    2. Fargate doesn't manage database connection. It is all about the application. You can think the Fargate as a managed EC2 which we don't need to be concerned about patch or AMI upgrades.
    3. There is no need of additional NAT GW. You will need to have a NATGW or IGW configured in your subnet. This is true for EC2 and Fargate.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content