By using AWS re:Post, you agree to the Terms of Use

How to scale in ec2 fargate containers if job is done?


We want to run containers on an ec2 fargate cluster that will run jobs of an arbitrary duration (30 minutes on average). We want to scale out based on active TCP connection metrics via the AWS loadbalancer (1 tcp connection contains stream that needs to be handled by 1 container). This seems possible with cloudwatch metrics. The question is how to scale in once the job is done. We do not want to stop a container when it's still processing an active connection. It could take hours for the job to finish (and that is OK).

Is there a way we can remove a container once it is done processing (aka no active tcp connection anymore?) If the process inside the container stops, is it removed from the desired active container total?

2 Answers

If you need to trigger a Fargate task that executes a batch job and then exits wouldn't make more sense to use a pattern such as the one I am describing in this blog post? Note that in my example I submit an AWS Batch job but API GW also supports launching an ECS task (on Fargate).

The idea is that the task would launch a program that does what you need it to do and then exits (you should not worry about scaling in at all with this pattern). Need another task? Hit the API GW API to submit/launch another job/task and that's it.

answered 6 months ago

Generally speaking, systems on AWS are conservative about scale ins, compared to scale out.

You do not want your system to be scaled in immediately after it completes certain tasks.

That being said, from a long term auto scaling policies such as target tracking do a fairly good job of matching capacity with demands. If there is no connection to a container, the auto scaling scale in event should be triggered.

Also, to another point in the question, you might want to check the de-registration delay value for your ELB as well to avoid terminating active long-running connections.

answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions