ECS: Tasks deployed on a terminating EC2 instance

0

Hello,

I have an web application which runs on ECS, with autoscaling enabled on service level. The deployment are automated with CodeBuild with bluegreen configuration. Recently, we encountered error on the webapp which are caused by a bad assignation from ECS to EC2 instance which are in terminating status. Theses EC2 instances were being terminated by the autoscaling.

Is there a way to prevent ECS to assign new containers on terminating EC2 instances ?

Saisissez la description de l'image ici

已提问 3 个月前176 查看次数
2 回答
3
已接受的回答

yes there is a way. ECS provides a features called Container Instance Draining" which allows you to gracefully remove instances from your ECS cluster without disrupting tasks running on those instances.

Here's how you can enable container instance draining in ECS: Enable Container Instance Draining: In the ECS console, navigate to the "Clusters" page -> Select your cluster -> Click on the "ECS Instances" tab -> Select the instance(s) that you want to enable draining for -> Choose "Update Container Instances State" from the "Actions" dropdown menu->Select "DRAINING" from the dropdown menu and click "Update Instances State".

since you are using Auto Scaling Groups (ASGs) to manage your EC2 instances, configure the ASG to wait for instances to be drained before terminating them.Set the "Termination Policy" of the ASG to "OldestInstance" or "Default" to allow instances to be drained before termination.

Open the AWS Management Console and navigate to the EC2 Dashboard -> In the navigation pane, under "Auto Scaling", select "Auto Scaling Groups" Select the Auto Scaling Group (ASG) that manages your EC2 instances->Click on the "Details" tab, then scroll down to the "Advanced" section ->Under "Instance refresh settings", enable "Instance Refresh" and configure the desired settings for instance draining-> Ensure that the "Minimum healthy percentage" is set to an appropriate value to allow for draining

Adjust your scaling policies to ensure that new instances are launched before existing instances are terminated. This can help maintain the desired capacity of your ECS cluster during scaling events :- https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-scaling-simple-step.html

Configure health checks for your ECS tasks to ensure that unhealthy tasks are stopped and replaced with healthy ones on other instances :- https://aws.amazon.com/blogs/containers/a-deep-dive-into-amazon-ecs-task-health-and-task-replacement/

Hope it clarifies and if does I would appreciate answer to be accepted so that community can benefit for clarity, thanks ;)

profile picture
专家
已回答 3 个月前
profile picture
专家
已审核 12 天前
profile pictureAWS
专家
已审核 3 个月前
1

Thank you !

已回答 3 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则