ECS tasks keeps draining for too long

I could see it might be due to ut-of-memory (OOM) error. If your tasks have a high memory/CPU utilization, then your container instance might not have enough resources to run the ECS agent, leading to disconnected agent [4]. You can choose to reserve some memory for the Amazon ECS container agent and other critical system processes on your container instances, so that your task's containers don't contend for the same memory. For more information, please see [5] for container instance memory management.

Please create a support case with ECS ARN, region, cluster name, service name, task definition name, task id, instance id and platform so that AWS team can help to stop incurring bill for task which is not stable.

References:

[1] Troubleshoot instances with failed status checks - Retrieve the system logs - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstances.html#troubleshooting-retrieve-system-logs

[2] Troubleshoot instances with failed status checks - Suggested actions - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstances.html#MemoryOOM-suggested-actions

[3] How do I change my container instance type in Amazon ECS? - https://aws.amazon.com/premiumsupport/knowledge-center/ecs-change-container-instance-type/

[4] How do I troubleshoot a disconnected Amazon ECS agent? https://aws.amazon.com/premiumsupport/knowledge-center/ecs-agent-disconnected-linux2-ami/

[5] Container Instance Memory Management - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/memory-management.html

[6] Deregister an Amazon EC2 backed container instance - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/deregister_container_instance.html