Here are some ideas how you could manage these on-demand jobs better:
- If you are just running the container to execute one time algorithm then AWS Batch might be better way for you to launch and manage those containers.
- If you don't want implement AWS Batch and you like your current ECS setup. Then you should consider AWS Fargate with ECS. This would launch your on-demand Tasks as individual compute and you wouldn't need to manage EC2 capacity.
- ECS also just launched new feature called scale-in protection. So you could tag the ECS Task running your algorythm as protected. This would prevent the capacity provider from scaling that specific EC2 instance down. You would have to implement this protection and removal of protection as part of your web service.
Fail to start an EC2 task on ECSasked 7 months ago
ECS Capacity Provider Auto-Scaler Instance SelectionAccepted Answerasked 6 months ago
ECS: Understanding of CapacityProviderReservationasked 8 months ago
ECS: Capacity Provider vs Autoscaling Groupasked a year ago
How to have ECS map ports automatically when a task is deployed in EC2?asked 2 years ago
Should ECS Service Task start be triggered by ASG capacity 0->1?Accepted Answerasked a year ago
ECS cluster with EC2 capacity provider is left with instances having only daemon task runningasked 25 days ago
ECS Capacity Provider Scale In Timeasked 3 months ago
ECS: Autoscaling scale down the wrong instanceasked 22 days ago