- Newest
- Most votes
- Most comments
I am going to assume in my answer that these tasks are in different definitions (i.e. you have a task definition for small jobs and another / several other one(s) for big jobs needing the 16GB of RAM).
In which case, I would almost simply create a different ASG (used as different cluster capacity providers) and set the capacity provider "per service" (i.e. have one for tasks from 1 to 8GB of RAM, another one for 16GB of RAM and more, and if need be, one for in-between). Given the ASG is driven by ECS, you either only have to have 1 Launch Template per ASG or 1 Launch Template re-used with overrides (3 different ones is probably best).
Now I get that, given you had the placement strategy, that this seems overkill, but certainly it would prevent ECS to deploy small instances in a race condition (i.e, get an instance type for small tasks first) and instead clearly isolate the ASG for the bigger jobs.
This also could help future proofing configuration: if you needed bigger disks for the bigger jobs, or GPUs, then you only have the one LT/ASG to change (and reflect into the capacity provider) instead of changing it for all of them or paying GPU instances for tasks that do not need these.
The good thing of capacity providers is that you can define these both into the cluster and "override" at the service level, so you could have 4 capacity providers in your cluster, default to one of them (base/weight assignment) and at the same time have some services specifically use their own configuration.
Hope this helps :)
For example, using x-ecs you can do the following.
services: small-jobs-service: deploy: resources: limits: memory: 2GB reservations: memory: 1GB x-ecs: CapacityProviderStrategy: - CapacityProvider: SMALL-ASG Base: 1 Weight: 2 - CapacityProvider: MEDIUM-ASG Base: 4 Weight: 8 medium-jobs-service: # No override, use cluster defaults. deploy: resources: reservations: memory: 6GB big-jobs-service: deploy: resources: reservations: memory: 12GB x-ecs: CapacityProviderStrategy: - CapacityProvider: BIG-ASG Base: 1 Weight: 2
Relevant content
- Accepted Answerasked 2 months ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
+1 to this suggestion. The capacity provider doesn't tell the ASG anything about the instance requirements, it just increases the CapacityProviderReservation metric so that the ASG scales the correct amount of total capacity. This means there's no way for ECS to influence which instances get picked (and in reality, the ASG doesn't actually control this either, it passes along the settings of the MixedInstancePolicy and EC2 picks the instance types)
This is very helpful. I think I was running with the assumption that the resources needs were being propagated from ECS tasks to the asg. This suggestions seems like a good way to achieve what I am looking for. Thank you!