- Newest
- Most votes
- Most comments
I asked a similar question at re:Invent and found much of the time is cleanup / downloading the image each invocation / start of a job. In the ECS cluster you can change the pull policy to use cached image if available or validate the cached image. This would allow the pull to be FAR faster, at the risk of running an old image if you aggressively cache (if your nodes fully turn off when the batch is done this may be OK). The settings aren't in batch however.
Unless the nodes generally only run one job then terminate then the only difference between caching and preloading would be the time to run the first job.
Good luck, hope it helps.
Hello. Is this cache policy configurable if using a managed compute environment ? I have not seen such configuration
Relevant content
- asked 4 years ago
- asked 3 years ago
AWS OFFICIALUpdated 8 months ago
Did you manage to do this? I had a similar problem, and created a custom AMI with my image loaded into docker. However i am not able to strat form this one in my AWS Batch jobs that use this compute environment. Not sure how to tell Batch to use local, and not even sure it is able to see it.