By using AWS re:Post, you agree to the Terms of Use

AWS Batch limit number of instances

0

When using AWS Batch is there a way to limit the maximum number of EC2 instances that AWS Batch will have running at one time? Ideally, this would apply across all compute instances. Does AWS Batch enable this or are there other AWS services that can be combined with AWS Batch to allow us to achieve instance based limits?

An example would be say I am running AWS Batch for some workload. But I want AWS Batch to at most have 10 EC2 instances running. Is there somewhere that I can configure AWS Batch such that it will respect that limit on EC2 instances?

asked 5 months ago94 views
2 Answers
1
Accepted Answer

There is no direct configuration which can limit the number of instances. However you can configure Max vCPU in compute environment such that it can launch the instances upto required numbers only.

For example : Suppose you create a compute environment with instance type as m4.large ( it comes with 2vCPU and 8 GiB memory). Now if you set the MAX vCPU as 20 for this compute environment, so this compute environment can scale max upto 20 vCPU means 10 instances of type m4.large.

https://docs.aws.amazon.com/batch/latest/userguide/create-compute-environment.html#create-compute-environment-managed-ec2

SUPPORT ENGINEER
answered 5 months ago
  • Thank you for responding and confirming that the direct configuration does not exist at the moment. I looked into the max vCPUs approach but it breaks down once we have instance types with differing amounts of vCPUs as that can no longer be a direct proxy to instance count.

1

If I understand your question, you should be able to limit the number nodes when you're setting up a multi-node parallel job. >See #8 in multi-node parallel job definition.< This is per job, not across all EC2 instances.

I'm not aware of a convenient way to limit EC2 instances across an account without restrictions on users & services (e.g. simply disallowing creation as a policy).

answered 5 months ago
  • Thank you for your response. I'm loooking more for limiting how many instances AWS Batch will create and run at a single time. Say I only want batch to have 10 instances maximum at a given time.

    I've added this example to the main question

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions