Running each BATCH job on dedicated instance. How to?

0

Here is an example of a BATCH setup.

  sampleBatch:
    Type: AWS::Batch::ComputeEnvironment
    Properties:
      State: ENABLED
      Type: MANAGED
      ServiceRole: !GetAtt awsBatchServiceRole.Arn
      ComputeResources:
        MaxvCpus: 20
        MinvCpus: 0
        SecurityGroupIds:
          - !Join [ ",", !Ref "securityGroups" ]
        Type: EC2
        ImageId: !Ref batchAmiImage
        InstanceRole: !GetAtt ecsInstanceProfile.Arn
        InstanceTypes:
          - m4.large
        Ec2KeyPair: !Ref ec2KeyPair
        Subnets: !Ref subnets
        Tags:
          Name: "Testing Batch Env"

  sampleBatchJobQueue:
    Type: "AWS::Batch::JobQueue"
    Properties:
      ComputeEnvironmentOrder:
        - Order: 1
          ComputeEnvironment: !Ref sampleBatch
      State: ENABLED
      Priority: 1000

  sampleCnvBatchJobDefinition:
    Type: "AWS::Batch::JobDefinition"
    Properties:
      Type: container
      ContainerProperties:
        JobRoleArn: !Ref ecsTaskRole
        Image: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${dockerCnvCallerRepository}:latest"
        Vcpus: 2
        Memory: 2000
        MountPoints:
          - ReadOnly: false
            SourceVolume: genomics
            ContainerPath: /genomics
        Volumes:
          - Host:
              SourcePath: /genomics
            Name: genomics

When I submit multiple jobs, they get queued.
What I want to achieve is that Batch start new instance for each job.

I cannot find a combination of settings that will allow me to do this.

Any advice welcome.

asked 4 years ago50 views
2 Answers
0

The solution is to wait a little bit in order to let Batch launch more instances = )

answered 4 years ago
0

I accomplished this by defining a job environment with a specific instance type. Then the job definition used that job environment with the specific cpu/memory requirements of that instance. This always ensured each job would run on its own instance.

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions