Set cpu and memory requirements for a Fargate AWS Batch job from an AWS Cloudwatch event

0

I am trying to automate Fargate AWS Batch jobs by means of AWS Cloudwatch Events. So far, so good. I am trying to run the same job definition with different configurations. I am able to set the batch job as a cloudwatch event target. I have learned how to use the Constant (JSON text) configuration to set a parameter of the job. Thus, I can set the name parameter successfully and the job runs. However, I am not able to also set the memory and cpu settings in the Cloudwatch event. I would like to use a larger machine for a a bigger port such as Singapore, without changing the job definition. After all, at the moment it still uses the default vpcu and memory settings of the job definition.

{ 
	"Parameters": {"name":"wilhelmshaven"},  
	"ContainerOverrides": {
		"Command": ["upload_to_day.py", "-port_name","Ref::name"], 	
        "resourceRequirements": [ 
			{"type": "MEMORY", "value": "4096"},
			{"type": "VCPU", "value": "2"}
		] 
	}
}

Does any one know how to set the Constant (JSON text) configuration or input transformer correctly?

Edit: If I try the same thing using the AWS CLI, I can achieve what I would like to do.

aws batch submit-job \
        --job-name "run-wilhelmshaven" \
        --job-queue "arn:aws:batch:eu-central-1:123666072061:job-queue/upload-raw-to-day-vtexplorer" \
        --job-definition "arn:aws:batch:eu-central-1:123666072061:job-definition/upload-to-day:2" \
        --container-overrides '{"command": ["upload_to_day.py", "-port_name","wilhelmshaven"], "resourceRequirements": [{"value": "2", "type": "VCPU"}, {"value": "4096", "type": "MEMORY"}]}'
1 Answer
1

The names of the keys in ContainerOverrides need to be capitalized. This is documented: AWS Batch Jobs as EventBridge Targets - AWS Batch

Note

The names of the members of the ContainerOverrides structure must be capitalized. For example, Command and ResourceRequirements instead of command and resourceRequirements.

Try this JSON:

{
 "Parameters": {"name": "wilhelmshaven"},
 "ContainerOverrides": {
  "Command": [
   "upload_to_day.py",
   "-port_name",
   "Ref::name"
  ],
  "ResourceRequirements": [
   {
    "Type": "MEMORY",
    "Value": "4096"
   },
   {
    "Type": "VCPU",
    "Value": "2"
   }
  ]
 }
}

Thanks,

Lee Hart AWS Documentation

AWS
answered 2 years ago
  • I have couple of follow-up questions here. First, it seems that this might be a little out-of-date? When I try with aws-cli/2.7.15 it complains that the parameters must be camel cased. I also followed the link in the first line of the answer and I am struggling to find and reference to the case. We are also trying to do this, but via Terraform and the AWS provider... any pointer here? I am going to likely start a new thread and/or a Stack Overflow questions.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions