Set cpu and memory requirements for a Fargate AWS Batch job from an AWS Cloudwatch event

0

I am trying to automate Fargate AWS Batch jobs by means of AWS Cloudwatch Events. So far, so good. I am trying to run the same job definition with different configurations. I am able to set the batch job as a cloudwatch event target. I have learned how to use the Constant (JSON text) configuration to set a parameter of the job. Thus, I can set the name parameter successfully and the job runs. However, I am not able to also set the memory and cpu settings in the Cloudwatch event. I would like to use a larger machine for a a bigger port such as Singapore, without changing the job definition. After all, at the moment it still uses the default vpcu and memory settings of the job definition.

{ 
	"Parameters": {"name":"wilhelmshaven"},  
	"ContainerOverrides": {
		"Command": ["upload_to_day.py", "-port_name","Ref::name"], 	
        "resourceRequirements": [ 
			{"type": "MEMORY", "value": "4096"},
			{"type": "VCPU", "value": "2"}
		] 
	}
}

Does any one know how to set the Constant (JSON text) configuration or input transformer correctly?

Edit: If I try the same thing using the AWS CLI, I can achieve what I would like to do.

aws batch submit-job \
        --job-name "run-wilhelmshaven" \
        --job-queue "arn:aws:batch:eu-central-1:123666072061:job-queue/upload-raw-to-day-vtexplorer" \
        --job-definition "arn:aws:batch:eu-central-1:123666072061:job-definition/upload-to-day:2" \
        --container-overrides '{"command": ["upload_to_day.py", "-port_name","wilhelmshaven"], "resourceRequirements": [{"value": "2", "type": "VCPU"}, {"value": "4096", "type": "MEMORY"}]}'
1 Antwort
1

The names of the keys in ContainerOverrides need to be capitalized. This is documented: AWS Batch Jobs as EventBridge Targets - AWS Batch

Note

The names of the members of the ContainerOverrides structure must be capitalized. For example, Command and ResourceRequirements instead of command and resourceRequirements.

Try this JSON:

{
 "Parameters": {"name": "wilhelmshaven"},
 "ContainerOverrides": {
  "Command": [
   "upload_to_day.py",
   "-port_name",
   "Ref::name"
  ],
  "ResourceRequirements": [
   {
    "Type": "MEMORY",
    "Value": "4096"
   },
   {
    "Type": "VCPU",
    "Value": "2"
   }
  ]
 }
}

Thanks,

Lee Hart AWS Documentation

AWS
beantwortet vor 2 Jahren
  • I have couple of follow-up questions here. First, it seems that this might be a little out-of-date? When I try with aws-cli/2.7.15 it complains that the parameters must be camel cased. I also followed the link in the first line of the answer and I am struggling to find and reference to the case. We are also trying to do this, but via Terraform and the AWS provider... any pointer here? I am going to likely start a new thread and/or a Stack Overflow questions.

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen