How to convince EventBridge to pass S3 event information to an ecs target?

Question

I currently use S3 PutObject Event to trigger a Lambda Function.  This lambda function receives details of the object (such as name) and then starts an ECS task where it passes relevant details via environment variables.

I recently came across this [tutorial][1] that shows how to use CloudWatch rules (now handled by Event Bridge) to start an ECS task when a file is uploaded to S3.

The tutorial explains how to configure the S3 Event and how to configure the ECS task that will be triggered.  What i can't find is how to get details of the S3 Event into the ECS Task.
 
Is there a way to pass details from the originating S3 event into the ECS task?

[1]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatch-Events-tutorial-ECS.html

MichaelFischer · Accepted Answer

Unfortunately, EventBridge lacks the ability to pass [task definition overrides][1] to the ECS RunTask API, which would allow you to specify environment variables or override the command line when starting a task.

As a workaround, I would recommend having S3 or EventBridge invoke a Lambda function instead. Your Lambda function can then call [RunTask][2] directly and pass the desired overrides to the task.

[1]: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerOverride.html
  [2]: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RunTask.html

BruceRoberts · Answer

In case anyone gets here from a Google search, it is now possible to do task definition overrides to the ECS RunTask.  You can use the Input (or InputMap and InputTemplate) parameters to the put_targets() method to assign environment overrides to your task, e.g. an Input like (Python code):

'Input': f"{{\"containerOverrides\": [{{\"name\":\"{task_container_name}\",\"environment\":{environment_vars}}}]}}"

Various event details (e.g. from an S3 event) can be used in the InputMap/InputTemplate to send any event details to the task as environment variables.

Hope this helps someone.

NikPinski-AWSEventBridge · Answer

The answer from BruceRoberts is the correct one not - Container overrides can be provided via inputTransformers from the event payload. Shape your event payload into the JSON matching the TaskOverride object and you can use this to configure any of the parameters (including container overrides and environment variables, but not just!)

For more on EventBridge input transformers: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-transform-target-input.html

How to convince EventBridge to pass S3 event information to an ecs target?

Relevant content