EventBridge pipes and ECS Task

Question

In EventBridge Pipes, how do you invoke ECS Task with parameters ?

My use case is to trigger an ECS Task when a message arrives in a SQS Queue.

According to documentation we can  
> use InputTransformers to shape the input event to a Target to match the Amazon ECS RunTask TaskOverride structure

I tried different ways (by overriding the command or the environment variables), but without success : the task is trigger, but overrides are not taken into account.

My Target Input Transformer for container command : 
```json
{
  "containerOverrides": [{
    "name": "my-container-name",
    "command": ["echo", "test"]
  }]
}
```

Regards,

David Bruce · Accepted Answer

The input transformer is used to reshape the incoming message. The ECS Task doesn't specifically do anything with this. Instead, if you want to provide container overrides then you need to update the EcsParameters for the target of the pipe. This doesn't seem to be editable in the console, but can be done from the CLI.

First, describe the pipe you wish to update the parameters for:

```
aws pipes describe-pipe --name test-pipe --query '{Name: Name, RoleArn: RoleArn, TargetParameters: TargetParameters}'
{
    "Name": "test-pipe",
    "RoleArn": "arn:aws:iam::123456789012:role/service-role/Amazon_EventBridge_Pipe_test-pipe",
    "TargetParameters": {
        "EcsTaskParameters": {
            "EnableECSManagedTags": true,
            "EnableExecuteCommand": false,
            "LaunchType": "FARGATE",
            "NetworkConfiguration": {
                "awsvpcConfiguration": {
                    "AssignPublicIp": "DISABLED",
                }
            },
            "TaskCount": 1,
            "TaskDefinitionArn": "arn:aws:ecs:us-east-1:123456789012:task-definition/test-test"
        }
    }
}
```

I've used `--query` to only select the three top level parameters that we need to pass to update pipe. Save this data out into a file and you can add the container override to this. For example if we want to have the following override then edit the file so it looks like this:

```
{
    "Name": "test-pipe",
    "RoleArn": "arn:aws:iam::123456789012:role/service-role/Amazon_EventBridge_Pipe_test-pipe",
    "TargetParameters": {
        "EcsTaskParameters": {
            "EnableECSManagedTags": true,
            "EnableExecuteCommand": false,
            "LaunchType": "FARGATE",
            "NetworkConfiguration": {
                "awsvpcConfiguration": {
                    "AssignPublicIp": "DISABLED",
                }
            },
            "Overrides": {
                "ContainerOverrides": [
                    {
                        "Command": [
                            "echo",
                            "test"
                        ],
                        "Name": "bash"
                    }
                ]
            },
            "TaskCount": 1,
            "TaskDefinitionArn": "arn:aws:ecs:us-east-1:123456789012:task-definition/test-test"
        }
    }
}
```

If we save this into a file called `update.json` then we can update the pipe using the following command:

```
aws pipes update-pipe --cli-input-json file://update.json
{
    "Arn": "arn:aws:pipes:us-east-1:123456789012:pipe/test-pipe",
    "CreationTime": "2023-06-30T16:10:27+02:00",
    "CurrentState": "UPDATING",
    "DesiredState": "RUNNING",
    "LastModifiedTime": "2023-06-30T16:49:19+02:00",
    "Name": "test-pipe"
}
```

You can see this configuration in the UI, though can't edit it.

If you want to dynamically change the overrides using parts of the message you can do so. Instead of a static parameter you use a JSON Path string to select part of the message.

So for example, if we wanted to use a field called `message` in the body of an SQS message we would write `$.body.message`.

The update.json would look like this:

```
{
    "Name": "test-pipe",
    "RoleArn": "arn:aws:iam::123456789012:role/service-role/Amazon_EventBridge_Pipe_test-pipe",
    "TargetParameters": {
        "EcsTaskParameters": {
            "EnableECSManagedTags": true,
            "EnableExecuteCommand": false,
            "LaunchType": "FARGATE",
            "NetworkConfiguration": {
                "awsvpcConfiguration": {
                    "AssignPublicIp": "DISABLED",
                }
            },
            "Overrides": {
                "ContainerOverrides": [
                    {
                        "Command": [
                            "echo",
                            "$.body.message"
                        ],
                        "Name": "bash"
                    }
                ]
            },
            "TaskCount": 1,
            "TaskDefinitionArn": "arn:aws:ecs:us-east-1:123456789012:task-definition/test-test"
        }
    }
}
```

Note that all of the ECS Parameters specified here can use these dynamic parameters. See the dynamic parameters section on this page for more information: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-target.html

EventBridge pipes and ECS Task

Relevant content