Unable to pass aws.events.event.json in input transformer template

0

I am using eventbridge to trigger batch run through custom put-events request(using boto3). My request is as follows :-

INPUT_DATA = {
    "data":[List<Dict>], # i.e. data key has list value where each list item is dict
    "sample_key":"sample_string_value"
}
client = boto3.client("events")
response = client.put_events(Entries=[
    {
        "Source": "my.app.com",
        "Detail": json.dumps(INPUT_DATA),
        "DetailType":"sample-type",
        "EventBusName":"my-event-bus"
    }
])

In order to pass data from eventbridge to batch, I have following input transformer template :-

input_paths = {
}

input_template = <<EOF
{
  "ContainerOverrides": {
    "Environment": [
      {
        "Name": "PAYLOAD",
        "Value": "<aws.events.event.json>"
      }
    ]
  }
}
EOF

where aws.events.event.json is pre-defined variable for event payload(i.e. detail key value) as string.

When I run the script, I am able to see metric for triggeredRulewith 1 but invocation for batch run fails i.e. failedInvocations count as 1 is also seen in the metrics. At the same time, if replace aws.events.event.json with aws.events.rule-arn or aws.events.rule-name or aws.events.event.ingestion-time, batch run is triggered and correct values of latter variables is also seen in job descriptions's environment section.

I tried to refer similar issue here but it does not seem solve the issue.

Can someone suggest where is the issue in above input transformer while using aws.events.event.json ? Is it due format of INPUT_DATA that I am sending ? Would appreciate any hint.

Thanks

1 Answer
0

Not sure on this but can you try this template and see if it works?

input_template = <<EOF
{
  "ContainerOverrides": {
    "Environment": [
      {
        "Name": "PAYLOAD",
        "Value": <aws.events.event.json>
      }
    ]
  }
}
EOF

I've removed the quotes around <aws.events.event.json> to avoid an escaping issues like the ones mentioned in the last line of this doc https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-transform-target-input.html#eb-transform-input-issues

EventBridge doesn't escape values extracted by Input Path, when populating the Input Template for a target.

EventBridge also doesn't unescape to when aws.events.event.json is "{\"a\":\"b\"}", the outcoming transformation may make "Value": "<aws.events.event.json>" to "Value": ""{\"a\":\"b\"}"". This will be an invalid JSON and lead to failureInvocation metric.

If this wasn't the issue, then I'd recommend checking if there's any other unescaped quotes in the event payload. The easiest way would be to send your events to CloudWatch Log target and inspect the event there.

profile pictureAWS
answered 2 years ago
  • Invocation is still failing. When I loaded my event on Cloudwatch, it looks like this:-

    {
        "version": "0",
        "id": "<event-id-here>",
        "detail-type": "sample-type",
        "source": "my.app.com",
        "detail": {
            "data": [List<Dict>],
            "sample_key": <Dict>,
            "sample_key_2": "sample_value"
        }
    }
  • and there's no escaped quotes in the data or sample_key ?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions