Unable to correctly configure InputTemplate json for Eventbridge PutTarget operation

0

Hi there,

I'm working on dynamically setting ECS container task environment variables by extracting values from the event that triggers the task via EventBridge.

I'm able to extract individual values from the Event detail attribute and pass that to the container, but I keep getting errors when trying to pass along the entire detail object.

I've set up a deadletter queue to help debug the eventbridge rule and target execution, but haven't yet found anything actionable. I've also tried this question (https://repost.aws/questions/QUYyKFM5vWQMmIECUXsx11Jw/unable-to-pass-aws-events-event-json-in-input-transformer-template) and this documentation (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-transform-target-input.html#eb-transform-input-issues) to verify what I'm trying to accomplish but have had no luck as of yet.

I'm using the Python Boto3 library to add the target for my eventbridge rule.

template = '{{"containerOverrides": [{{"name": "{}", "environment": [{{"name": "PAYLOAD", "value": "TEST VALUE"}}, {{"name": "CLIENT", "value": <client>}}, {{"name": "DETAIL", "value": <detail>}}]}}]}}'.format(container_name)

config = {
    'Id': target_id,
    'Arn': ecs_cluster_arn,
    'RoleArn': eventbridge_role_arn,
    'InputTransformer': {
        'InputPathsMap': {
            'client': '$.detail.CLIENT',
            'detail': '$.detail'
        },
        'InputTemplate': template
    },
    ... # commented out for brevity
}

response = self.__client.put_targets(
    Rule=rule_name,
    Targets=[config]
)

I don't run into any issues when creating the rule target; however, when attempting to test the rule with a PutEvents command from the AWS CLI, the task fails to be invoked and the event goes directly to the deadletter queue.

When debugging my failed task invocation via the dead-letter queue, I see the detail object in my event, yet am getting INVALID JSON errors when the json actually looks okay to me.

{'Records': [{'messageId': '17aa13e7-e3da-4e7f-808b-657493faccd2', 'receiptHandle': 'AQEBmUP+EPXVt6dMdreKBgrl24o1+e6hnGB+Ijq88q/AjB8NIgaGb5bq0fKH1U4zV8grzW4vIXHPwRudQYMzIGUVlJScSYv+E0pFFTZfoL1gMCRArjXynabxEvirS2WkMzJlwvG7fV9vtFtG1qu340Jenujy8QBoAD4JXj4pm6VJ9LEFTNcHY5X6koSgIJpROki+2pzpS5zo8cNP2PICDP8lBpIPUTGnzt7X/KRe/+mKUvA7L9r6FWgo9z3lDQchR7FVLTQ+dFj6J0bVYlHd4ff1lvsC9hIJ35JBp/Vi2M0FPK2VBZD3zLZf7uTOChHK40ucNAPad5SIyubcRePNars2lN9pJzRLGv/SuoGZDxP7tMU8B1VcWkyTd8ukoLRTUb1l8moyoML+s/KlM3p9jEnF73nsetfaPJeOEA2I9zSgOxo=', 'body': '{"containerOverrides": [{"name": "dataplatform-job-1000", "environment": [{"name": "PAYLOAD", "value": "TEST VALUE"}, {"name": "CLIENT", "value": "MERCER"}, {"name": "DETAIL", "value": {"JOB_NAME":"RUN_MODEL_CHANGE_TRACKING","CLIENT":"MERCER"}}]}]}', 'attributes': {'ApproximateReceiveCount': '1', 'SentTimestamp': '1704214692999', 'SenderId': 'AIDAIE6VAXUQU2DKHA7DK', 'ApproximateFirstReceiveTimestamp': '1704214693001'}, 'messageAttributes': {'RULE_ARN': {'stringValue': 'arn:aws:events:us-west-2:871137929234:rule/dataplatform-job-1000-custom', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'TARGET_ARN': {'stringValue': 'arn:aws:ecs:us-west-2:871137929234:cluster/dataplatform-dev', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_MESSAGE': {'stringValue': 'Invalid input for target.', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_CODE': {'stringValue': 'INVALID_JSON', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}}, 'md5OfMessageAttributes': 'dfb37a1907310a2f33a09ec242208633', 'md5OfBody': 'e59eb72c7e4eb650639245425e62496f', 'eventSource': 'aws:sqs', 'eventSourceARN': 'arn:aws:sqs:us-west-2:871137929234:dataplatform-deadletter-queue-dev', 'awsRegion': 'us-west-2'}]}

I feel like there needs to be some escaping done but I can't identify where.

Any help is appreciated!


As a follow-up:

According to ECS run task docs, the ContainerOverrides -> Environment object should be comprised of KeyValue pairs (https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerOverride.html) and the values should be a string type. Is there a way to force objects (like the $.detail object above) to be escaped strings instead of passed as an object? I'm wondering if that may be an underlying issue.

Chris
已提问 4 个月前250 查看次数
1 回答
0

Hello.

I think it is necessary to escape double quotes when JSON is stored as a string within JSON.
So, escape the JSON double quotes in 'body' and try running the test event with this.

{'Records': [{'messageId': '17aa13e7-e3da-4e7f-808b-657493faccd2', 'receiptHandle': 'AQEBmUP+EPXVt6dMdreKBgrl24o1+e6hnGB+Ijq88q/AjB8NIgaGb5bq0fKH1U4zV8grzW4vIXHPwRudQYMzIGUVlJScSYv+E0pFFTZfoL1gMCRArjXynabxEvirS2WkMzJlwvG7fV9vtFtG1qu340Jenujy8QBoAD4JXj4pm6VJ9LEFTNcHY5X6koSgIJpROki+2pzpS5zo8cNP2PICDP8lBpIPUTGnzt7X/KRe/+mKUvA7L9r6FWgo9z3lDQchR7FVLTQ+dFj6J0bVYlHd4ff1lvsC9hIJ35JBp/Vi2M0FPK2VBZD3zLZf7uTOChHK40ucNAPad5SIyubcRePNars2lN9pJzRLGv/SuoGZDxP7tMU8B1VcWkyTd8ukoLRTUb1l8moyoML+s/KlM3p9jEnF73nsetfaPJeOEA2I9zSgOxo=', 'body': '{\"containerOverrides\": [{\"name\": \"dataplatform-job-1000\", \"environment\": [{\"name\": \"PAYLOAD\", \"value\": \"TEST VALUE\"}, {\"name\": \"CLIENT\", \"value\": \"MERCER\"}, {\"name\": \"DETAIL\", \"value\": {\"JOB_NAME\":\"RUN_MODEL_CHANGE_TRACKING\",\"CLIENT\":\"MERCER\"}}]}]}', 'attributes': {'ApproximateReceiveCount': '1', 'SentTimestamp': '1704214692999', 'SenderId': 'AIDAIE6VAXUQU2DKHA7DK', 'ApproximateFirstReceiveTimestamp': '1704214693001'}, 'messageAttributes': {'RULE_ARN': {'stringValue': 'arn:aws:events:us-west-2:871137929234:rule/dataplatform-job-1000-custom', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'TARGET_ARN': {'stringValue': 'arn:aws:ecs:us-west-2:871137929234:cluster/dataplatform-dev', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_MESSAGE': {'stringValue': 'Invalid input for target.', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_CODE': {'stringValue': 'INVALID_JSON', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}}, 'md5OfMessageAttributes': 'dfb37a1907310a2f33a09ec242208633', 'md5OfBody': 'e59eb72c7e4eb650639245425e62496f', 'eventSource': 'aws:sqs', 'eventSourceARN': 'arn:aws:sqs:us-west-2:871137929234:dataplatform-deadletter-queue-dev', 'awsRegion': 'us-west-2'}]}
profile picture
专家
已回答 4 个月前
  • I agree that there needs to be some escaping done. But how can I accomplish that using the InputTransformer field (InputTemplate) as shown in my example above?

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则