Unable to correctly configure InputTemplate json for Eventbridge PutTarget operation

0

Hi there,

I'm working on dynamically setting ECS container task environment variables by extracting values from the event that triggers the task via EventBridge.

I'm able to extract individual values from the Event detail attribute and pass that to the container, but I keep getting errors when trying to pass along the entire detail object.

I've set up a deadletter queue to help debug the eventbridge rule and target execution, but haven't yet found anything actionable. I've also tried this question (https://repost.aws/questions/QUYyKFM5vWQMmIECUXsx11Jw/unable-to-pass-aws-events-event-json-in-input-transformer-template) and this documentation (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-transform-target-input.html#eb-transform-input-issues) to verify what I'm trying to accomplish but have had no luck as of yet.

I'm using the Python Boto3 library to add the target for my eventbridge rule.

template = '{{"containerOverrides": [{{"name": "{}", "environment": [{{"name": "PAYLOAD", "value": "TEST VALUE"}}, {{"name": "CLIENT", "value": <client>}}, {{"name": "DETAIL", "value": <detail>}}]}}]}}'.format(container_name)

config = {
    'Id': target_id,
    'Arn': ecs_cluster_arn,
    'RoleArn': eventbridge_role_arn,
    'InputTransformer': {
        'InputPathsMap': {
            'client': '$.detail.CLIENT',
            'detail': '$.detail'
        },
        'InputTemplate': template
    },
    ... # commented out for brevity
}

response = self.__client.put_targets(
    Rule=rule_name,
    Targets=[config]
)

I don't run into any issues when creating the rule target; however, when attempting to test the rule with a PutEvents command from the AWS CLI, the task fails to be invoked and the event goes directly to the deadletter queue.

When debugging my failed task invocation via the dead-letter queue, I see the detail object in my event, yet am getting INVALID JSON errors when the json actually looks okay to me.

{'Records': [{'messageId': '17aa13e7-e3da-4e7f-808b-657493faccd2', 'receiptHandle': 'AQEBmUP+EPXVt6dMdreKBgrl24o1+e6hnGB+Ijq88q/AjB8NIgaGb5bq0fKH1U4zV8grzW4vIXHPwRudQYMzIGUVlJScSYv+E0pFFTZfoL1gMCRArjXynabxEvirS2WkMzJlwvG7fV9vtFtG1qu340Jenujy8QBoAD4JXj4pm6VJ9LEFTNcHY5X6koSgIJpROki+2pzpS5zo8cNP2PICDP8lBpIPUTGnzt7X/KRe/+mKUvA7L9r6FWgo9z3lDQchR7FVLTQ+dFj6J0bVYlHd4ff1lvsC9hIJ35JBp/Vi2M0FPK2VBZD3zLZf7uTOChHK40ucNAPad5SIyubcRePNars2lN9pJzRLGv/SuoGZDxP7tMU8B1VcWkyTd8ukoLRTUb1l8moyoML+s/KlM3p9jEnF73nsetfaPJeOEA2I9zSgOxo=', 'body': '{"containerOverrides": [{"name": "dataplatform-job-1000", "environment": [{"name": "PAYLOAD", "value": "TEST VALUE"}, {"name": "CLIENT", "value": "MERCER"}, {"name": "DETAIL", "value": {"JOB_NAME":"RUN_MODEL_CHANGE_TRACKING","CLIENT":"MERCER"}}]}]}', 'attributes': {'ApproximateReceiveCount': '1', 'SentTimestamp': '1704214692999', 'SenderId': 'AIDAIE6VAXUQU2DKHA7DK', 'ApproximateFirstReceiveTimestamp': '1704214693001'}, 'messageAttributes': {'RULE_ARN': {'stringValue': 'arn:aws:events:us-west-2:871137929234:rule/dataplatform-job-1000-custom', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'TARGET_ARN': {'stringValue': 'arn:aws:ecs:us-west-2:871137929234:cluster/dataplatform-dev', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_MESSAGE': {'stringValue': 'Invalid input for target.', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_CODE': {'stringValue': 'INVALID_JSON', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}}, 'md5OfMessageAttributes': 'dfb37a1907310a2f33a09ec242208633', 'md5OfBody': 'e59eb72c7e4eb650639245425e62496f', 'eventSource': 'aws:sqs', 'eventSourceARN': 'arn:aws:sqs:us-west-2:871137929234:dataplatform-deadletter-queue-dev', 'awsRegion': 'us-west-2'}]}

I feel like there needs to be some escaping done but I can't identify where.

Any help is appreciated!


As a follow-up:

According to ECS run task docs, the ContainerOverrides -> Environment object should be comprised of KeyValue pairs (https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerOverride.html) and the values should be a string type. Is there a way to force objects (like the $.detail object above) to be escaped strings instead of passed as an object? I'm wondering if that may be an underlying issue.

Chris
質問済み 4ヶ月前250ビュー
1回答
0

Hello.

I think it is necessary to escape double quotes when JSON is stored as a string within JSON.
So, escape the JSON double quotes in 'body' and try running the test event with this.

{'Records': [{'messageId': '17aa13e7-e3da-4e7f-808b-657493faccd2', 'receiptHandle': 'AQEBmUP+EPXVt6dMdreKBgrl24o1+e6hnGB+Ijq88q/AjB8NIgaGb5bq0fKH1U4zV8grzW4vIXHPwRudQYMzIGUVlJScSYv+E0pFFTZfoL1gMCRArjXynabxEvirS2WkMzJlwvG7fV9vtFtG1qu340Jenujy8QBoAD4JXj4pm6VJ9LEFTNcHY5X6koSgIJpROki+2pzpS5zo8cNP2PICDP8lBpIPUTGnzt7X/KRe/+mKUvA7L9r6FWgo9z3lDQchR7FVLTQ+dFj6J0bVYlHd4ff1lvsC9hIJ35JBp/Vi2M0FPK2VBZD3zLZf7uTOChHK40ucNAPad5SIyubcRePNars2lN9pJzRLGv/SuoGZDxP7tMU8B1VcWkyTd8ukoLRTUb1l8moyoML+s/KlM3p9jEnF73nsetfaPJeOEA2I9zSgOxo=', 'body': '{\"containerOverrides\": [{\"name\": \"dataplatform-job-1000\", \"environment\": [{\"name\": \"PAYLOAD\", \"value\": \"TEST VALUE\"}, {\"name\": \"CLIENT\", \"value\": \"MERCER\"}, {\"name\": \"DETAIL\", \"value\": {\"JOB_NAME\":\"RUN_MODEL_CHANGE_TRACKING\",\"CLIENT\":\"MERCER\"}}]}]}', 'attributes': {'ApproximateReceiveCount': '1', 'SentTimestamp': '1704214692999', 'SenderId': 'AIDAIE6VAXUQU2DKHA7DK', 'ApproximateFirstReceiveTimestamp': '1704214693001'}, 'messageAttributes': {'RULE_ARN': {'stringValue': 'arn:aws:events:us-west-2:871137929234:rule/dataplatform-job-1000-custom', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'TARGET_ARN': {'stringValue': 'arn:aws:ecs:us-west-2:871137929234:cluster/dataplatform-dev', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_MESSAGE': {'stringValue': 'Invalid input for target.', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_CODE': {'stringValue': 'INVALID_JSON', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}}, 'md5OfMessageAttributes': 'dfb37a1907310a2f33a09ec242208633', 'md5OfBody': 'e59eb72c7e4eb650639245425e62496f', 'eventSource': 'aws:sqs', 'eventSourceARN': 'arn:aws:sqs:us-west-2:871137929234:dataplatform-deadletter-queue-dev', 'awsRegion': 'us-west-2'}]}
profile picture
エキスパート
回答済み 4ヶ月前
  • I agree that there needs to be some escaping done. But how can I accomplish that using the InputTransformer field (InputTemplate) as shown in my example above?

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ