Unable to correctly configure InputTemplate json for Eventbridge PutTarget operation

0

Hi there,

I'm working on dynamically setting ECS container task environment variables by extracting values from the event that triggers the task via EventBridge.

I'm able to extract individual values from the Event detail attribute and pass that to the container, but I keep getting errors when trying to pass along the entire detail object.

I've set up a deadletter queue to help debug the eventbridge rule and target execution, but haven't yet found anything actionable. I've also tried this question (https://repost.aws/questions/QUYyKFM5vWQMmIECUXsx11Jw/unable-to-pass-aws-events-event-json-in-input-transformer-template) and this documentation (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-transform-target-input.html#eb-transform-input-issues) to verify what I'm trying to accomplish but have had no luck as of yet.

I'm using the Python Boto3 library to add the target for my eventbridge rule.

template = '{{"containerOverrides": [{{"name": "{}", "environment": [{{"name": "PAYLOAD", "value": "TEST VALUE"}}, {{"name": "CLIENT", "value": <client>}}, {{"name": "DETAIL", "value": <detail>}}]}}]}}'.format(container_name)

config = {
    'Id': target_id,
    'Arn': ecs_cluster_arn,
    'RoleArn': eventbridge_role_arn,
    'InputTransformer': {
        'InputPathsMap': {
            'client': '$.detail.CLIENT',
            'detail': '$.detail'
        },
        'InputTemplate': template
    },
    ... # commented out for brevity
}

response = self.__client.put_targets(
    Rule=rule_name,
    Targets=[config]
)

I don't run into any issues when creating the rule target; however, when attempting to test the rule with a PutEvents command from the AWS CLI, the task fails to be invoked and the event goes directly to the deadletter queue.

When debugging my failed task invocation via the dead-letter queue, I see the detail object in my event, yet am getting INVALID JSON errors when the json actually looks okay to me.

{'Records': [{'messageId': '17aa13e7-e3da-4e7f-808b-657493faccd2', 'receiptHandle': 'AQEBmUP+EPXVt6dMdreKBgrl24o1+e6hnGB+Ijq88q/AjB8NIgaGb5bq0fKH1U4zV8grzW4vIXHPwRudQYMzIGUVlJScSYv+E0pFFTZfoL1gMCRArjXynabxEvirS2WkMzJlwvG7fV9vtFtG1qu340Jenujy8QBoAD4JXj4pm6VJ9LEFTNcHY5X6koSgIJpROki+2pzpS5zo8cNP2PICDP8lBpIPUTGnzt7X/KRe/+mKUvA7L9r6FWgo9z3lDQchR7FVLTQ+dFj6J0bVYlHd4ff1lvsC9hIJ35JBp/Vi2M0FPK2VBZD3zLZf7uTOChHK40ucNAPad5SIyubcRePNars2lN9pJzRLGv/SuoGZDxP7tMU8B1VcWkyTd8ukoLRTUb1l8moyoML+s/KlM3p9jEnF73nsetfaPJeOEA2I9zSgOxo=', 'body': '{"containerOverrides": [{"name": "dataplatform-job-1000", "environment": [{"name": "PAYLOAD", "value": "TEST VALUE"}, {"name": "CLIENT", "value": "MERCER"}, {"name": "DETAIL", "value": {"JOB_NAME":"RUN_MODEL_CHANGE_TRACKING","CLIENT":"MERCER"}}]}]}', 'attributes': {'ApproximateReceiveCount': '1', 'SentTimestamp': '1704214692999', 'SenderId': 'AIDAIE6VAXUQU2DKHA7DK', 'ApproximateFirstReceiveTimestamp': '1704214693001'}, 'messageAttributes': {'RULE_ARN': {'stringValue': 'arn:aws:events:us-west-2:871137929234:rule/dataplatform-job-1000-custom', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'TARGET_ARN': {'stringValue': 'arn:aws:ecs:us-west-2:871137929234:cluster/dataplatform-dev', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_MESSAGE': {'stringValue': 'Invalid input for target.', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_CODE': {'stringValue': 'INVALID_JSON', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}}, 'md5OfMessageAttributes': 'dfb37a1907310a2f33a09ec242208633', 'md5OfBody': 'e59eb72c7e4eb650639245425e62496f', 'eventSource': 'aws:sqs', 'eventSourceARN': 'arn:aws:sqs:us-west-2:871137929234:dataplatform-deadletter-queue-dev', 'awsRegion': 'us-west-2'}]}

I feel like there needs to be some escaping done but I can't identify where.

Any help is appreciated!


As a follow-up:

According to ECS run task docs, the ContainerOverrides -> Environment object should be comprised of KeyValue pairs (https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerOverride.html) and the values should be a string type. Is there a way to force objects (like the $.detail object above) to be escaped strings instead of passed as an object? I'm wondering if that may be an underlying issue.

Chris
질문됨 4달 전250회 조회
1개 답변
0

Hello.

I think it is necessary to escape double quotes when JSON is stored as a string within JSON.
So, escape the JSON double quotes in 'body' and try running the test event with this.

{'Records': [{'messageId': '17aa13e7-e3da-4e7f-808b-657493faccd2', 'receiptHandle': 'AQEBmUP+EPXVt6dMdreKBgrl24o1+e6hnGB+Ijq88q/AjB8NIgaGb5bq0fKH1U4zV8grzW4vIXHPwRudQYMzIGUVlJScSYv+E0pFFTZfoL1gMCRArjXynabxEvirS2WkMzJlwvG7fV9vtFtG1qu340Jenujy8QBoAD4JXj4pm6VJ9LEFTNcHY5X6koSgIJpROki+2pzpS5zo8cNP2PICDP8lBpIPUTGnzt7X/KRe/+mKUvA7L9r6FWgo9z3lDQchR7FVLTQ+dFj6J0bVYlHd4ff1lvsC9hIJ35JBp/Vi2M0FPK2VBZD3zLZf7uTOChHK40ucNAPad5SIyubcRePNars2lN9pJzRLGv/SuoGZDxP7tMU8B1VcWkyTd8ukoLRTUb1l8moyoML+s/KlM3p9jEnF73nsetfaPJeOEA2I9zSgOxo=', 'body': '{\"containerOverrides\": [{\"name\": \"dataplatform-job-1000\", \"environment\": [{\"name\": \"PAYLOAD\", \"value\": \"TEST VALUE\"}, {\"name\": \"CLIENT\", \"value\": \"MERCER\"}, {\"name\": \"DETAIL\", \"value\": {\"JOB_NAME\":\"RUN_MODEL_CHANGE_TRACKING\",\"CLIENT\":\"MERCER\"}}]}]}', 'attributes': {'ApproximateReceiveCount': '1', 'SentTimestamp': '1704214692999', 'SenderId': 'AIDAIE6VAXUQU2DKHA7DK', 'ApproximateFirstReceiveTimestamp': '1704214693001'}, 'messageAttributes': {'RULE_ARN': {'stringValue': 'arn:aws:events:us-west-2:871137929234:rule/dataplatform-job-1000-custom', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'TARGET_ARN': {'stringValue': 'arn:aws:ecs:us-west-2:871137929234:cluster/dataplatform-dev', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_MESSAGE': {'stringValue': 'Invalid input for target.', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}, 'ERROR_CODE': {'stringValue': 'INVALID_JSON', 'stringListValues': [], 'binaryListValues': [], 'dataType': 'String'}}, 'md5OfMessageAttributes': 'dfb37a1907310a2f33a09ec242208633', 'md5OfBody': 'e59eb72c7e4eb650639245425e62496f', 'eventSource': 'aws:sqs', 'eventSourceARN': 'arn:aws:sqs:us-west-2:871137929234:dataplatform-deadletter-queue-dev', 'awsRegion': 'us-west-2'}]}
profile picture
전문가
답변함 4달 전
  • I agree that there needs to be some escaping done. But how can I accomplish that using the InputTransformer field (InputTemplate) as shown in my example above?

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠