Are EventBridge events (fully) passed to Glue Workflows?

2

Starting a Glue workflow with an EventBridge event set only (as Run properties) the IDs of the events that started the workflow. Are passing the full EventBridge events supported in Glue Workflows?

已提问 2 年前2710 查看次数
2 回答
0

I'm not sure I understand your question. Did you follow the instructions here?

https://docs.aws.amazon.com/glue/latest/dg/starting-workflow-eventbridge.html

profile pictureAWS
已回答 2 年前
  • I'm referring to access to the EventBridge event details from a Glue Job included in the workflow. According to the documentation, only the event Ids (and not the event details) are shared as Run Properties (aws:eventIds) among the jobs.

  • I have the same use case i.e. to pass the Eventbridge event details to a Glue workflow (as run properties). Currently there seems to be no way to achieve this. I also tried having lambda as a target for the Eventbridge rule and starting the Glue workflow from lambda. Even in that case, the glue client "start_workflow_run" API does not have an option to pass run properties

0

There is no native option to pass EventBridge event details to Glue job. But I used the following workaround to do this:

  1. You need to get an event ID from Glue workflow properties
event_id = glue_client.get_workflow_run_properties(Name=self.args['WORKFLOW_NAME'],
                       RunId=self.args['WORKFLOW_RUN_ID'])['RunProperties']['aws:eventIds'][1:-1]
  1. Get all NotifyEvent events for the last several minutes. It's up to you to decide how much time can pass between the workflow start and your job start.
response = event_client.lookup_events(LookupAttributes=[{'AttributeKey': 'EventName',
                                                         'AttributeValue': 'NotifyEvent'}],
                                                         StartTime=(datetime.datetime.now() - datetime.timedelta(minutes=5)),
                                                         EndTime=datetime.datetime.now())['Events']
  1. Check which event has an enclosed event with the event ID we get from Glue workflow.
for i in range(len(response)):
      event_payload = json.loads(response[i]['CloudTrailEvent'])['requestParameters']['eventPayload']
      if event_payload['eventId'] == event_id:
                event = json.loads(event_payload['eventBody'])

In the event variable you get the full content of the event that triggered Glue workflow.

profile picture
已回答 1 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则