Are EventBridge events (fully) passed to Glue Workflows?

2

Starting a Glue workflow with an EventBridge event set only (as Run properties) the IDs of the events that started the workflow. Are passing the full EventBridge events supported in Glue Workflows?

質問済み 2年前2699ビュー
2回答
0

I'm not sure I understand your question. Did you follow the instructions here?

https://docs.aws.amazon.com/glue/latest/dg/starting-workflow-eventbridge.html

profile pictureAWS
回答済み 2年前
  • I'm referring to access to the EventBridge event details from a Glue Job included in the workflow. According to the documentation, only the event Ids (and not the event details) are shared as Run Properties (aws:eventIds) among the jobs.

  • I have the same use case i.e. to pass the Eventbridge event details to a Glue workflow (as run properties). Currently there seems to be no way to achieve this. I also tried having lambda as a target for the Eventbridge rule and starting the Glue workflow from lambda. Even in that case, the glue client "start_workflow_run" API does not have an option to pass run properties

0

There is no native option to pass EventBridge event details to Glue job. But I used the following workaround to do this:

  1. You need to get an event ID from Glue workflow properties
event_id = glue_client.get_workflow_run_properties(Name=self.args['WORKFLOW_NAME'],
                       RunId=self.args['WORKFLOW_RUN_ID'])['RunProperties']['aws:eventIds'][1:-1]
  1. Get all NotifyEvent events for the last several minutes. It's up to you to decide how much time can pass between the workflow start and your job start.
response = event_client.lookup_events(LookupAttributes=[{'AttributeKey': 'EventName',
                                                         'AttributeValue': 'NotifyEvent'}],
                                                         StartTime=(datetime.datetime.now() - datetime.timedelta(minutes=5)),
                                                         EndTime=datetime.datetime.now())['Events']
  1. Check which event has an enclosed event with the event ID we get from Glue workflow.
for i in range(len(response)):
      event_payload = json.loads(response[i]['CloudTrailEvent'])['requestParameters']['eventPayload']
      if event_payload['eventId'] == event_id:
                event = json.loads(event_payload['eventBody'])

In the event variable you get the full content of the event that triggered Glue workflow.

profile picture
回答済み 1年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン