Are EventBridge events (fully) passed to Glue Workflows?

2

Starting a Glue workflow with an EventBridge event set only (as Run properties) the IDs of the events that started the workflow. Are passing the full EventBridge events supported in Glue Workflows?

preguntada hace 2 años2740 visualizaciones
2 Respuestas
0

I'm not sure I understand your question. Did you follow the instructions here?

https://docs.aws.amazon.com/glue/latest/dg/starting-workflow-eventbridge.html

profile pictureAWS
respondido hace 2 años
  • I'm referring to access to the EventBridge event details from a Glue Job included in the workflow. According to the documentation, only the event Ids (and not the event details) are shared as Run Properties (aws:eventIds) among the jobs.

  • I have the same use case i.e. to pass the Eventbridge event details to a Glue workflow (as run properties). Currently there seems to be no way to achieve this. I also tried having lambda as a target for the Eventbridge rule and starting the Glue workflow from lambda. Even in that case, the glue client "start_workflow_run" API does not have an option to pass run properties

0

There is no native option to pass EventBridge event details to Glue job. But I used the following workaround to do this:

  1. You need to get an event ID from Glue workflow properties
event_id = glue_client.get_workflow_run_properties(Name=self.args['WORKFLOW_NAME'],
                       RunId=self.args['WORKFLOW_RUN_ID'])['RunProperties']['aws:eventIds'][1:-1]
  1. Get all NotifyEvent events for the last several minutes. It's up to you to decide how much time can pass between the workflow start and your job start.
response = event_client.lookup_events(LookupAttributes=[{'AttributeKey': 'EventName',
                                                         'AttributeValue': 'NotifyEvent'}],
                                                         StartTime=(datetime.datetime.now() - datetime.timedelta(minutes=5)),
                                                         EndTime=datetime.datetime.now())['Events']
  1. Check which event has an enclosed event with the event ID we get from Glue workflow.
for i in range(len(response)):
      event_payload = json.loads(response[i]['CloudTrailEvent'])['requestParameters']['eventPayload']
      if event_payload['eventId'] == event_id:
                event = json.loads(event_payload['eventBody'])

In the event variable you get the full content of the event that triggered Glue workflow.

profile picture
respondido hace un año

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas