How to pass parameters from an event rule through a glue workflow trigger to a job
I have an event rule rule that triggers a glue job. I would like to pass information from the event details as a parameter to the Glue job. For example, say my event contains: {"details": {"database_name": "my_database"}} and and my job has a parameter --DATABASE_NAME. Is it possible to pass database_name from the event as the --DATABASE_NAME parameter in the job? I haven't been able to find anything in the documentation or online on how to do it.
Thanks!
To pass a parameter to a workflow, use StartWorkflowRun[1] API with RunProperties which takes key-value pairs and it can be accessed in the any job in the workflow using GetWorkflowRunProperties[2]. Further, if required, it can be modified in any job in the workflow using PutWorkflowRunProperties[3].
If triggering an ETL job instead of a workflow, use StartJobRun[4] and set the job arguments. To access these job arguments in the script, use getResolvedOptions[5]
Relevant questions
Cant trigger a rule for a job execution update (UpdateJobExecution)
asked 3 years agoBoto3 Lambda - Existing custom IoT Rule as event source
asked 4 months agoAre EventBridge events (fully) passed to Glue Workflows?
asked 5 months agotrigger glue job from s3
Accepted Answerasked 3 months agoCall a glue Job from within another without using vpc endppoints or SG
asked 5 months agoGlue Workflow: Add same glue job multiple times, each with different parameters
asked 9 days agoHow to pass parameters from an event rule through a glue workflow trigger to a job
asked a month agoHow to convince EventBridge to pass S3 event information to an ecs target?
Accepted AnswerInvoke specified CloudWatch Rule on MediaConvert Event
asked 3 years agosend email from glue job
Accepted Answerasked 5 months ago