EventBridge and Glue Workflow

0

Hi,

I don't have much expertise with EventBridge + Glue Workflow.

I do have an AWS DMS configured to migrate our database to a S3 Bucket, I want to perform ETL on the landed data.

I can enable the Event Notification to notify whenever a file is written on the bucket, and create EventBridge rules to filter by the S3 Key.

Is it possible to multiples EventBridge rules trigger the same Glue workflow passing different parameters? Or I should have one event bridge and glue workflow for each table ?

e.g.

1st Approach

/database/table1/file1.csv -> EventBridge Rule 1 -> Glue Workflow 1
/database/table2/file2.csv -> EventBridge Rule 2 -> Glue Workflow 2

vs

2nd Approach, different events share the same glue workflow but passing different parameters.

/database/table1/file1.csv -> EventBridge Rule 1 -> Glue Workflow
/database/table2/file2.csv -> EventBridge Rule 2 -> Glue Workflow

The glue job will perform deduplication and will do the upsert on a S3 bucket using the Apache Iceberg.

Marcus
質問済み 1年前468ビュー
1回答
1

Nothing prevents you from having two different rules invoking the same target, a Glue workflow in your case. You can use Input Transformer to pass different information from each rule.

profile pictureAWS
エキスパート
Uri
回答済み 1年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ