Processing multiple tables in one glue job.

0

I want to create a glue job to process multiple tables in parallel. If all the tables are to be processed in the same manner, is it possible to do it in only one glue job?

質問済み 8ヶ月前343ビュー
1回答
0

Yes it is possible to process multiple tables in parallel with one Glue job. You can simply call each table in a separate DynamicFrame, and you should be able to process this data.

My only recommendation is that you take into consideration how much of data is being consumed in the job and the type of transformations which are being processed. Too much data will consume more resources. You will need to provide the correct amount of workers and the appropriate worker type for your job.

You can read more about the worker types and their properties via the following documentation

[+] Glue Adding Jobs- https://docs.aws.amazon.com/glue/latest/dg/add-job.html

profile picture
エキスパート
回答済み 8ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ