1回答
- 新しい順
- 投票が多い順
- コメントが多い順
0
All logs will show when you enable "Continuous logging" in the job settings.
Err log really shows all the Spark/Glue logging. If you don't see those scheduler log entries is probably because they are omitted, by default most INFO logging are disabled.
You can try chaging sparkContext.setLogLevel()
関連するコンテンツ
AWS公式更新しました 2年前- AWS公式更新しました 1年前
- AWS公式更新しました 3年前
- AWS公式更新しました 3年前
When you are configuring your glue job and if it is a spark job, you can mention the spark log path in your job configuration. Also if you are having simple etl pipelines, try glue studio and under the config, you can set up the log files path to S3 and when a job is failed or success, you can see the logs. Error logs is important when the job is failed so that you can track the issue.