1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
All logs will show when you enable "Continuous logging" in the job settings.
Err log really shows all the Spark/Glue logging. If you don't see those scheduler log entries is probably because they are omitted, by default most INFO logging are disabled.
You can try chaging sparkContext.setLogLevel()
관련 콘텐츠
AWS 공식업데이트됨 3년 전- AWS 공식업데이트됨 일 년 전
When you are configuring your glue job and if it is a spark job, you can mention the spark log path in your job configuration. Also if you are having simple etl pipelines, try glue studio and under the config, you can set up the log files path to S3 and when a job is failed or success, you can see the logs. Error logs is important when the job is failed so that you can track the issue.