1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
All logs will show when you enable "Continuous logging" in the job settings.
Err log really shows all the Spark/Glue logging. If you don't see those scheduler log entries is probably because they are omitted, by default most INFO logging are disabled.
You can try chaging sparkContext.setLogLevel()
Contenus pertinents
- demandé il y a un an
- demandé il y a un an
- demandé il y a 14 jours
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 10 mois
- AWS OFFICIELA mis à jour il y a 3 ans
When you are configuring your glue job and if it is a spark job, you can mention the spark log path in your job configuration. Also if you are having simple etl pipelines, try glue studio and under the config, you can set up the log files path to S3 and when a job is failed or success, you can see the logs. Error logs is important when the job is failed so that you can track the issue.