How to set up / access the Spark UI for AWS Glue jobs and notebooks
0
The checkbox in a Glue job is select to enable the Spark UI, and an s3 path is specified as the location for those logs. How can I access the Spark UI URL?
1 Risposta
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
1
You need to deploy you own history server to visualize those longs, you can use a CloudFormation template, a docker container (you can also run locally a history server from a Spark distribution but adding s3 support requires a bit of work). See steps here: https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html
Contenuto pertinente
AWS UFFICIALEAggiornata 2 anni fa- AWS UFFICIALEAggiornata 3 anni fa
- AWS UFFICIALEAggiornata 3 anni fa