How to set up / access the Spark UI for AWS Glue jobs and notebooks
0
The checkbox in a Glue job is select to enable the Spark UI, and an s3 path is specified as the location for those logs. How can I access the Spark UI URL?
1 Antwort
- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
1
You need to deploy you own history server to visualize those longs, you can use a CloudFormation template, a docker container (you can also run locally a history server from a Spark distribution but adding s3 support requires a bit of work). See steps here: https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html
Relevanter Inhalt
AWS OFFICIALAktualisiert vor 3 Jahren- AWS OFFICIALAktualisiert vor 2 Jahren
- Wie behebe ich den Fehler „java.lang.OutOfMemoryError: Java heap space“ in einem AWS Glue-Spark-Job?AWS OFFICIALAktualisiert vor 2 Jahren