1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
1
Hello,
I understand you would like to have access to spark ui server to trace the job performance and other details, while running jobs through Glue interactive session.
First, kindly configure the spark ui logs path as shown below before starting the session [1],
%%configure { "--enable-spark-ui": "true", "--spark-event-logs-path": "s3://<your-bucket>/sparkui/" }
Secondly, You can launch the Spark history server using a AWS CloudFormation template that hosts the server on an EC2 instance, or launch locally using Docker. Refer [2] for the steps.
References: [1] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-jobs.html [2] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html
Contenus pertinents
- demandé il y a un an
- demandé il y a un an
- demandé il y a un an
- demandé il y a 2 mois