1 個回答
- 最新
- 最多得票
- 最多評論
1
Hello,
I understand you would like to have access to spark ui server to trace the job performance and other details, while running jobs through Glue interactive session.
First, kindly configure the spark ui logs path as shown below before starting the session [1],
%%configure { "--enable-spark-ui": "true", "--spark-event-logs-path": "s3://<your-bucket>/sparkui/" }
Secondly, You can launch the Spark history server using a AWS CloudFormation template that hosts the server on an EC2 instance, or launch locally using Docker. Refer [2] for the steps.
References: [1] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-jobs.html [2] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html
相關內容
- AWS 官方已更新 2 年前
- AWS 官方已更新 1 年前