1回答
- 新しい順
- 投票が多い順
- コメントが多い順
1
Hello,
I understand you would like to have access to spark ui server to trace the job performance and other details, while running jobs through Glue interactive session.
First, kindly configure the spark ui logs path as shown below before starting the session [1],
%%configure { "--enable-spark-ui": "true", "--spark-event-logs-path": "s3://<your-bucket>/sparkui/" }
Secondly, You can launch the Spark history server using a AWS CloudFormation template that hosts the server on an EC2 instance, or launch locally using Docker. Refer [2] for the steps.
References: [1] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-jobs.html [2] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html
関連するコンテンツ
- 質問済み 9ヶ月前
- AWS公式更新しました 3年前