1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
1
Hello,
I understand you would like to have access to spark ui server to trace the job performance and other details, while running jobs through Glue interactive session.
First, kindly configure the spark ui logs path as shown below before starting the session [1],
%%configure { "--enable-spark-ui": "true", "--spark-event-logs-path": "s3://<your-bucket>/sparkui/" }
Secondly, You can launch the Spark history server using a AWS CloudFormation template that hosts the server on an EC2 instance, or launch locally using Docker. Refer [2] for the steps.
References: [1] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-jobs.html [2] https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html
관련 콘텐츠
- AWS 공식업데이트됨 일 년 전