By using AWS re:Post, you agree to the Terms of Use

How do I access the Spark UI?

0

Hello!
So I enabled the Spark UI for a dev endpoint as described here:
https://docs.aws.amazon.com/en_pv/glue/latest/dg/monitor-spark-ui-dev-endpoints.html
After creating a GlueContext a table with a link to the Spark UI is printed in the Notebook.
The link leads to:
http://ip-172-32-2-76.eu-central-1.compute.internal:20888/proxy/application_123456789_0001/
But requesting that site with the browser yields a "Server not Found".
So, I suppose a setting is missing.
I would like to see that fancy Spark UI Web Interface.
Kind Regards

asked 3 years ago264 views
3 Answers
0

If I log on to the endpoint server I can curl that URL successfully. How do I make it accessible via a browser?

answered 3 years ago
0

https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-webui.html

An SSH tunnel is the solution ...
... or lynx on the server.

answered 3 years ago
0

Sorry for late reply.

The link you mentioned is for EMR, not for Glue.
Actually you can access to Spark history server running on dev endpoint with SSH tunnel, but you do not need to configure it anymore.

Currently Glue provides two different ways to access to Spark UI; EC2(CloudFormation) and local Docker.
See details here; https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui.html

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions