Passer au contenu

How do I access the Spark UI?

0

Hello!
So I enabled the Spark UI for a dev endpoint as described here:
https://docs.aws.amazon.com/en_pv/glue/latest/dg/monitor-spark-ui-dev-endpoints.html
After creating a GlueContext a table with a link to the Spark UI is printed in the Notebook.
The link leads to:
http://ip-172-32-2-76.eu-central-1.compute.internal:20888/proxy/application_123456789_0001/
But requesting that site with the browser yields a "Server not Found".
So, I suppose a setting is missing.
I would like to see that fancy Spark UI Web Interface.
Kind Regards

demandé il y a 7 ans2,3 k vues

4 réponses
0

If I log on to the endpoint server I can curl that URL successfully. How do I make it accessible via a browser?

répondu il y a 7 ans

0

https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-webui.html

An SSH tunnel is the solution ...
... or lynx on the server.

répondu il y a 7 ans

0

Sorry for late reply.

The link you mentioned is for EMR, not for Glue.
Actually you can access to Spark history server running on dev endpoint with SSH tunnel, but you do not need to configure it anymore.

Currently Glue provides two different ways to access to Spark UI; EC2(CloudFormation) and local Docker.
See details here; https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui.html

AWS

répondu il y a 7 ans

0
AWS
EXPERT

répondu il y a 8 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.