How do I access the Spark UI?

Question

Hello!  
So I enabled the Spark UI for a dev endpoint as described here:  
https://docs.aws.amazon.com/en_pv/glue/latest/dg/monitor-spark-ui-dev-endpoints.html  
After creating a GlueContext a table with a link to the Spark UI is printed in the Notebook.  
The link leads to:  
http://ip-172-32-2-76.eu-central-1.compute.internal:20888/proxy/application_123456789_0001/  
But requesting that site with the browser yields a "Server not Found".  
So, I suppose a setting is missing.   
I would like to see that fancy Spark UI Web Interface.  
Kind Regards

Answer

Sorry for late reply.  
  
The link you mentioned is for EMR, not for Glue.  
Actually you can access to Spark history server running on dev endpoint with SSH tunnel, but you do not need to configure it anymore.  
  
Currently Glue provides two different ways to access to Spark UI; EC2(CloudFormation) and local Docker.  
See details here; https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui.html

Answer

If I log on to the endpoint server I can curl that URL successfully. How do I make it accessible via a browser?

Answer

https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-webui.html  
  
An SSH tunnel is the solution ...  
... or lynx on the server.

How do I access the Spark UI?

相關內容