[AWS EMR] HTTP 403: Forbidden (Workspace is not attached to cluster]


Hi everyone,

I am a bit new to AWS so pardon me on this. I am trying to run Spark on a notebook in EMR. I have created a cluster to achieve this effect. However, when I tried to select Pyspark or Spark as the kernel, the aforementioned error pops out.

Enter image description here

I have tried creating workspaces in EMR Studio but it doesn't seem to solve the problem.

The next thing I have tried was to try other versions of EMR Release versions. The ones that have succeeded are

  1. emr-6.0.0 (JupyterHub 1.0.0, Hive 3.1.2, Hue 4.4.0, Livy 0.6.0, Spark 2.4.4)
  2. emr-5.18.0 (Hive 2.3.3, Pig 0.17.0, Hue 4.2.0, JupyterHub 0.8.1, Spark 2.3.2)
  3. emr-5.2.3 (Hive 2.1.0, Pig 0.16.0, Hue 3.10.0, Spark 2.0.2)

However, the following failed and gave the error as shown in the picture.

  1. emr-6.3.1 (JupyterHub 1.2.0, Hive 3.1.2, Pig 0.17.0, Hue 4.9.0, Spark 3.1.1, JupyterEnterpriseGateway 2.1.0)
  2. emr-5.36.0 (Hive 2.3.9, Pig 0.17.0, Hue 4.10.0, JupyterEnterpriseGateway 2.1.0, JupyterHub 1.4.1, Spark 2.4.8)

What could be the issue behind this?

Thanks so much everyone!

asked a year ago349 views
1 Answer

If the notebook is attached to cluster and if you are still encountering this error, it probably means notebook is unable to communicate to EMR Cluster's JupyterEnterpriseGateway(JEG) endpoint. Please make sure your EMR cluster / notebook security groups are not restrictive. https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-security-groups.html

you may check logs on your EMR cluster on /var/log/applications/Jupyter*

Unfortunately, you will not able to see logs on EMR notebooks as its managed by EMR service.

if this is working differently between versions, Please make sure you review these

EMR Studio Considerations : https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-studio-considerations.html

EMR Notebook Considerations : https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-considerations.html

profile pictureAWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions