Jupyter notebook persistence not working in EMR 6.12?

0

Confirmed this by

  • setting s3.persistence.bucket to the same value as in EMR 6.12, but my old notebooks did not appear,
  • setting s3.persistence.bucket to a garbage value (i.e., non-existent bucket), and the notebook started without any issues.

Clearly, it looks like the functionality is not being verified/is broken. Some help please?

Dev
asked 8 months ago233 views
1 Answer
0

Hi,

Thank you for your question, to answer your query.

The above scenario is replicable, after launching EMR 6.12 cluster with below configurations :

[
    {"classification":"jupyter-s3-conf", 
                    "properties":{
                                             "s3.persistence.bucket":"<s3-bucket-name>/jupyterhublogs", 
                                             "s3.persistence.enabled":"true"
                                          }, 
                     "configurations":[]
    }
]

Older jupyter notebooks were not loaded and you can check error at "/mnt/var/log/jupyter/jupyter.log":

 File "/opt/conda/lib/python3.9/site-packages/botocore/utils.py", line 33, in <module>
        import botocore.httpsession
      File "/opt/conda/lib/python3.9/site-packages/botocore/httpsession.py", line 9, in <module>
        from urllib3.util.ssl_ import (
    ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/opt/conda/lib/python3.9/site-packages/urllib3/util/ssl_.py)

As a workaround, you can try below commands to use older notebooks in EMR 6.12 :

sudo docker exec jupyterhub pip uninstall urllib3 -y
sudo docker exec jupyterhub pip install urllib3==1.26.16
sudo docker stop jupyterhub
sudo docker start jupyterhub

It is recommended that you try above workaround in testing environment first. Also, it is recommended that if possible please open a case with premium support [1] so that we may debug the issue and look into your resources accordingly.

Reference: [1] https://console.aws.amazon.com/support/home#/case/create

AWS
SUPPORT ENGINEER
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions