Jupyter notebook persistence not working in EMR 6.12?


Confirmed this by

  • setting s3.persistence.bucket to the same value as in EMR 6.12, but my old notebooks did not appear,
  • setting s3.persistence.bucket to a garbage value (i.e., non-existent bucket), and the notebook started without any issues.

Clearly, it looks like the functionality is not being verified/is broken. Some help please?

質問済み 9ヶ月前274ビュー


Thank you for your question, to answer your query.

The above scenario is replicable, after launching EMR 6.12 cluster with below configurations :


Older jupyter notebooks were not loaded and you can check error at "/mnt/var/log/jupyter/jupyter.log":

 File "/opt/conda/lib/python3.9/site-packages/botocore/utils.py", line 33, in <module>
        import botocore.httpsession
      File "/opt/conda/lib/python3.9/site-packages/botocore/httpsession.py", line 9, in <module>
        from urllib3.util.ssl_ import (
    ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/opt/conda/lib/python3.9/site-packages/urllib3/util/ssl_.py)

As a workaround, you can try below commands to use older notebooks in EMR 6.12 :

sudo docker exec jupyterhub pip uninstall urllib3 -y
sudo docker exec jupyterhub pip install urllib3==1.26.16
sudo docker stop jupyterhub
sudo docker start jupyterhub

It is recommended that you try above workaround in testing environment first. Also, it is recommended that if possible please open a case with premium support [1] so that we may debug the issue and look into your resources accordingly.

Reference: [1] https://console.aws.amazon.com/support/home#/case/create

回答済み 9ヶ月前

ログインしていません。 ログイン 回答を投稿する。