By using AWS re:Post, you agree to the Terms of Use
/AWS Glue 3.0 Docker Image - cannot install Jupyter NBExtensions/

AWS Glue 3.0 Docker Image - cannot install Jupyter NBExtensions

1

I have set up my local glue development environment following instructions at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-libraries.html and its working . I am trying to extend Jupyter by installing NB Extensions as in https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/install.html

pip command executes fine

pip install jupyter_contrib_nbextensions --user

I encounter an error on this step :

jupyter contrib nbextension install --user

PermissionError: [Errno 13] Permission denied: '/home/glue_user/.jupyter/migrated'

I get a similar error on the Docker Log as well

[W 2022-06-01 07:43:44.083 LabApp] [Errno 13] Permission denied: '/home/glue_user/.jupyter/lab/workspaces'

Any help ?

**Docker Image : **amazon/aws-glue-libs:glue_libs_3.0.0_image_01

**Docker Started using : **

docker run -it -v ~/.aws:/home/glue_user/.aws -v $JUPYTER_WORKSPACE_LOCATION:/home/glue_user/workspace/jupyter_workspace/ -e AWS_PROFILE=$PROFILE_NAME -e DISABLE_SSL=true --rm -p 4040:4040 -p 18080:18080 -p 8998:8998 -p 8888:8888 --name glue_jupyter_lab amazon/aws-glue-libs:glue_libs_3.0.0_image_01 /home/glue_user/jupyter/jupyter_start.sh
  • Setting up the VS was easier.

    I did not find where to do this step though which is Step 8 on the VS set up

    Paste the following JSON and save it.

    { "python.defaultInterpreterPath": "/usr/bin/python3", "python.analysis.extraPaths": [ "/home/glue_user/aws-glue-libs/PyGlue.zip:/home/glue_user/spark/python/lib/py4j-0.10.9-src.zip:/home/glue_user/spark/python/", ] }

asked a month ago23 views
1 Answers
0

Permissions appear to be incorrect.

1st terminal

docker run -it -v ~/.aws:/home/glue_user/.aws -v $JUPYTER_WORKSPACE_LOCATION:/home/glue_user/workspace/jupyter_workspace/ -e AWS_PROFILE=$PROFILE_NAME -e DISABLE_SSL=true --rm -p 4040:4040 -p 18080:18080 -p 8998:8998 -p 8888:8888 --name glue_jupyter_lab amazon/aws-glue-libs:glue_libs_3.0.0_image_01 /home/glue_user/jupyter/jupyter_start.sh

2nd terminal

$ docker exec glue_jupyter_lab ls -la /home/glue_user
total 84
drwx------ 1 glue_user root 4096 Jun  2 14:14 .
drwxr-xr-x 1 root      root 4096 Feb 25 05:27 ..
drwxrwxrwx 1      1000 1000 4096 Apr 13 09:44 .aws
drwxr-xr-x 1 glue_user root 4096 Feb 25 05:28 aws-glue-libs
-rw-r--r-- 1 glue_user root   18 Jul 15  2020 .bash_logout
-rw-r--r-- 1 glue_user root  193 Jul 15  2020 .bash_profile
-rw-r--r-- 1 glue_user root 5441 Mar 31 11:22 .bashrc
drwxr-xr-x 1 glue_user root 4096 Jun  2 14:14 .cache
drwxr-xr-x 2 glue_user root 4096 Apr  1 06:10 .certs
drwxr-xr-x 2 glue_user root 4096 Jun  2 14:14 .ipython
drwxr-xr-x 1 glue_user root 4096 Apr  1 06:10 jupyter
drwxr-xr-x 3 root      root 4096 Apr  1 06:10 .jupyter
drwxr-xr-x 1 glue_user root 4096 Jun  2 14:14 livy
drwx------ 1 glue_user root 4096 Apr  1 06:42 .local
drwxr-xr-x 1 glue_user root 4096 Jun  2 14:13 spark
drwxr-xr-x 2 glue_user root 4096 Apr  1 06:10 .sparkmagic
drwxr-xr-x 3 glue_user root 4096 Apr  1 06:39 workspace
$ 

The owner of .jupyter is root.

How to fix

2nd terminal

docker exec -u 0:0 glue_jupyter_lab chown -R glue_user.root /home/glue_user/.jupyter

Now the .jupyter directory will be owned by glue_user.

answered a month ago
  • Yes , but then the chown permission is denied on the docker console. Its asking for the root password , which I do not know

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions