Hello Team ,
I have created Docker compose file mentioned below :
version: "2"
services:
spark:
image: glue/spark:latest
container_name: spark
** build: ./spark**
hostname: spark
ports:
- "8888:8888"
- "4040:4040"
entrypoint : sh
command : -c "/home/glue_user/jupyter/jupyter_start.sh"
volumes:
- ../app/territoryhub-replication:/home/glue_user/workspace/jupyter_workspace
Docker file which is getting called in build section is as follows :
FROM amazon/aws-glue-libs:glue_libs_3.0.0_image_01
USER root
RUN mkdir -p /root/.aws
RUN echo "[default]\nregion=us-east-1" >> /root/.aws/config
My Docker is getting started but failing (sharing logs )
Starting Jupyter with SSL
/home/glue_user/jupyter/jupyter_start.sh: line 4: livy-server: command not found
[I 2022-05-12 15:41:33.032 ServerApp] jupyterlab | extension was successfully linked.
[I 2022-05-12 15:41:33.044 ServerApp] nbclassic | extension was successfully linked.
[I 2022-05-12 15:41:33.046 ServerApp] Writing Jupyter server cookie secret to /root/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2022-05-12 15:41:33.541 ServerApp] sparkmagic | extension was found and enabled by notebook_shim. Consider moving the extension to Jupyter Server's extension paths.
[I 2022-05-12 15:41:33.541 ServerApp] sparkmagic | extension was successfully linked.
[I 2022-05-12 15:41:33.541 ServerApp] notebook_shim | extension was successfully linked.
[W 2022-05-12 15:41:33.556 ServerApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
[I 2022-05-12 15:41:33.558 ServerApp] notebook_shim | extension was successfully loaded.
[I 2022-05-12 15:41:33.560 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.7/site-packages/jupyterlab
[I 2022-05-12 15:41:33.560 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
[I 2022-05-12 15:41:33.565 ServerApp] jupyterlab | extension was successfully loaded.
[I 2022-05-12 15:41:33.569 ServerApp] nbclassic | extension was successfully loaded.
[I 2022-05-12 15:41:33.569 ServerApp] sparkmagic extension enabled!
[I 2022-05-12 15:41:33.569 ServerApp] sparkmagic | extension was successfully loaded.
Traceback (most recent call last):
File "/usr/local/bin/jupyter-lab", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/jupyter_server/extension/application.py", line 584, in launch_instance
serverapp = cls.initialize_server(argv=args)
File "/usr/local/lib/python3.7/site-packages/jupyter_server/extension/application.py", line 557, in initialize_server
find_extensions=find_extensions,
File "/usr/local/lib/python3.7/site-packages/traitlets/config/application.py", line 88, in inner
return method(app, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/jupyter_server/serverapp.py", line 2421, in initialize
self.init_httpserver()
File "/usr/local/lib/python3.7/site-packages/jupyter_server/serverapp.py", line 2251, in init_httpserver
max_buffer_size=self.max_buffer_size,
File "/usr/local/lib64/python3.7/site-packages/tornado/util.py", line 288, in new
instance.initialize(*args, **init_kwargs)
File "/usr/local/lib64/python3.7/site-packages/tornado/httpserver.py", line 191, in initialize
read_chunk_size=chunk_size,
File "/usr/local/lib64/python3.7/site-packages/tornado/tcpserver.py", line 134, in init
'certfile "%s" does not exist' % self.ssl_options["certfile"]
ValueError: certfile "/home/glue_user/.certs/my_key_store.pem" does not exist
Please help in resolving this asap
Many Thanks
