Glue Interactive Python Jupyter notebook sessions

Question

Hello.

I'm following the AWS docs here [https://docs.aws.amazon.com/glue/latest/dg/interactive-sessions-chapter.html] for developing Glue ETL pyspark jobs on my laptop using remote Glue Spark services in AWS.  I get the errors below as soon as I follow the directions to start a session and attempt to actually use the session to do something.  Is Glue still a supported service?  I see that the documentation is quite old.  Error message below.

```
Traceback (most recent call last):
  File "/Users/xxx/.asdf/installs/python/3.9.7/lib/python3.9/site-packages/aws_glue_interactive_sessions_kernel/glue_python_kernel/GlueKernel.py", line 95, in do_execute
    self.create_session()
  File "/Users/xxx/.asdf/installs/python/3.9.7/lib/python3.9/site-packages/aws_glue_interactive_sessions_kernel/glue_python_kernel/GlueKernel.py", line 518, in create_session
    self.set_session_id(self.glue_client.create_session(
  File "/Users/xxx/.asdf/installs/python/3.9.7/lib/python3.9/site-packages/botocore/client.py", line 657, in __getattr__
    raise AttributeError(
AttributeError: 'Glue' object has no attribute 'create_session'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/xxx/.asdf/installs/python/3.9.7/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 357, in dispatch_shell
    await result
  File "/Users/xxx/.asdf/installs/python/3.9.7/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 652, in execute_request
    reply_content = await reply_content
  File "/Users/xxx/.asdf/installs/python/3.9.7/lib/python3.9/site-packages/aws_glue_interactive_sessions_kernel/glue_python_kernel/GlueKernel.py", line 97, in do_execute
    self.ipython_display.send_error(f'Exception encountered while creating session: {e} 
')
  File "/Users/xxx/.asdf/installs/python/3.9.7/lib/python3.9/site-packages/hdijupyterutils/ipythondisplay.py", line 30, in send_error
    self._ipython_shell.write_err(u"{}
".format(error))
AttributeError: 'ZMQInteractiveShell' object has no attribute 'write_err'
```
Here is my test code....
```
%iam_role arn:aws:iam::xxxxxx:role/service-role/AWSGlueServiceRole-dev
%session_id test-glue-session1
!jupyter --version
%list_sessions
%connections "dev-poc-glue-aurora-db"
sc = SparkContext()
```

and the output....
```
Welcome to the Glue Interactive Sessions Kernel
For more information on available magic commands, please type %help in any new cell.

Current iam_role is None
iam_role has been set to arn:aws:iam::xxxxxxxx:role/service-role/AWSGlueServiceRole-dev.
There is no current session.
Selected Jupyter core packages...
IPython          : 8.0.1
ipykernel        : 6.7.0
ipywidgets       : 7.6.5
jupyter_client   : 7.1.2
jupyter_core     : 4.9.1
jupyter_server   : not installed
jupyterlab       : not installed
nbclient         : 0.5.10
nbconvert        : 6.4.1
nbformat         : 5.1.3
notebook         : 6.4.8
qtconsole        : 5.2.2
traitlets        : 5.1.1
There is no current session.
Connections to be included:
dev-poc-glue-aurora-db
Authenticating with profile=default
glue_role_arn defined by user: arn:aws:iam::xxxxxxxxxxx:role/service-role/AWSGlueServiceRole-dev
Attempting to use existing AssumeRole session credentials.
Trying to create a Glue session for the kernel.
Worker Type: G.1X
Number of Workers: 5
Session ID: AROAJXDQBQMRZML4ALH7A:xxxxxxxxxx-8cff17de-6d96-4021-9f9c-5e6c9f3296b9
```

thanks so much for your assistance.

Fabrizio@AWS · Answer

Hello,

AWS Glue interactive session is currently in preview,  the documentation is reflecting the steps that will need to be run once it is general available.

At the moment, you need still to run a manual step before you start the notebook the first time:
* add the service model included with the install to botocore.

With your venv active run:

```
SITE_PACKAGES=$(pip3 show aws-glue-sessions | grep Location | awk '{print $2}')
cp $SITE_PACKAGES/aws_glue_interactive_sessions_kernel/service-2.json $SITE_PACKAGES/botocore/data/glue/2017-03-31/service-2.json
```
hope this helps

AWS-User-1287467 · Answer

Interactive Glue Sessions are now generally available. I made a step-by-step video  tutorial on how to configure it to develop locally in pycharm for anyone who wants to follow along.  https://youtu.be/04LMQxDxjGM

Mario Silva · Answer

Hello, I have a similar issue. I follow the guidelines to install jupyter notebook (https://docs.aws.amazon.com/glue/latest/dg/interactive-sessions.html) in a WSL Ubuntu 22.04 and when I try create a session I got the message: *Session failed to reach READY instead reaching terminal state FAILED.
Current session is in an unhealthy terminal state. If you wish to create a new session use %stop_session before executing any further statements.*

```
pip -V
pip 23.3.2 from /home/user/.local/lib/python3.10/site-packages/pip (python 3.10)

pip list
aws-glue-sessions         1.0.4
aws-requests-auth         0.4.3
boto3                     1.34.21
botocore                  1.34.21
jupyter                   1.0.0
jupyter_client            8.6.0
jupyter-console           6.6.3
jupyter_core              5.7.1
jupyter-events            0.9.0
jupyter-lsp               2.2.2
jupyter_server            2.12.5
jupyter_server_terminals  0.5.1
jupyterlab                4.0.10
jupyterlab_pygments       0.3.0
jupyterlab_server         2.25.2
jupyterlab-widgets        3.0.9

```

I tried installing directly on Ubuntu and on a virtualenv but I always get the same error.

![Enter image description here](/media/postImages/original/IMCWOp4sALSRaYdoG78I2xCw)

![Enter image description here](/media/postImages/original/IMvWSnc32zTTaBW_2WcPZq9g)

![Enter image description here](/media/postImages/original/IMCKhpmxrLQAuf88McPcyc9g)

Glue Interactive Python Jupyter notebook sessions

Relevant content