AWS Glue local lnteractive sessions for Ray: no kernel

Question

Hi,

I'm attempting to create a Ray glue interactive session, Based on the [announcement blog post](https://aws.amazon.com/blogs/big-data/introducing-aws-glue-for-ray-scaling-your-data-integration-workloads-using-python/) a couple years back, it looks like local interactive sessions should be supported for Ray jobs. In fact, using Jupyter notebooks locally is the example they go through.

> For our development environment, we use a Jupyter notebook to run the code.

>You’re required to install the AWS Glue interactive sessions locally or run interactive sessions with an AWS Glue Studio notebook. Using AWS Glue Interactive sessions will help you follow and run the series of demonstration steps.

The instructions link to [this page](https://docs.aws.amazon.com/glue/latest/dg/interactive-sessions.html) in the docs, which instructs you to pip install `aws-glue-sessions` and run `install-glue-kernels`. When I do so, only the `glue_pyspark` and `glue_spark` kernels are installed (neither of which work for Ray jobs).

What am I missing here? Was this functionality removed?

Gnanesh_N · Accepted Answer

Hi,

From the docs, https://pypi.org/project/aws-glue-sessions/1.0.6/ => support for the glue_ray session was removed starting from aws-glue-sessions 1.0.6 and above, so if you want to use glue_ray session use aws-glue-sessions 1.0.5 or below.

check change log in https://pypi.org/project/aws-glue-sessions/1.0.6/ for wording => v1.0.6 (2024-05-23)
Removed glue_ray session type and related magics.

Further, It is recommended to migrate your AWS Glue to Ray on Elastic Compute Cloud (Amazon EC2)[1] or Ray on Amazon SageMaker[2].

[1] https://docs.ray.io/en/latest/cluster/vms/user-guides/launching-clusters/aws.html
[2] https://aws.amazon.com/blogs/opensource/scaling-ai-and-machine-learning-workloads-with-ray-on-aws/

Thank you!

AWS Glue local lnteractive sessions for Ray: no kernel

Add your answer

Relevant content