Glue PyShell Python Version

Question

There was recently Glue 3.0 released which indicates newer Spark version and same Python version compared to Glue 2.0. Now as I understood Glue 2.0 updated only Spark type jobs but did not include anything for PyShell type jobs.

Does Glue 3.0. bring any updates for Glue PyShell? I.e. is PyShell still on python 3.6?

Accepted Answer

Hi ,

[AWS Glue 3.0 updated only Spark type jobs](https://aws.amazon.com/blogs/big-data/introducing-aws-glue-3-0-with-optimized-apache-spark-3-1-runtime-for-faster-data-integration/) (introduced support for Spark 3.1.1 and Scala 2.1.12).

UPDATE as of August 2022
AWS Glue Python Shell currently support python 3.9 and has introduced additional libraries out of the box, and simplified the installation of libraries that are not included in the base engine.

To learn more check [this blog post.](https://aws.amazon.com/blogs/big-data/aws-glue-python-shell-now-supports-python-3-9-with-a-flexible-pre-loaded-environment-and-support-to-install-additional-libraries/)

hope this helps,

Glue PyShell Python Version

Contenuto pertinente