External python libraries in a AWS Glue python shell job

Question

I am trying to use geopandas in a python shell job, but can't get it working.

I have tried following this guide: https://repost.aws/knowledge-center/glue-import-error-no-module-named
and also providing a zip file of the library as well. 
In the first case it says 'No module named shapely' (with or without shapely in install_requires in setup.py).
In the second case, it says 'No module named shapely.lib'

Any help would be appreciated. Thanks!

Answer

The easiest thing is just to use --additional-python-modules=shapely so it installs from pip.  
If you use a zip, make sure the module inside is correct (you can unzip it locally, start a Python interpreter on the same directory and see if you can import it and it)

External python libraries in a AWS Glue python shell job

Contenuto pertinente