Unable to import/install external library 'psycopg2' in AWS Glue
I have a utility of 'psycopg2' library in AWS Glue script. Since it is not included in standard library set, I would need to add externally. I tried to add library name using "--additional-python-modules" in the 'Job Parameters' sub section under 'Job Details' section, but still it said no library found.
What's the correct way of importing external library? Let's say I have uploaded 'whl' file in my S3 storage and have given the path in the value of 'Job Parameters'. Would I need to specify "!pip install psycopg2" in my Glue script and then "import psycopg2"?
Or what if I include a zip file in the 'Python library path' subsection, can I directly write "import psycopg2". Need assistance with using this library in Glue 4.0.
- Más nuevo
- Más votos
- Más comentarios
That library has native code dependencies and as indicated in the docs, Glue cannot compile the gcc libraries on installation.
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#addl-python-modules-support
You could try to find/create a precompiled wheel for that version of Python and Amazon Linux2.
Normally you don't connect from Python to the databases but I guess you want to run some custom command, you can do it using the JDBC driver already included using P4J calls.
Contenido relevante
OFICIAL DE AWSActualizada hace 2 años- OFICIAL DE AWSActualizada hace un año