Unable to import/install external library 'psycopg2' in AWS Glue

Question

I have a utility of 'psycopg2' library in AWS Glue script. Since it is not included in standard library set, I would need to add externally. I tried to add library name using "--additional-python-modules" in the 'Job Parameters' sub section under 'Job Details' section, but still it said no library found.

![Enter image description here](/media/postImages/original/IMZ-M88R2ZTGmDTE8EkP0Lew)

What's the correct way of importing external library? Let's say I have uploaded 'whl' file in my S3 storage and have given the path in the value of 'Job Parameters'. Would I need to specify "!pip install psycopg2" in my Glue script and then "import psycopg2"?

Or what if I include a zip file in the 'Python library path' subsection, can I directly write "import psycopg2".
Need assistance with using this library in Glue 4.0.

Answer

That library has native code dependencies and as indicated in the docs, Glue cannot compile the gcc libraries on installation.   
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#addl-python-modules-support   
You could try to find/create a precompiled wheel for that version of Python and Amazon Linux2.   
Normally you don't connect from Python to the databases but I guess you want to run some custom command, you can do it using the JDBC driver already included using P4J calls.

Unable to import/install external library 'psycopg2' in AWS Glue

Relevant content