Cannot seem to import psycopg2 in AWS Glue 3.0

Question

I have tried a myriad of ways to get psycopg2 imported into a script.

* import psycopg2   (on it's own as psycopg2 is listed as available in Glue 3.0 when I print(help("modules"))
* Downloaded both 64-bit and 32-bit and tried both in Python library path (individually) - log actually says these loaded but import still fails
* Added --additional-python-modules as a parameter and tried both with and without a specific version - I think this fails on my because of network security implemented (no downloading from external sources).

I will get the error "ModuleNotFoundError: No module named 'psycopg2._psycopg'" or "ModuleNotFoundError: No module named 'psycopg2'". The former is when I use the whl objects. The latter is when I try to use the module that appears to be available by default.

Answer

Hello,

Please try using the below under job parameters of your Glue 3.0 job.

--additional-python-modules **psycopg2-binary==2.9.3**

> psycopg2 installation appears to be failing due to a missing dependency as shown below and the developer is suggesting to use psycopg2-binary instead

**Error: ** 
```
pg_config executable not found. pg_config is required to build psycopg2 from source.

If you prefer to avoid building psycopg2 from source, please install the PyPI 'psycopg2-binary' package instead.
```

Cannot seem to import psycopg2 in AWS Glue 3.0

Relevant questions

How do you update packages on AWS Glue?

AWS Glue - Read a 'local' file in Python

unable to install psycopg2 in glue JUPYTER notebook

How can i run SQL statements in my AWS Glue Script

Call a stored procedure in Redshift from Glue

Psycopg2 on AWS Lambda not connecting to RDS database

Using External Python Packages on AWS Glue.

Connecting to Redshift with Python - Operation Timed Out

Cannot seem to import psycopg2 in AWS Glue 3.0

No module named 'psycopg2'