By using AWS re:Post, you agree to the Terms of Use
/Cannot seem to import psycopg2 in AWS Glue 3.0/

Cannot seem to import psycopg2 in AWS Glue 3.0

0

I have tried a myriad of ways to get psycopg2 imported into a script.

  • import psycopg2 (on it's own as psycopg2 is listed as available in Glue 3.0 when I print(help("modules"))
  • Downloaded both 64-bit and 32-bit and tried both in Python library path (individually) - log actually says these loaded but import still fails
  • Added --additional-python-modules as a parameter and tried both with and without a specific version - I think this fails on my because of network security implemented (no downloading from external sources).

I will get the error "ModuleNotFoundError: No module named 'psycopg2._psycopg'" or "ModuleNotFoundError: No module named 'psycopg2'". The former is when I use the whl objects. The latter is when I try to use the module that appears to be available by default.

asked 2 months ago185 views
1 Answers
0

Hello,

Please try using the below under job parameters of your Glue 3.0 job.

--additional-python-modules psycopg2-binary==2.9.3

psycopg2 installation appears to be failing due to a missing dependency as shown below and the developer is suggesting to use psycopg2-binary instead

**Error: **

pg_config executable not found. pg_config is required to build psycopg2 from source.

If you prefer to avoid building psycopg2 from source, please install the PyPI 'psycopg2-binary' package instead.
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions