OperationalError: SCRAM authentication requires libpq version 10 or above

The version of libpq used by AWS Glue is managed by AWS and cannot be updated by users. However, you can try using an older version of psycopg2 that is compatible with the version of libpq used by AWS Glue.

You can also try using the pure Python implementation of PostgreSQL client library, psycopg2-binary, which does not have any dependency on libpq. You can install it by running the following command:

pip install psycopg2-binary

Once you have installed psycopg2-binary, you can import it in your Glue job and use it instead of psycopg2. For example:

import psycopg2_binary as psycopg2
# use psycopg2 in your code as usual

This should allow you to use PostgreSQL with AWS Glue without encountering the SCRAM authentication requires libpq version 10 or above error.

A temporary solution until libraries are updated is to revert the databases back to password encryption as MD5 after the database upgrade. This is accomplished by creating DB cluster custom parameter group, changing the parameter 'password_encryption' to MD5, and applying it to the cluster. Take note all nodes will need to be restarted.

You can see what password encryption is set to by running: CREATE EXTENSION rds_tools; SELECT * FROM rds_tools.role_password_encryption_type();

This will solve the problem quickly allowing for time to get all the libraries upgraded and tested.