Why did my Secrets Manager Lambda function rotation fail with a “pg module not found“ error?

Short description

You might receive this error if you're running an earlier Lambda function that was automatically upgraded from Python 3.7 to a newer version of Python.

Resolution

Change the Lambda function version back to Python 3.7, and redeploy the Lambda function.

Check the Lambda function Python version

1.    Open the Lambda console.

2.    In the navigation pane, choose Functions, and then select the Lambda function name that failed to rotate.

3.    Choose the Code tab.

4.    In Runtime settings, note the Python version.

5.    Choose Actions, choose Export function, and then choose Download deployment package.

6.    Run the following Linux find command from the directory of the uncompressed .zip file:

find ./ -name “*cpython-*-x86_64-linux-gnu.so”

7.    Check that the version of the Python runtime is in the name of the .so file. For example _cffi_backend.cpython-37m-x86_64-linux-gnu.so.

If the .so version isn't Python 3.7, then Lambda can't import the library.

Modify the AWS Cloudformation template

To resolve the Lambda function failure, modify the CloudFormation template that you used to create the Lambda function. Add the Runtime property to the HostedRotationLambda object in AWS::SecretsManager::RotationSchedule. Then, redeploy the CloudFormation stack. CloudFormation changes the Lambda rotation function back to Python 3.7.

Note: For Python 3.7, the Runtime key value must be python3.7.

Related information

Troubleshooting AWS Secrets Manager