Hi,
I have a python script which is running on my local machine and I want to move it to AWS Lambda for periodic execution. I have 3 import statements in the script for which I am adding layers but facing some issues.
from googleapiclient.discovery import build
import pandas as pd
from datetime import date
For googleapiclient api, I downloaded it in a folder and uploaded to AWS layer and lambda is able to find this module. I wanted to use this along with AWS Data wrangler package but running into layer size restriction issues. So I downloaded pandas to the same folder as google api and then uploaded the zip file to layer. But now I am get a numpy dependency error though numpy was downloaded as part of pandas install. Two folders as part of my libraries folder are numpy, numpy-1.22.0.dist-info which is correct version as per error message below. I also tried downloading numpy separately in same package but that also not working.
Please let me know if I am missing something and if this is the correct approach for installing python packages for AWS Lambda.
Response
{
"errorMessage": "Unable to import module 'lambda_function': Unable to import required dependencies:\nnumpy: \n\nIMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!\n\nImporting the numpy C-extensions failed. This error can happen for\nmany reasons, often due to issues with your setup or how NumPy was\ninstalled.\n\nWe have compiled some common reasons and troubleshooting tips at:\n\n https://numpy.org/devdocs/user/troubleshooting-importerror.html\n\nPlease note and check the following:\n\n * The Python version is: Python3.7 from "/var/lang/bin/python3.7"\n * The NumPy version is: "1.22.0"\n\nand make sure that they are the versions you expect.\nPlease carefully study the documentation linked above for further help.\n\nOriginal error was: No module named 'numpy.core._multiarray_umath'\n",
"errorType": "Runtime.ImportModuleError",
"stackTrace": []
}
Function Logs
START RequestId: de7259a6-cd11-4ecc-9f93-1a624f1c0c6d Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
- The Python version is: Python3.7 from "/var/lang/bin/python3.7"
- The NumPy version is: "1.22.0"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
END RequestId: de7259a6-cd11-4ecc-9f93-1a624f1c0c6d
REPORT RequestId: de7259a6-cd11-4ecc-9f93-1a624f1c0c6d Duration: 1.68 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 55 MB Init Duration: 570.09 ms
Request ID
de7259a6-cd11-4ecc-9f93-1a624f1c0c6d
Regards,
Dbeings
Thanks a ton! This was indeed the issue. I now used EC2 instance to install pandas files in a folder ( pip install pandas -t .) and then copied the files to my machine and then put the zip folder on my AWS layer. This worked.