Lambda function failing with "cannot import name 'etree' from 'lxml' (/var/task/lxml/__init__.py)"

Question

I'm creating a lambda function using a virtual env in Windows 10. I'm adding site-packages to a .zip archive and updating the lambda function.

Currently this is failing with "cannot import name 'etree' from 'lxml' (/var/task/lxml/__init__.py)"

My venv is using Python 3.8 as is the runtime environment in lambda. Some [previous solutions](https://stackoverflow.com/questions/53406638/importerror-cannot-import-name-etree-on-python-3-6) suggest setting runtime on lambda to Python 3.6 but this is no longer an option. I've tried 3.7 and 3.9 but the problem persists. Other solutions on [re:Post](https://repost.aws/questions/QUwIZg6DlXTgKHjJyMIlG1dQ) discuss docker containers rather than zip archives and linux environments rather than windows.

Ojaybee · Accepted Answer

I fixed this by spinning up an ec2 machine, creating a directory named python, using pip3 to install the libraries I needed, creating a .zip archive out of the python directory and uplodaing that .zip to s3. From there I could create a layer for aws lambda using the .zip archive.

If you're running into similar problems it may be because you are using Mac OS or Windows to build libraries. Much better to use a Amazon AMI machine to ensure compatibility wiith Lambda.

KalindiP · Answer

Hi, 
This might be a little late to answer, but here is an alternative approach that helped me fixed this issue without spinning an ec2  instance. 
For reference, I was getting the same issue with my Lambda function when I was using the python-docx library.

1) The easiest way is to utilize your cloud shell environment. The AWS cloud shell had a similar runtime environment as my python lambda. You can pip install your libraries, package them as you would traditionally do for a normal zip file and then the AWS cloud shell has an option to download the zip file to your local desktop. You can download it and then upload the zip file to your lambda layer and use it as a dependency.  Worked well for me and saves me the hassle of spinning a VM every time.

Few things to note would be:
1) The python version on your cloud shell might not be the same as the version on your lambda, but it very easy to upgrade/downgrade it on your cloud shell before you use it. 
2) The initial zip file that I created had a folder named python-3.12.x but it did not work with that, but renaming it 3.12 worked.

Hope this helps!

Lambda function failing with "cannot import name 'etree' from 'lxml' (/var/task/lxml/init.py)"

Relevant content