Python Layers and Requests import module

Question

Hello 
I am quite new on AWS world and facing some difficulties:
1) I looked at lambda to transfer and manipulate (ETL) data sources into a database (through Stitch API). 
     Most of my legacy data infrastructure is built  like this (with Heroku Ruby, which I don't like)
2) I have done most of the work fine using Python 3.9 AWS Lambda, but the last bit (post transfer to Stitch API) 
    fails because that accepts only JSON data and the libraries I use (urllib, urlopen) only send byte encoded data (afaik)
3) I read around and attempted to build a 3.8 Layer with the recommended python 'requests' module, I managed to 
    build a 3.8 layer, I moved my lambda function back to 3.8. 
4) Now that's failing because  "urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2k-fips 26"
     Reading around I should move to 3.10, but that is not available in amazon-linux-extras when creating a layer. I read that I should build it myself 
     from sources, which seems a bit too far for me, opening other cans of worms.
  
Do you think the direction of work is fine? Do you have suggestion how to proceed, without getting into other difficulties? 
I thought that Python, AWS working with simple APIs would be quite straightforward, but it seems it is not. 
Thanks so much for your help
Denis

Accepted Answer

Hello.  
There is a Lambda layer created by volunteers.  
Using this one you may not have to create layers.  
https://github.com/keithrozario/Klayers

Answer

I got a **DEFAULT_CIPHERS' from 'urllib3.util.ssl_' error** when I trying a post request from a lambda function to an external API as well. I fix the problem installing a previous version of urllib3 ( The latest version has problems with boto3 ).

Try creating a new zip file to import requests, but this time after make the pip install requests, force again the instalation of urllib3==1.26.18, like this:

pip install  --target=. --implementation cp --python-version 3.10 --only-binary=:all: urllib3==1.26.18 --upgrade

After that I got rid of the problem.

Python Layers and Requests import module

Relevant content