How do I create a Lambda layer using a simulated Lambda environment with Docker?

4 minute read

I want to create an AWS Lambda layer that's compatible with target environment runtimes. How can I do that?

Short description

When you develop Lambda functions, you choose between supported Lambda runtime versions. To create Lambda functions and layers that are compatible with specific runtimes, you can do either of the following:

For more information, see Creating and sharing Lambda layers.


Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

The following instructions use Python as the coding language and a Linux-based computer platform as an example. For other coding languages and platforms, adapt the steps accordingly.

Install Docker on your computer

If you haven't already done so, install Docker for Linux, Windows, or macOS on your computer.

Create the appropriate directory structure and specify your dependencies in the pip requirements file (requirements.txt)

For more information on creating directory structures, see Including library dependencies in a layer. For more information on specifying dependencies, see Requirements files in the pip user guide.

Example directory structure for a Lambda layer that's compatible with Python versions 3.6 and 3.8

├── requirements.txt
└── python/
    └── lib/
        ├── python3.6/
        │   └── site-packages/
        └── python3.8/
            └── site-packages/

Note: The total unzipped size of the function and all layers can't exceed the unzipped deployment package size limit of 250 MB. For more information about Python version support in Lambda, see Building Lambda functions with Python.

Install the library dependencies to their appropriate subfolders based on the runtimes that you specified in your directory structure

Run the following AWS CLI command once for each runtime that you specified in the directory structure:

Important: Replace 3.6 with 3.7 or 3.8, depending on the compatible libraries that you want to install.

docker run -v "$PWD":/var/task "" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.6/site-packages/; exit"

Create a .zip file archive for your layer

Run the following command to zip the python folder and create the layer package:

Note: You can replace mypythonlibs with any name that you prefer.

zip -r python > /dev/null

The .zip file is deflated in the /opt folder of the Lambda environment. The /opt folder is accessible from your Lambda function code.

Create or update your layer

Run the following command to either create or update your Lambda layer:

Important: Make sure that you include the compatible runtimes that you specified earlier. Replace mypythonlibs with the package name that you used in the previous step. Replace My python libs with any description that you prefer.

aws lambda publish-layer-version --layer-name mypythonlibs --description "My python libs" --zip-file fileb:// --compatible-runtimes "python3.6" "python3.8"

Review the output and copy the layer's Amazon Resource Name (ARN). You need the layer's ARN to complete the next step.

Update your Lambda function configuration to use the layer

Run the following command to update your Lambda function configuration to use the layer:

Important: Replace arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 with the layer ARN.

aws lambda update-function-configuration --layers arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 --function-name my-function

(Optional) To use the Docker images for your Lambda package instead of an AWS SAM file

Instead of using an AWS SAM file, you can also use the amazon/aws-sam-cli-build-image Docker images to create your Lambda package. Using the Docker images allows you to create the package without creating a layer.

1.    Run the following command to get the required versions of your dependencies:

Important: Replace 3.6 with 3.7 or 3.8, depending on the compatible libraries that you want to install.

docker run -v "$PWD":/var/task "" /bin/sh -c "pip install -r requirements.txt -t libs; exit"

2.    Create a Lambda deployment package that includes your function code and the correct libraries.

Related information

Why am I getting "unknown service", "parameter validation failed", or "object has no attribute" errors with my Python (Boto 3) Lambda function?

Building layers

Use any programming language and share common components

Lambda (AWS CLI Command Reference)

AWS OFFICIALUpdated 2 years ago