Greengrass Python Lambda using awsiotsdk: No module named '_awscrt'

Question

I'm trying to create a Greengrass Lambda function that publishes to an IPC topic using the the IoT SDK. My Lambda code includes:

```
from awsiot import greengrasscoreipc  # type: ignore
from awsiot.greengrasscoreipc.model import (  # type: ignore
    JsonMessage,
    PublishMessage,
    PublishToTopicRequest,
)

ipc_client = greengrasscoreipc.connect()
```

This leads to the following import error:

```
FATAL: lambda_runtime.py:426,Failed to initialize Lambda runtime due to exception: No module named '_awscrt'.
```

I can see that the Lambda artifact includes both `awscrt` and `awsiot` in `/greengrass/v2/packages/artifacts-unarchived/componentName/version/lambda-artifact/`, so I'm not sure why the import is failing.

I'm using a non-pinned Python 3.8 Lambda in NoContainer mode with LambdaManager v2.2.3, LambdaLauncher v2.0.10, and LambdaRuntimes v2.0.8

I have been able to use `awsiot` to publish to IPC topics in a non-Lambda component. That component runs a `pip install -r requirements.txt` command to install `awsiotsdk` to the site-packages for `ggc_user` during the install lifecycle, whereas the Lambda component is delivered to the device with its package dependencies included. Any ideas what I need to do differently? I haven't seen this problem yet with other Lambdas that have different dependencies.

Accepted Answer

I think this is essentially the same issue faced here: https://repost.aws/questions/QUvlMRwS8sTYWi2-1Sav0kog/lambda-can-not-load-python-modules-installed-with-component

I intended to try some of the solutions listed there, but first I aligned the Python versions for my Lambda and non-Lambda components. I am using the Dockerfile provided here: https://github.com/aws-greengrass/aws-greengrass-docker

That includes: 
```
# Install Greengrass v2 dependencies
RUN yum update -y && yum install -y python37 tar unzip wget sudo procps which && \
    amazon-linux-extras enable python3.8 && yum install -y python3.8 java-11-amazon-corretto-headless && \
```
So Python3.7 is the system default and was being used for my non-Lambda components while the Lambda components are using the Python3.8 runtime. I updated the Dockerfile to use 3.8 everywhere, changing the above to: 
```
RUN yum update -y  \
    && amazon-linux-extras install python3.8 \
    && update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1 \
    && update-alternatives --install /usr/bin/pip3 pip3 /usr/bin/pip3.8 1 \
    && yum install -y tar unzip wget sudo procps which \
    && yum install -y java-11-amazon-corretto-headless \
```
Now 3.8 is being used for all components. I already had a non-Lambda component that pip-installs `awsiotsdk` for `ggc_user`. The new Lambda component that also uses `awsiotsdk` is now also able to import from `awsiot` without issue. I assume that whatever was missing from the Lambda artifact is now being picked up from the `ggc_user` site-packages, which is similar to the solutions mentioned in the other post.

This is fine for my use case because the new Lambda component is dependent on the existing non-Lambda component, although I think it might be nice make `awsiot` importable into Lambdas without requiring that extra install if possible.

Greengrass Python Lambda using awsiotsdk: No module named '_awscrt'

Relevant content