Python cfn_tools module won't load in CodeBuild

Question

I have been getting the following error in my CodeBuild execution: ModuleNotFoundError: No module named 'cfn_tools'

Interesting note, the first time I ran this through CodeBuild with this module I had no issues. It only started happening after I made my next gitHub push that kicked off my pipeline that I saw this. The files that are related to this didn't change, and the modifications in that next push were to an unrelated section of the repo.

I have since tried to do:

pip install cfn-tools & pip3 install cfn-tools which mentioned that the module was already installed. These were added to the BuildSpec section. No success - still got the error
I've added a requirements.txt file with no success still got the error. I created this file using pip freeze also within the BuildSpec. The module shows up, but still get the error.
Originally used runtime version 3.7 of python and then tried with 3.9 which still didn't work.

python runtime 3.9 Any assistance would be appreciated.

UPDATE: To add more information I download a .tar.gz file from S3 that contains the python scripts I need for running in this build. I extract the .tar.gz then I run the script that is having the error. Here is the output for when I install cfn-tools and do a pip freeze You will see below that cfn-tools loads and is part of the output of pip freeze but yet when I run my script it give me the above error.

[Container] 2021/12/18 20:02:33 Running command pip3 install cfn-tools
Collecting cfn-tools
Downloading cfn-tools-0.1.6.tar.gz (3.9 kB)
Requirement already satisfied: Click>=6.0 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from cfn-tools) (7.1.2)
Requirement already satisfied: boto3>=1.3.1 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from cfn-tools) (1.18.58)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from boto3>=1.3.1->cfn-tools) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from boto3>=1.3.1->cfn-tools) (0.10.0)
Requirement already satisfied: botocore<1.22.0,>=1.21.58 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from boto3>=1.3.1->cfn-tools) (1.21.58)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from botocore<1.22.0,>=1.21.58->boto3>=1.3.1->cfn-tools) (2.8.2)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from botocore<1.22.0,>=1.21.58->boto3>=1.3.1->cfn-tools) (1.26.7)
Requirement already satisfied: six>=1.5 in /root/.pyenv/versions/3.9.5/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.22.0,>=1.21.58->boto3>=1.3.1->cfn-tools) (1.16.0)
Building wheels for collected packages: cfn-tools
Building wheel for cfn-tools (setup.py): started
Building wheel for cfn-tools (setup.py): finished with status 'done'
Created wheel for cfn-tools: filename=cfn_tools-0.1.6-py3-none-any.whl size=5456 sha256=9cd3471445f6552165508b0bd797498a535d3ef264059c9739cc6b72f7b96a26
Stored in directory: /root/.cache/pip/wheels/51/1f/6f/f50a0600d46c29ca31519968efefdc4547e8cda7a756584837
Successfully built cfn-tools
Installing collected packages: cfn-tools
Successfully installed cfn-tools-0.1.6
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.2; however, version 21.3.1 is available.
You should consider upgrading via the '/root/.pyenv/versions/3.9.5/bin/python3.9 -m pip install --upgrade pip' command.
[Container] 2021/12/18 20:02:36 Running command pip3 freeze
arrow==1.2.0
attrs==21.2.0
aws-lambda-builders==1.8.1
aws-sam-cli==1.33.0
aws-sam-translator==1.39.0
awscli==1.20.58
backports.entry-points-selectable==1.1.0
binaryornot==0.4.4
boto3==1.18.58
botocore==1.21.58
certifi==2021.10.8
**cfn-tools==0.1.6**
chardet==4.0.0
chevron==0.14.0
click==7.1.2
colorama==0.4.3
cookiecutter==1.7.3
dateparser==1.1.0
distlib==0.3.3
docker==4.2.2
docutils==0.15.2
filelock==3.3.0
Flask==1.1.4
idna==2.10
itsdangerous==1.1.0
Jinja2==2.11.3
jinja2-time==0.2.0
jmespath==0.10.0
jsonschema==3.2.0
MarkupSafe==2.0.1
pipenv==2021.5.29
platformdirs==2.4.0
poyo==0.5.0
pyasn1==0.4.8
pyrsistent==0.18.0
python-dateutil==2.8.2
python-slugify==5.0.2
pytz==2021.3
PyYAML==5.4.1
regex==2021.10.8
requests==2.25.1
rsa==4.7.2
s3transfer==0.5.0
serverlessrepo==0.1.10
six==1.16.0
text-unidecode==1.3
tomlkit==0.7.2
tzlocal==3.0
urllib3==1.26.7
virtualenv==20.8.1
virtualenv-clone==0.5.7
watchdog==2.1.2
websocket-client==1.2.1
Werkzeug==1.0.1

Accepted Answer

The module I was trying to install wasn't the one that was being used.

The module that needed to be installed was `cfn_flip` it has the `cfn_tools` module that the code was trying to use. The CodeBuild didn't have it installed, so how it worked on the first run is still a mystery.

[This StackOverflow question helped][1]

[1]: https://stackoverflow.com/questions/50914422/parse-an-aws-cloudformation-template-with-the-pyyaml-library/55349491#55349491

Answer

How are you defining the python packages within your buildspec.yaml?
For example, below would be a suggestion and running the command pip freeze will output the packages currently installed

```
pre_build:
  commands:
    - pip3 install cfn-tools
build:
  commands:
    - pip freeze
```

Python cfn_tools module won't load in CodeBuild

Relevanter Inhalt