By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

How to change the numpy version when use variant.TensorFlowLite

0

Hi AWS team,

I am currently running the Greengrass qualification test. But my userdata.json is set PreInstalled to 'no'. From the logs, it seems that IDT installed numpy by itself. The version of numpy installed on my DUT is also below 2. How to change the numpy version when use variant.TensorFlowLite.

variant.TensorFlowLite.log:

2024-10-29T10:00:08.982Z [WARN] (Copier) variant.TensorFlowLite: stderr. + source ../variant.TensorFlowLite/greengrass_ml/greengrass_ml_tflite_venv/bin/activate. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:08.997Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ deactivate nondestructive. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:08.999Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' -n '' ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' -n '' ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.001Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' -n /usr/bin/bash -o -n '' ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.002Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ hash -r. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.003Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' -n '' ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.004Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ unset VIRTUAL_ENV. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.005Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' '!' nondestructive = nondestructive ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.006Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ VIRTUAL_ENV=/greengrass/v2/idt-055f425063677087b33/work/variant.TensorFlowLite/greengrass_ml/greengrass_ml_tflite_venv. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.008Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ export VIRTUAL_ENV. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.009Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.010Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ PATH=/greengrass/v2/idt-055f425063677087b33/work/variant.TensorFlowLite/greengrass_ml/greengrass_ml_tflite_venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.011Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ export PATH. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.012Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' -n '' ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.013Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' -z '' ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.014Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ _OLD_VIRTUAL_PS1=. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.015Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ PS1='(greengrass_ml_tflite_venv) '. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.016Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ export PS1. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.017Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ '[' -n /usr/bin/bash -o -n '' ']'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.018Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ hash -r. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.019Z [WARN] (Copier) variant.TensorFlowLite: stderr. + cd ../variant.TensorFlowLite/greengrass_ml/greengrass_ml_tflite_venv. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.020Z [WARN] (Copier) variant.TensorFlowLite: stderr. + install_python3_libraries. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.021Z [WARN] (Copier) variant.TensorFlowLite: stderr. ++ python3 -c 'try:. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.023Z [WARN] (Copier) variant.TensorFlowLite: stderr. import cv2. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.024Z [WARN] (Copier) variant.TensorFlowLite: stderr. print(cv2.__version__). {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.025Z [WARN] (Copier) variant.TensorFlowLite: stderr. except Exception as e:. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.026Z [WARN] (Copier) variant.TensorFlowLite: stderr. print(e)'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.094Z [WARN] (Copier) variant.TensorFlowLite: stderr. + opencv='No module named '\''cv2'\'''. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.094Z [INFO] (Copier) variant.TensorFlowLite: stdout. Installing opencv.... {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.096Z [WARN] (Copier) variant.TensorFlowLite: stderr. + [[ No module named 'cv2' == *\N\o\ \m\o\d\u\l\e\ \n\a\m\e\d\ \'\c\v\2\'* ]]. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.097Z [WARN] (Copier) variant.TensorFlowLite: stderr. + echo 'Installing opencv...'. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:09.098Z [WARN] (Copier) variant.TensorFlowLite: stderr. + pip3 install opencv-python. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:27.910Z [INFO] (Copier) variant.TensorFlowLite: stdout. Collecting opencv-python. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:28.328Z [INFO] (Copier) variant.TensorFlowLite: stdout. Using cached opencv_python-4.10.0.84-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (41.7 MB). {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:36.166Z [INFO] (Copier) variant.TensorFlowLite: stdout. Collecting numpy>=1.17.3. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:36.322Z [INFO] (Copier) variant.TensorFlowLite: stdout. Using cached numpy-2.0.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.9 MB). {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:37.239Z [INFO] (Copier) variant.TensorFlowLite: stdout. Installing collected packages: numpy, opencv-python. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-29T10:00:58.396Z [INFO] (Copier) variant.TensorFlowLite: stdout. Successfully installed numpy-2.0.2 opencv-python-4.10.0.84. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}

aws.greengrass.TensorFlowLiteImageClassification.log:

A module that was compiled using NumPy 1.x cannot be run in. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.224Z [WARN] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stderr. NumPy 2.0.2 as it may crash. To support both 1.x and 2.x. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.225Z [WARN] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stderr. versions of NumPy, modules must be compiled with NumPy 2.0.. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.225Z [INFO] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stdout. Exception occured during the allocation of tensors: <built-in method CreateWrapperFromFile of PyCapsule object at 0x7f8e531390> returned a result with an error set. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.225Z [WARN] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stderr. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.226Z [WARN] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stderr. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.227Z [WARN] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stderr. If you are a user of the module, the easiest solution will be to. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.228Z [WARN] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stderr. downgrade to 'numpy<2' or try to upgrade the affected module.. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
2024-10-29T10:02:27.229Z [WARN] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stderr. We expect that some modules will need time to support NumPy 2.. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
Topics
Internet of Things (IoT)
Tags
AWS IoT Greengrass
Language
English
Jan
asked a month ago72 views
3 Answers
1

Based on the logs you've provided, it appears that the IDT (IoT Device Tester) for AWS IoT Greengrass is installing NumPy version 2.0.2, which is causing compatibility issues with some modules that were compiled using NumPy 1.x.

To change the NumPy version when using variant.TensorFlowLite, you have a few options:

  1. Downgrade NumPy: You can modify the installation script to install a specific version of NumPy that is compatible with your TensorFlow Lite setup. For example, you could change the pip install command to:

    pip3 install numpy==1.21.0

    This would install NumPy 1.21.0, which should be compatible with modules compiled for NumPy 1.x.

  2. Use a requirements file: Create a requirements.txt file with specific versions of the libraries you need, including NumPy. Then modify the installation script to use this file:

    pip3 install -r requirements.txt

  3. Modify the virtual environment setup: If you have control over the virtual environment creation, you could pre-install the correct version of NumPy before running the IDT tests.

  4. Update dependent modules: If possible, update the modules that are causing issues to versions that are compatible with NumPy 2.x. This might involve rebuilding some modules with updated dependencies.

To implement these changes, you would need to modify the installation scripts used by the IDT. This might involve creating a custom component or modifying existing component recipes in your AWS IoT Greengrass setup.

Remember to test your changes thoroughly, as modifying dependencies can have wide-ranging effects on your Greengrass components and applications.

Sources
TensorFlow Lite runtime - AWS IoT Greengrass
Run the AWS IoT Greengrass qualification suite - AWS IoT Greengrass
Tutorial: Build and run the sample IDT test suite - AWS IoT Greengrass

profile picture
re:Post Agent
answered a month ago
0

After uninstalling NUMPY were you unblocked?

profile pictureAWS
N9WXU
answered a month ago
  • Jan
    a month ago

    From variant.TensorFlowLite.log. It seems that greengrass install TensorFlowLite by script. It will uninstall the NUMPY and install NUMPY>2 version. Where can it modify the script

    2024-10-24T03:15:25.222Z [INFO] (pool-2-thread-15) variant.TensorFlowLite: shell-runner-start. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW, command=["\n                                    bash /home/linaro/greengrass_test/ggquali..."]}
2024-10-24T03:15:25.229Z [DEBUG] (pool-2-thread-15) variant.TensorFlowLite: Created process with pid 27109. {serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-24T03:15:25.280Z [WARN] (Copier) variant.TensorFlowLite: stderr. + ml_root_path=. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
2024-10-24T03:15:25.280Z [INFO] (Copier) variant.TensorFlowLite: stdout. Running TensorFlow Lite install script.... {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}

    2024-10-24T03:17:29.657Z [WARN] (Copier) variant.TensorFlowLite: stderr. + pip3 install --find-links https://google-coral.github.io/py-repo/tflite-runtime/ tflite_runtime==2.5.0.post1 --force-reinstall. {scriptName=services.variant.TensorFlowLite.lifecycle.install.script, serviceName=variant.TensorFlowLite, currentState=NEW}
0

It seem that following command will uninstall my numpy

pip3 install --find-links https://google-coral.github.io/py-repo/tflite-runtime/ tflite_runtime==2.5.0.post1 --force-reinstall.
Jan
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content