greengrass v2 ml component

0

hello I created aws greengrass v2 ml component.

I want to run the ml model on Raspberry Pi with greengrass installed. So I wrote the ml component and deployed it. The result I expect is that the ml code I wrote is executed on Raspberry Pi.

This is a custom component. I referred to https://docs.aws.amazon.com/greengrass/v2/developerguide/ml-customization.html this link.

But there is a problem. If I deployed the components, say that the Deploy is complete. But there's an error. Looking at the log, it seems that there is a problem opening the virtual environment(venv), or it is because the tensorflow is not downloaded

I want to write a script that opens the virtual environment properly in the recipe of the component. The recipe I wrote looks like this. { "RecipeFormatVersion": "2020-01-25", "ComponentName": "com.example.jamesML", "ComponentVersion": "1.0.5", "ComponentType": "aws.greengrass.generic", "ComponentDescription": "Capstone Design james machine learning.", "ComponentPublisher": "Me", "ComponentConfiguration": { "DefaultConfiguration": { "accessControl": { "aws.greengrass.ipc.mqttproxy": { "com.example.jamesML:mqttproxy:1": { "policyDescription": "Allows access to publish via topic ml/dlr/image-classification.", "operations": [ "aws.greengrass#PublishToIoTCore" ], "resources": [ "ml/dlr/image-classification" ] } } } } }, "Manifests": [ { "Platform": { "os": "linux", "architecture": "arm" }, "Lifecycle": { "setEnv": { "Script": "cd venv&&activate venv" }, "run": { "RequiresPrivilege": "true", "script": "pip install opencv-python;pip install tensorflow==2.3;python3 james.py" } }, "Artifacts": [ { "Uri": "s3://greengrass-sagemaker-0930/james_ml.zip", "Digest": "CY6f7pUyMRrgbxbKoDMig3GWQCJ4LvKyA5xnhGWGhlY=", "Algorithm": "SHA-256", "Unarchive": "ZIP", "Permission": { "Read": "OWNER", "Execute": "NONE" } } ] } ], "Lifecycle": {} } Can you solve the problem here? Is this part I wrote wrong? "setEnv": { "Script": "cd venv&&activate venv" }, I want to know the answer. Please. Please.

This is my greengrass v2 component log (Raspberry Pi) 2022-10-13T10:01:44.004Z [INFO] (pool-2-thread-28) com.example.jamesML: shell-runner-start. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=STARTING, command=["pip install opencv-python;pip install tensorflow==2.3;python3 james.py"]} 2022-10-13T10:01:45.326Z [INFO] (Copier) com.example.jamesML: stdout. Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:45.329Z [INFO] (Copier) com.example.jamesML: stdout. Requirement already satisfied: opencv-python in /usr/local/lib/python3.9/dist-packages (4.6.0.66). {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:45.373Z [INFO] (Copier) com.example.jamesML: stdout. Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.9/dist-packages (from opencv-python) (1.23.4). {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:45.901Z [WARN] (Copier) com.example.jamesML: stderr. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:47.470Z [INFO] (Copier) com.example.jamesML: stdout. Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:48.909Z [WARN] (Copier) com.example.jamesML: stderr. ERROR: Could not find a version that satisfies the requirement tensorflow==2.3 (from versions: none). {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:48.910Z [WARN] (Copier) com.example.jamesML: stderr. ERROR: No matching distribution found for tensorflow==2.3. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:49.161Z [WARN] (Copier) com.example.jamesML: stderr. python3: can't open file 'james.py': [Errno 2] No such file or directory. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T10:01:49.169Z [INFO] (Copier) com.example.jamesML: Run script exited. {exitCode=2, serviceName=com.example.jamesML, currentState=RUNNING}

hyorim
asked 2 years ago548 views
1 Answer
1
Accepted Answer

Hi,

your recipe seems wrong. setenv is used to set environment variables and in your case you are creating a variable named Script with value cd venv&&activate venv. Move the cd venv&&activate venv inside the Run lifecycle script. You should also avoid using "RequiresPrivilege": "true" and instead assign the correct permission to the the user executing the script (eg ggcuser).

The fact that tensorflow is not found might be related to the fact that the virtualenv is not activated.

Massimiliano

AWS
EXPERT
answered 2 years ago
profile picture
EXPERT
reviewed a month ago
  • Thank you for answering me! I did it as you told me. { "RecipeFormatVersion": "2020-01-25", "ComponentName": "com.example.jamesML", "ComponentVersion": "1.0.6", "ComponentType": "aws.greengrass.generic", "ComponentDescription": "Capstone Design james machine learning.", "ComponentPublisher": "Me", "ComponentConfiguration": { "DefaultConfiguration": { "accessControl": { "aws.greengrass.ipc.mqttproxy": { "com.example.jamesML:mqttproxy:1": { "policyDescription": "Allows access to publish via topic ml/dlr/image-classification.", "operations": [ "aws.greengrass#PublishToIoTCore" ], "resources": [ "ml/dlr/image-classification" ] } } } } }, "Manifests": [ { "Platform": { "os": "linux", "architecture": "arm" }, "Lifecycle": { "run": { "script": "cd venv&&activate venv&&pip install opencv-python&&pip install tensorflow==2.3&&python3 james.py" } }, "Artifacts": [ { "Uri": "s3://greengrass-sagemaker-0930/james_ml.zip", "Digest": "CY6f7pUyMRrgbxbKoDMig3GWQCJ4LvKyA5xnhGWGhlY=", "Algorithm": "SHA-256", "Unarchive": "ZIP", "Permission": { "Read": "OWNER", "Execute": "NONE" } } ] } ], "Lifecycle": {} } Is this how you told me to do it?

  • Even so, I have an error again.

    2022-10-13T12:33:21.675Z [INFO] (pool-2-thread-33) com.example.jamesML: shell-runner-start. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=STARTING, command=["cd venv&&activate venv&&pip install opencv-python&&pip install tensorflow==2.3..."]} 2022-10-13T12:33:21.755Z [WARN] (Copier) com.example.jamesML: stderr. sh: 1: cd: can't cd to venv. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T12:33:21.762Z [INFO] (Copier) com.example.jamesML: Run script exited. {exitCode=2, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T12:33:23.308Z [INFO] (pool-2-thread-32) com.example.jamesML: shell-runner-start. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=STARTING, command=["cd venv&&activate venv&&pip install opencv-python&&pip install tensorflow==2.3..."]} 2022-10-13T12:33:23.329Z [WARN] (Copier) com.example.jamesML: stderr. sh: 1: cd: can't cd to venv. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T12:33:23.334Z [INFO] (Copier) com.example.jamesML: Run script exited. {exitCode=2, serviceName=com.example.jamesML, currentState=RUNNING}

  • 2022-10-13T12:33:25Z [INFO] (pool-2-thread-32) com.example.jamesML: shell-runner-start. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=STARTING, command=["cd venv&&activate venv&&pip install opencv-python&&pip install tensorflow==2.3..."]} 2022-10-13T12:33:25.022Z [WARN] (Copier) com.example.jamesML: stderr. sh: 1: cd: can't cd to venv. {scriptName=services.com.example.jamesML.lifecycle.run.script, serviceName=com.example.jamesML, currentState=RUNNING} 2022-10-13T12:33:25.026Z [INFO] (Copier) com.example.jamesML: Run script exited. {exitCode=2, serviceName=com.example.jamesML, currentState=RUNNING} This is my raspberry pie log.

    Could you answer this, too? I'm begging you.

  • Hi. What is creating the virtual environment? It seems it has not been created, hence you can't change directory into it. Typically it would be created in the Install lifecycle. Massimiliano has an example here: https://github.com/awslabs/aws-greengrass-labs-jupyterlab/blob/main/recipes/aws.greengrass.labs.jupyterlab.yaml

  • Please note that to activate a virtual environment you should use . venv/bin/activate. Do not use source venv/bin/activate in a component recipe since the commands are run via /bin/sh by default which doesn't have the source command.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions