I have a YOLOv5-trained model. Exported as Tensorflow, I'm trying to compile and deploy to my device as a Greengrass Noe edge component.
Device configs are,
device: NVIDIA Jetson AGX Xavier
Jetpack:5.1
CUDA: 11.4
TensorRT:8.5
I compiled the model with different configurations.
**Compilation 1. **
model:tensorflow 2.4
Platform: Linux
Arch: ARM64
Can able to load the model and run inference. But its running on CPU and the inference time is very high.
**Compilation 2. **
model:tensorflow 2.4
Platform: Linux
Arch: ARM64
Accelerator: NVIDIA
Config: {"gpu-code":"sm_62","trt-ver": "8.5", "cuda-ver": "11.4"}
Unable to load the model.
GRPC error : "UNKNOWN:Error received from peer unix:/tmp/aws.greengrass.SageMakerEdgeManager.sock {grpc_message:"LoadModel request failed", grpc_status:13"
**Compilation 3. **
model:tensorflow 2.4
Device: jetson_xavier
Config: {"trt-ver": "6.0.1", "cuda-ver": "10.0"}
Unable to load the model.
GRPC error : "UNKNOWN:Error received from peer unix:/tmp/aws.greengrass.SageMakerEdgeManager.sock {grpc_message:"LoadModel request failed", grpc_status:13"
Either the not running on GPU or the deployed model I was not able to load.
Cloud you please help me over here.