Sagemaker Neo compiled model not using NVIDIA GPU

I have a YOLOv5-trained model. Exported as Tensorflow, I'm trying to compile and deploy to my device as a Greengrass Noe edge component.

Device configs are, device: NVIDIA Jetson AGX Xavier Jetpack:5.1 CUDA: 11.4 TensorRT:8.5

I compiled the model with different configurations.

**Compilation 1. ** model:tensorflow 2.4 Platform: Linux Arch: ARM64

Can able to load the model and run inference. But its running on CPU and the inference time is very high.

**Compilation 2. ** model:tensorflow 2.4 Platform: Linux Arch: ARM64 Accelerator: NVIDIA Config: {"gpu-code":"sm_62","trt-ver": "8.5", "cuda-ver": "11.4"}

Unable to load the model. GRPC error : "UNKNOWN:Error received from peer unix:/tmp/aws.greengrass.SageMakerEdgeManager.sock {grpc_message:"LoadModel request failed", grpc_status:13"

**Compilation 3. ** model:tensorflow 2.4 Device: jetson_xavier Config: {"trt-ver": "6.0.1", "cuda-ver": "10.0"}

Unable to load the model. GRPC error : "UNKNOWN:Error received from peer unix:/tmp/aws.greengrass.SageMakerEdgeManager.sock {grpc_message:"LoadModel request failed", grpc_status:13"

Either the not running on GPU or the deployed model I was not able to load. Cloud you please help me over here.