Deploy an model trained using Sagemaker's built-in k-NN algorithm for AWS Panorama

Question

Hi, I'm trying to use a k-Nearest Neighbour model for deployment to an Edge device (AWS Panorama). I understand I need to create an optimized model to suit the target device. As I understand it the built-in algorithms all include a SageMaker Neo 'compile_model' function which I am running over my trained model -

Build model..
```
knn = sagemaker.estimator.Estimator(container,
                                       role, 
                                       instance_count=1, 
                                       instance_type='ml.m5.2xlarge',
                                       output_path=output_location,
                                       sagemaker_session=sess,
                                       input_mode='Pipe'
                                   )
# Setup the hyperparameters
knn.set_hyperparameters(**hyperparams)

knn.fit(fit_input, job_name=job_name)
```

Build optimised model...
```
optimized_ic = knn.compile_model(
    target_instance_family="ml_c5",
    target_platform_os="LINUX",
    target_platform_arch="ARM64",
    input_shape={"data": [1,3,512,512]},
    output_path=s3_optimized_output_location,
    framework="mxnet",
    framework_version="1.8",
)
```
While the initial model builds fine I get an error running the optimised model / compile_model function:

> Failed. Reason: ClientError: InputConfiguration: No valid Mxnet model file -symbol.json found. Please make sure the framework you select is correct.

I can find a few google references for attempts at something similar with KMeans algorithm, but is k-NN algorithm dramatically different? Do I just have the wrong settings? I understand all of the built-in algorithms use were originally mxnet trained? When I extract my model I only have three files:
- model_algo-1
- model_algo-1.json
- model_algo-1-labels.npy

I am struggling to work out where I am going wrong in porting this model!

Answer

The default Panorama service leverages NEO to compile and run the models.  Unfortunately, the 1P Sagemaker algorithms are not compatible with NEO.  Also, the Panorama service leverages Nvidia Jetson hardware, which is not compatible with the 'ml.m5.2xlarge' hardware.  With that said, Panorama now supports an option to "bring your own runtime" to Panorama that has recently been introduced that would allow you to leverage 1P Sagemaker models.  If you were to train your model with a GPU enabled instance (p3 or g4dn), the process would involve building your own custom container for Panorama with the MXNET 1.4 version installed.  If this is still something you are pursuing, reach out and I can help walk you through the process.

Deploy an model trained using Sagemaker's built-in k-NN algorithm for AWS Panorama

Relevant content