Using SageMaker as a backend for a web app

Question

[Continued/simplified from this post.](https://repost.aws/questions/QU-iZHXP55T_6BLKhHPET_0Q/if-i-open-a-web-socket-connection-with-one-of-my-sage-makers-notebook-cells-how-long-will-the-connection-last)

Let's say I have a web app UI and I want to connect it to a SageMaker notebook using the Churn Predictor model in SageMaker Sudio. Think of it like using that model hosted in SageMaker as a backend to my web app.

I could create an API endpoint but my web app has a very simple solution for opening a web socket which I'd like to use. Question: **Whats the simplest/best way to deploy the SageMaker notebook so that it keeps the kernel running and connection open?**

The websocket means I don't need to set up and configure individual API endpoints which is what all the documentation I can find suggests.

The desired workflow is:
- Configure model in SageMaker notebook environment 
- Add a cell that establishes a websocket connection
- Export notebook as a Python script (Maybe?)
- Host & Run Python script on an AWS service

With the web socket I can simply establish the connection:

`anvil.server.connect(app_key)`

And then give each function that I want my web app to call a decorator. i.e:
```
@anvil.server.callable
def function(foo):
  return foo
```

Answer

Hello,
According to the description, I assume you are going to use the Sagemaker notebooks for the development of the inference code and then export the code as a script.

If you model is light weight, you can cache the model inside you server application and call the predictions in it.

```
model = load_model()

@anvil.server.callable
def function(foo):
  return model.predict("")
```

Alternatively you can create a SageMaker endpoint and inside the same function above you can call the Sagemaker endpoint and pass it back to the client as a response to the web-socket invocation.

Using SageMaker as a backend for a web app

Relevanter Inhalt