SageMaker Autopilot , Ensemble / Autogluon , Realtime Inference Request Body

Question

Hello,

I'm following this AWS blog to run the Titantic problem in SageMaker AutoPilot + Ensemble + Realtime Inference.
https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-autopilot-is-up-to-eight-times-faster-with-new-ensemble-training-mode-powered-by-autogluon/

I've got the below real time inference request to return a **successful** response.  This only has 1 observation (one row of values to predict).  This is using the built-in JSON editor to post a request to a deployed model endpoint.
```
{
    "data": {
        "features": {
            "values":[892,3,"Kelly Mr. James","male",34.5,0,0,"330911",7.8292,"","Q"]
        }
    }
}
```

I want to send multiple observations (multiple rows of values to predict) at a time, using the realtime inference.  I feel like I've tried all the below common formats, and none of them are supported except the above.
https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-inference.html

Any ideas / Advice for me to try?

Answer

I was able to get multiple rows predicted.  It appears to only support CSV body format.

```
$> cat body 
892,3,"Kelly Mr. James","male",34.5,0,0,"330911",7.8292,"","Q"
892,3,"Kelly Mr. James","male",34.5,0,0,"330911",7.8292,"","Q"

$> aws sagemaker-runtime invoke-endpoint --endpoint-name your-endpoint --body fileb://body --content-type text/csv --accept application/json response
{
    "ContentType": "application/json",
    "InvokedProductionVariant": "default-variant-name"
}

$> cat response 
{"predictions": [{"predicted_label": "0", "labels": "['0', '1']", "probability": 0.8901974558830261, "probabilities": "[0.8901974558830261, 0.10980253666639328]"}, {"predicted_label": "0", "labels": "['0', '1']", "probability": 0.8901974558830261, "probabilities": "[0.8901974558830261, 0.10980253666639328]"}]}
```

SageMaker Autopilot , Ensemble / Autogluon , Realtime Inference Request Body

Relevant content