Error Invoking endpoint deployed locally using SageMaker SDK for a xgboost model

Question

I am deploying a SageMaker endpoint locally for xgboost model and running to some issues when invoking the endpoint.

I am able to successfully deploy the endpoint in local model using the following code sample:

```
from sagemaker.xgboost import XGBoost, XGBoostModel
from sagemaker.session import Session

xgb_inference_model = XGBoostModel(
                                   model_data=models3_uri,
                                   role=role,
                                   entry_point="inference.py",
                                   framework_version="0.90-2",
                                   sagemaker_session = None # sagemaker_session if cloud / prod mode     
)

print('Deploying endpoint in local mode')
predictor = xgb_inference_model.deploy(
                                       initial_instance_count = 1,
                                       #instance_type="ml.m5.xlarge",
                                       instance_type = "local",
)

```

I have the `inference.py` that includes the functions for accepting input, making predictions and output. 
Link here: https://github.com/aws-samples/amazon-sagemaker-local-mode/blob/main/xgboost_script_mode_local_training_and_serving/code/inference.py

The issue I am running into is with type of data that `input_fn` accepts. I have tried passing passing a numpy array  / dataframe / bytes object as input data, but still get the error.

```
def input_fn(request_body, request_content_type):
    """
    The SageMaker XGBoost model server receives the request data body and the content type,
    and invokes the `input_fn`.
    
    Return a DMatrix (an object that can be passed to predict_fn).
    """
    # Handle numpy array type
    if request_content_type == "application/x-npy":
        
        print(type(request_body))
        
        array = np.load(BytesIO(request_body))
        
        return xgb.DMatrix(request_body)
    
    if request_content_type == "text/csv":
        
        print("request body", request_body)
        
        # change to request_body to Pandas DataFrame
 
        return xgb_encoders.libsvm_to_dmatrix(request_body)
        #perform encoding on the input data here
    
    else:
        raise ValueError("Content type {} is not supported.".format(request_content_type))

```

Encoder object. Training data is encoded before fit. Doing the same with test data. Posting here for reference.

I tried making predictions using NumpySerializer and CSVSerializer [1]. Both don't work -

```
from sagemaker.serializers import NumpySerializer
predictor.serializer = NumpySerializer()

testpoint = encoder.transform(df).toarray()

print(testpoint)

[[0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00 0.000e+00
  0.000e+00 0.000e+00 0.000e+00.........1.000e+01 2.000e+00 5.312e-02]]
```

Traceback with CSVSerializer(), when passing body of type text/csv

```
Exception on /invocations [POST]
 | Traceback (most recent call last):

File "/opt/ml/code/inference.py", line 35, in input_fn
return xgb_encoders.libsvm_to_dmatrix(request_body)
packages/sagemaker_xgboost_container/encoder.py", line 65, in libsvm_to_dmatrix
TypeError: a bytes-like object is required, not 'str'
```

Traceback with NumpySerializer() when passing  type body

```
Exception on /invocations [POST]
Traceback (most recent call last):
raise TypeError('no supported conversion for types: %r' % (args,))
TypeError: no supported conversion for types: (dtype('O'),)
```

[1] https://sagemaker.readthedocs.io/en/stable/api/inference/serializers.html

Answer

You appear to have conflicting data types in your `input_fn`:

```python
    if request_content_type == "text/csv":
        
        print("request body", request_body)
        
        # change to request_body to Pandas DataFrame
 
        return xgb_encoders.libsvm_to_dmatrix(request_body)
        #perform encoding on the input data here
```

Probably the logic you want is:

```python
    if request_content_type == "text/csv":
        return xgb_encoders.csv_to_dmatrix(request_body)
    elif request_content_type == "text/libsvm":
        return xgb_encoders.libsvm_to_dmatrix(request_body)
```

Answer

Since the error is occuring in the call to libsvm_to_dmatrix(), the content type is set to "text/csv".  Have you tried passing in a csv string?

Error Invoking endpoint deployed locally using SageMaker SDK for a xgboost model

Relevant content