Batch transform on a big npy file - error 413

Question

I have this case where I try to make a batch transform on a very sizable npy file. The file contains a 3D array (3000, 100, 100) where each observation is one element along the first axis and has the dim (1, 100, 100) for a total of 3000 observations.

Now I worked with batch transform in the past and for .csv or .jsonlines files it is a relatively easy way with split_type='Line' in the transform method. However, when I try to pass the big npy file, it does not seem to work with split_type='Line'. It works without it on smaller files, and the batch transform is successful. The image I am using is a pytorch cpu one, and the code that works for smaller files is:
```
predictor = Transformer(model_name='testingdep3',
                        instance_count= 1,
                        instance_type= 'ml.m5.large',
                        strategy= "SingleRecord",
                        output_path= 's3://xxx/inferencetesting/output/',
                        accept= "application/x-npy")
```

```
predictor.transform(data='s3://xxx/inferencetesting/testdata111.npy',
                    content_type='application/x-npy',
                    # split_type='Line'
                   )
```

My input_fn and predict_fn functions are as follows:

```
def input_fn(request_body, content_type='application/x-npy'):
    return np.load(io.BytesIO(request_body), allow_pickle=True)
```

```
def predict_fn(input_object, model):
    predictions = model.predict(input_object)
    return predictions
```

The maximum acceptable payload is MaxPayloadInMB=6 for this specific image. I was thinking of different ways of approaching this problem, including:

- having multiple smaller .npy files in the data directory
- changing the npy file to a protobuf format and then changing my input_fn to account for that, as well as uncommenting the split_line argument with the RecordIO value

Is there really no way to actually feed a very big numpy file to a batch transform job so that it's split somehow by using the standard arguments split_type and/or strategy?

Wael · Answer

You are providing `.npy` file which is not a splitable type. So your transform job will only work in SingleRecord Batch Strategy.

To be able to work with files larger than 100MB you can do one of the following:

- Use processing job to run predictions (see below example for scikit learn framework):
```
https://sagemaker-examples.readthedocs.io/en/latest/sagemaker_processing/scikit_learn_data_processing_and_model_evaluation/scikit_learn_data_processing_and_model_evaluation.html#Model-Evaluation 
```

Here you basically provide the model, inference code and input data to the processing job as input channels and your code should be able to produce inferences and pass them as an output.

- Or, compress your input file, then when you provide it to batch transform job, your inference code should handle extracting data and inference at the same time.

Batch transform on a big npy file - error 413

Add your answer

Relevant content