how to configure max concurrent transforms and instance count parameter in batch transform ?

Question

configuring a batch job for inference in sagemaker ( sample code below) . for my use case,  there are multiple input files and i'm trying to configure such that it can process files one by one or if the instance type allows process them in parallel. if value of max concurrrent transforms is greater than 1, does it processes file in parallel. as things are predefined for the job , before it starts , for exampel the instance_count is already set to 1. how does it do horizontal scaling? can it add add more instances. does the value of instance type or instance count dictate what value we can configure for max concurrent transforms parameter?

```
from sagemaker.transformer import Transformer

transformer = Transformer(model_name='my-previously-trained-model',
                          max_concurrent_transforms=0
                          instance_count=1,
                          instance_type='ml.m4.xlarge')
```

Answer

To add to it, you can set the [BatchStrategy](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html#sagemaker-CreateTransformJob-request-BatchStrategy) to `MultiLine` in order to speed up the processing.
General guideline is - number of workers/instances is a multiple of number of files in S3 to process.
If MaxConcurrentTransforms is set to 0 or left unset, Amazon SageMaker checks the optional execution-parameters to determine the settings for your chosen algorithm

Answer

It partitions the Amazon S3 objects in the input by key. Please checkout https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html

how to configure max concurrent transforms and instance count parameter in batch transform ?

Relevant content