How do you pass an S3 path as a parameter in an asynchronous inference call to a model in SageMaker?

Question

I need to make different calls to the endpoint, and for each call, I need to specify a different path for the model results to be saved.

Based on this documentation https://boto3.amazonaws.com/v1/documentation/api/1.26.83/reference/services/sagemaker-runtime/client/invoke_endpoint_async.html, it's possible to pass the path to the data to be processed as a parameter. However, I can't find a way to pass the S3 path for the model output, since I want the result to be saved in a different result set for each call.

Is this possible?

re:Post Agent · Answer

Yes, it is possible to specify a different S3 path for the model results to be saved for each asynchronous inference call in Amazon SageMaker. However, the approach is slightly different from what you're describing.

When setting up an asynchronous inference endpoint, you typically specify a base output path in the AsyncInferenceConfig object during endpoint configuration. This base path is set once for the endpoint.

For individual inference calls, you don't directly specify the output S3 path in the InvokeEndpointAsync API call. Instead, SageMaker automatically generates a unique output location for each request within the base output path you specified.

Here's how you can achieve what you're looking for:

1. When creating the endpoint, specify a base output path in the AsyncInferenceConfig:

```python
async_config = AsyncInferenceConfig(
    output_path=s3_path_join("s3://", bucket_name, "base_output_path/")
)
```

2. When invoking the endpoint asynchronously, you'll receive a response that includes the specific output location for that request:

```python
response = sagemaker_runtime.invoke_endpoint_async(
    EndpointName='your-endpoint-name',
    InputLocation='s3://input-bucket/input-file.csv',
    InferenceId=str(uuid.uuid4()),
    ContentType='text/csv'
)

output_location = response['OutputLocation']
```

3. You can then use this output_location to retrieve the results for each specific call.

This approach ensures that each inference request gets its own unique output location within the base path you specified, effectively giving you different result sets for each call.

If you need more control over the exact output path for each call, you might consider using a pre-processing step to generate a unique output path for each input, and then pass this information along with your input data. The endpoint could then use this information to determine where to save the results. However, this would require custom implementation in your endpoint code.

Remember, when using asynchronous inference, it's also possible to set up Amazon SNS notifications for successful completions or errors, which can be helpful for tracking the status of your inference jobs.  
**Sources**  
[Asynchronous inference - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/async-inference.html)  
[Accelerate your ML lifecycle using the new and improved Amazon SageMaker Python SDK – Part 2: ModelBuilder | AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/accelerate-your-ml-lifecycle-using-the-new-and-improved-amazon-sagemaker-python-sdk-part-2-modelbuilder/)  
[Check prediction results - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/async-inference-check-predictions.html)

How do you pass an S3 path as a parameter in an asynchronous inference call to a model in SageMaker?

Relevant content