Sagemaker Async Inference Endpoint: Linking input identifier, container log, output identifier?

0

Hi, based on https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb I've created a docker image which contains the model code implemented in Python. This model image is deployed as async inference endpoint. Everything works well, the endpoint can be invoked via InvokeEndpointAsync, custom attributes are provided to the model code via "X-Amzn-SageMaker-Custom-Attributes" header, the input located on S3 is provided as http body. After succesful processing the result is uploaded to S3 with an automatically generated output filename ("*.out"). The container behavior is logged to CloudWatch via Python logging module, initialized as:

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

However, the request arriving inside the container neither contains an "X-Amzn-SageMaker-Inference-Id" nor "X-Amzn-SageMaker-InputLocation" field (mentioned here https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpointAsync.html). Is there a way to access the inference_id/output filename from inside the container? Or any other opportunity to link input filename/container log/output filename? Otherwise it's not possible to fully track and examine requests.

Thanks in advance for your advice!

asked 2 years ago315 views
1 Answer
0

Hello,

Thank you for using AWS Sagemaker.

On investigating the issue that you have shared with us. I got the opportunity to work with the internal team on this issue, and it was identified that at this moment Async Inference is not logging those details in the container.

That being said, I have raised a feature request with the service team on your behalf. While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our What's New and Blog pages for any new feature announcements.

AWS
SUPPORT ENGINEER
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions