Sagemaker Async Inference Endpoint: Linking input identifier, container log, output identifier?

0

Hi, based on https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb I've created a docker image which contains the model code implemented in Python. This model image is deployed as async inference endpoint. Everything works well, the endpoint can be invoked via InvokeEndpointAsync, custom attributes are provided to the model code via "X-Amzn-SageMaker-Custom-Attributes" header, the input located on S3 is provided as http body. After succesful processing the result is uploaded to S3 with an automatically generated output filename ("*.out"). The container behavior is logged to CloudWatch via Python logging module, initialized as:

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

However, the request arriving inside the container neither contains an "X-Amzn-SageMaker-Inference-Id" nor "X-Amzn-SageMaker-InputLocation" field (mentioned here https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpointAsync.html). Is there a way to access the inference_id/output filename from inside the container? Or any other opportunity to link input filename/container log/output filename? Otherwise it's not possible to fully track and examine requests.

Thanks in advance for your advice!

已提問 2 年前檢視次數 324 次
1 個回答
0

Hello,

Thank you for using AWS Sagemaker.

On investigating the issue that you have shared with us. I got the opportunity to work with the internal team on this issue, and it was identified that at this moment Async Inference is not logging those details in the container.

That being said, I have raised a feature request with the service team on your behalf. While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our What's New and Blog pages for any new feature announcements.

AWS
支援工程師
已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南