Sagemaker Async Inference Endpoint: Linking input identifier, container log, output identifier?

0

Hi, based on https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb I've created a docker image which contains the model code implemented in Python. This model image is deployed as async inference endpoint. Everything works well, the endpoint can be invoked via InvokeEndpointAsync, custom attributes are provided to the model code via "X-Amzn-SageMaker-Custom-Attributes" header, the input located on S3 is provided as http body. After succesful processing the result is uploaded to S3 with an automatically generated output filename ("*.out"). The container behavior is logged to CloudWatch via Python logging module, initialized as:

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

However, the request arriving inside the container neither contains an "X-Amzn-SageMaker-Inference-Id" nor "X-Amzn-SageMaker-InputLocation" field (mentioned here https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpointAsync.html). Is there a way to access the inference_id/output filename from inside the container? Or any other opportunity to link input filename/container log/output filename? Otherwise it's not possible to fully track and examine requests.

Thanks in advance for your advice!

gefragt vor 2 Jahren324 Aufrufe
1 Antwort
0

Hello,

Thank you for using AWS Sagemaker.

On investigating the issue that you have shared with us. I got the opportunity to work with the internal team on this issue, and it was identified that at this moment Async Inference is not logging those details in the container.

That being said, I have raised a feature request with the service team on your behalf. While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our What's New and Blog pages for any new feature announcements.

AWS
SUPPORT-TECHNIKER
beantwortet vor 2 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen