Sagemaker Async Inference Endpoint: Linking input identifier, container log, output identifier?

0

Hi, based on https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb I've created a docker image which contains the model code implemented in Python. This model image is deployed as async inference endpoint. Everything works well, the endpoint can be invoked via InvokeEndpointAsync, custom attributes are provided to the model code via "X-Amzn-SageMaker-Custom-Attributes" header, the input located on S3 is provided as http body. After succesful processing the result is uploaded to S3 with an automatically generated output filename ("*.out"). The container behavior is logged to CloudWatch via Python logging module, initialized as:

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

However, the request arriving inside the container neither contains an "X-Amzn-SageMaker-Inference-Id" nor "X-Amzn-SageMaker-InputLocation" field (mentioned here https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpointAsync.html). Is there a way to access the inference_id/output filename from inside the container? Or any other opportunity to link input filename/container log/output filename? Otherwise it's not possible to fully track and examine requests.

Thanks in advance for your advice!

preguntada hace 2 años323 visualizaciones
1 Respuesta
0

Hello,

Thank you for using AWS Sagemaker.

On investigating the issue that you have shared with us. I got the opportunity to work with the internal team on this issue, and it was identified that at this moment Async Inference is not logging those details in the container.

That being said, I have raised a feature request with the service team on your behalf. While I am unable to comment on if/when this feature may get released, I request you to keep an eye on our What's New and Blog pages for any new feature announcements.

AWS
INGENIERO DE SOPORTE
respondido hace 2 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas