Parsing logs in Sagemaker Endpoints

0

I have deployed a sagemaker endpoint and I'm trying to get my logs logged to the cloudwatch logstream. I did so by plainly printing the dictionary to stdout. While the logs get logged, I'm unable to parse them as sagemaker internally appends it's own prefix, making doing any kind of downstream analytics difficult

Here's an eg of what gets logged to the logstream


@message	
2023-10-10T14:20:30,885 [INFO ] W-9000-model_1.0-stdout MODEL_LOG - {'level': 'info', 'timestamp': '2023-10-10 14:20:30 GMT', 'service': 'lorem ipsum', 'description': 'some desc', 'log_event': 'lorem', 'environment': 'env', 'model_version': 'v0', 'call_id': 'some uuid'}

Is there a good way to bypass this prefix(2023-10-10T14:20:30,885 [INFO ] W-9000-model_1.0-stdout MODEL_LOG), so I can make the log parsable?

1개 답변
0

Hello.

Parsing Logs in CloudWatch: Since the prefix remains consistent, you can use a combination of CloudWatch insights query language or other log processing tools to split the log lines into prefix and the actual log message, and then parse the JSON content.

For instance, using CloudWatch Insights, you can structure your query like:

fields @timestamp, @message
| parse @message as prefix, json_content
| filter @message like "MODEL_LOG"

This will give you the raw JSON logs in the json_content variable which you can then further parse or manipulate as needed.

Best regards, Andrii

profile picture
전문가
답변함 7달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠