Visualize Batch-Transform-Model-Quality-On-schedule Result Metrics


Following AWS Sagemaker example Notebook. " Batch-Transform-Model-Quality-On-schedule" and trying to visualize model quality metrics results same like real time endpoint results. How can i do this?

1 Answer

You can use Amazon SageMaker Model Monitor to monitor a batch transform job like a real-time endpoint. In this case, instead of receiving requests to an endpoint and tracking the predictions, Model Monitor will monitor inference inputs and outputs. Here the announcement and the standard documentation. You can also find samples notebooks for Model Monitor with Batch Transform in this Github repository.

Note that, batch transform data capture is unlike Endpoint data capture, it does not capture the data and log to s3, Instead, batch transform captures data in manifests. The manifests contain the source transform input or output s3 locations.

Like for Endpoints you can create a model monitoring schedule, you can inspect a specific execution, and you can list the generated reports, as well as the violation reports. These reports are saved in S3 in JSON format and you can visualize them in the same way as for the real-time endpoints.

answered a year ago
  • Thanks for this but can you explain how can I visualize reports in graphs using batch transform because in endpoints there is built-in option of visualize at sagemaker studio.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions