Visualising batch transform model monitoring results within sagemaker

Question

I've set up a model and run regular batch transforms on it. Data capture is enabled and there is a data quality monitoring schedule that runs every hour. The problem I have is linking the results of the data quality monitor to the model and viewing it within sagemaker. Right now all the examples suggest using a notebook to view the results of a data quality monitor for batch transforms.

1. Can we visualise the results within sagemaker like data monitoring on real time endpoints? All examples suggest using a notebook to visualise the results. I don't want to keep running a notebook every time to visualise results.
2. Can we link the data quality schedule to model for batch transform jobs? Right now the monitoring jobs take batch transform output as input and are not linked to the model in any way.

The work around I have come up is using a lambda instead of the notebook to serve a static page with results so that it could be easily accessed without having to run a notebook every time.

Answer

Hi,

Following are couple of AWS documentations that  addresses the requirement mentioned above without having to configure lambda.

1. Result Visualization
https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-interpreting-results.html

https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-interpreting-visualize-results.html

2. On-Demand Monitoring of Batch Transform Jobs:
        https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/model_monitor_batch_transform/SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb

Answer

Hi Vik,

Thank you for the clarification.  Quoting your queries from above questions in highlighted text followed by my response

**1. Can we visualise the results within sagemaker like data monitoring on real time endpoints? All examples suggest using a notebook to visualise the results. I don't want to keep running a notebook every time to visualise results.** --> With reference to this query, the answer lies in this link - https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-interpreting-visualize-results.html however , visualization on batch inferences is not supported as of now using this feature.

**2. Can we link the data quality schedule to model for batch transform jobs? Right now the monitoring jobs take batch transform output as input and are not linked to the model in any way.** --> Following are couple of reference article and code sample that could help you move needle on this problem
       https://aws.amazon.com/blogs/machine-learning/create-sagemaker-pipelines-for-training-consuming-and-monitoring-your-batch-use-cases/

https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/model_monitor_batch_transform/SageMaker-ModelMonitoring-Batch-Transform-Model-Quality-With-SageMaker-Pipelines-On-Demand.ipynb

I hope this gives you direction towards your solution.

Visualising batch transform model monitoring results within sagemaker

Relevant content