Hello,

I understand that you are concerned about how to import the dependent py files and make sure they are available in the container while using model monitoring and would like to gather more information on the same.

Firstly, with Amazon SageMaker Model Monitor, you can select the data you would like to monitor and analyze without the need to write any code. SageMaker Model Monitor lets you select data from a menu of options, such as prediction output, and captures metadata such as timestamp, model name, and endpoint, so you can analyze model predictions based on the metadata. You can specify the sampling rate of data capture as a percentage of overall traffic in the case of high-volume real-time predictions, and the data is stored in your own Amazon S3 bucket. You can also encrypt this data, configure fine-grained security, define data retention policies, and implement access control mechanisms for secure access.

After you deploy a model into your production environment, use the Amazon SageMaker model monitor to continuously monitor the quality of your machine learning models in real time. The Amazon SageMaker model monitor enables you to set up an automated alert triggering system when there are deviations in the model quality, such as data drift and anomalies. Amazon CloudWatch Logs collects log files for monitoring the model status and notifies you when the quality of your model hits certain thresholds that you preset. CloudWatch stores the log files to an Amazon S3 bucket you specify. Early and pro-active detection of model deviations through AWS model monitor products enables you to take prompt actions to maintain and improve the quality of your deployed model.

Furthermore, I would like to mention that you can't actually import a few dependent Python scripts. The pre-processing script should be a single Python file. One workaround that might work is to bundle the dependency files as a Python package, publish them to a repository, and then install it in the preprocessing script itself.

I would request that you please refer to the aforementioned documentation once, and please reach out to AWS [4] with the detailed use case so that we can assist you better.

If you have any difficulty verifying any of the above-mentioned points or if you still run into issues, please reach out to AWS Support [4] (Sagemaker) along with your issue or use case in detail, and we would be happy to assist you further.

References:

[1] https://aws.amazon.com/sagemaker/model-monitor/

[2] Monitoring a Model in Production - https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-model-monitor.html

[3] Monitor data and model quality - https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor.html

[4] Creating support cases and case management - https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-casehttps://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-case

Hello,

Thank you for using AWS SageMaker.

While using Model Monitoring feature of SageMaker along with custom container, Under the hood, when you create a MonitoringSchedule, Model Monitor ultimately kicks off processing jobs. Hence the container needs to be aware of the processing job contract documented in the Build Your Own Processing Container (Advanced Scenario) topic. So when you build the docker container, make sure that has correct and needed dependencies to be utilized for model monitoring. So to answer your question, all the dependencies can be verify with your docker container built file before the docker is been pushed to ECR. I'm also sharing reference of the AWS blog and public documentation for your reference.

Reference:

[1] https://aws.amazon.com/blogs/machine-learning/bring-your-own-container-to-project-model-accuracy-drift-with-amazon-sagemaker-model-monitor/

[2] https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-byoc-containers.html