Skip to content

DefaultModelMonitor.run_monitoring_job missing in SageMaker SDK 2.254.1

0

I am using Amazon SageMaker AI Model Monitor and trying to run an ON-DEMAND (data-quality) monitoring job using DefaultModelMonitor.

Code:

from sagemaker.model_monitor import DefaultModelMonitor, EndpointInput

monitor = DefaultModelMonitor( role=role, instance_count=1, instance_type="ml.m5.large", volume_size_in_gb=40, max_runtime_in_seconds=3600, sagemaker_session=sess, )

monitor.run_monitoring_job( monitoring_job_name="fincrimes-data-quality-now", endpoint_input=EndpointInput( endpoint_name=ENDPOINT_NAME, destination="/opt/ml/processing/input/endpoint", ), output_s3_uri="s3://<bucket>/<prefix>/monitoring-results/on-demand/", statistics=stats_s3, constraints=const_s3, wait=True, logs=True, )

Error:

AttributeError: 'DefaultModelMonitor' object has no attribute 'run_monitoring_job'

Environment:

  • SageMaker SDK version: 2.254.1
  • Python: 3.10.19 (conda-forge)
  • OS: Amazon Linux 2 (glibc 2.26)
  • Endpoint: SageMaker endpoint with data capture enabled

I confirmed that run_monitoring_job is not present in this SDK version: hasattr(DefaultModelMonitor, "run_monitoring_job") == False

Questions:

  1. Is run_monitoring_job expected to be available in SageMaker SDK 2.254.1?
  2. If not, what is the AWS-recommended way to run an ON-DEMAND Model Monitor data-quality job (without creating a recurring schedule)?
  3. Is the supported workaround to create a monitoring schedule, wait for the first execution, and then stop/delete it?

Any official guidance or documentation would be helpful.

2 Answers
0

Based on the information available, the run_monitoring_job method does not appear to be available in the DefaultModelMonitor class in SageMaker SDK version 2.254.1.

For running on-demand monitoring jobs in SageMaker Model Monitor, the recommended approach is to use the CreateMonitoringSchedule API. You can create a one-time monitoring schedule that executes immediately rather than creating a recurring schedule.

To run an on-demand data quality monitoring job, you would:

  1. Create a monitoring schedule with a one-time execution using the CreateMonitoringSchedule API
  2. View the status of your monitoring job in the SageMaker Model Dashboard under the "Monitor schedule" section
  3. Check the "Status schedule" column to track the progress (which can be "Failed," "Pending," "Stopped," or "Scheduled")

You can also monitor the job through the SageMaker console by navigating to the Governance section and selecting Model Dashboard, then finding your model and checking the monitoring schedule section.

The approach you suggested (creating a monitoring schedule, waiting for the first execution, and then stopping/deleting it) would work as a workaround if you need to run a one-time monitoring job.

For more detailed monitoring of your jobs, you can use the monitoring capabilities in SageMaker, which include tracking job status, resource utilization, logs, and metrics.
Sources
View scheduled monitors - Amazon SageMaker AI
CreateMonitoringSchedule - Amazon SageMaker
Monitoring jobs and job runs - Amazon MaxDome

answered 8 months ago

0

Hello,

Please allow me to inform you that the run_monitoring_job method does not exist in the DefaultModelMonitor class in SageMaker SDK version 2.254.1. This is by design—SageMaker Model Monitor uses monitoring schedules rather than direct job execution methods.

To run an on-demand data quality monitoring job, use the create_monitoring_schedule() method with CronExpressionGenerator.now() as the schedule expression. This triggers immediate, one-time execution without creating a recurring schedule.

CreateMonitoringSchedule - https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateMonitoringSchedule.html

Schedule monitoring jobs - https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-scheduling.html

Model Monitor - https://sagemaker.readthedocs.io/en/stable/api/inference/model_monitor.html

Model Monitor FAQs - https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-faqs.html

Additionally, I kindly request you to please stay updated through these blogs -

What's New with AWS - https://aws.amazon.com/new/

AWS News Blog - https://aws.amazon.com/blogs/aws/

If you have any questions or would like more details about the feature or roadmap, please reach out to AWS Support, and we would be happy to assist you further.

—References—

[1] Creating support cases and case management: https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-casehttps://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-case

AWS

answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.