I am trying to set up a data quality job for a batch transfer model. All inputs look right to me but the job does not kick off and displays above error when job is described.
my_default_monitor.create_monitoring_schedule(
monitor_schedule_name=mon_schedule_name,
batch_transform_input=BatchTransformInput(
data_captured_destination_s3_uri=s3_data_capture_path,
destination="/opt/ml/processing/input",
dataset_format=MonitoringDatasetFormat.csv(header=False)
),
output_s3_uri=s3_reports_path,
statistics= statistics_path,
constraints = constraints_path,
schedule_cron_expression= CronExpressionGenerator.daily(hour=19),
enable_cloudwatch_metrics=True
)
the file in 's3_data_capture_path' is a .csv file. It is created every month format it being same as training input file for model monitoring baseline job.
Trouble shooting, I did so far as below ... still same error. The error message is not at all helpful!
- flip s3_data_capture_path to relative path to path to actual file copied from aws console.
- try both daily cron job and hourly cron job
Questions
Other than SMS notebook, can I trace more log details elsewhere? It does not seem to kick off and seems to fail in validation...
Is it not actually finding the file (which is unlikely as I verified this in different ways) or is it looking for a specific format?
Could it be possible that the daily run or hourly looks for values in the last hour or last day?
Please suggest!