- Newest
- Most votes
- Most comments
It sounds like you've encountered an issue with data capture for your batch transform job in Amazon SageMaker. Let's go through a few potential reasons for this error and some steps you can take to resolve it:
-
Data Capture Configuration: Ensure that you've properly enabled data capture for your batch transform job. When creating the transform job, you need to specify a BatchDataCaptureConfig with the correct destination S3 URI.
-
Inference ID Generation: For model quality monitoring, it's crucial to set the "generate_inference_id" parameter to True in your BatchDataCaptureConfig. This appends an inference ID and transform job start time to each record, which is necessary for merging with Ground Truth data.
-
S3 Permissions: Verify that SageMaker has the necessary permissions to access the S3 bucket where your captured data is stored.
-
Data Format: Check that the captured data is in the correct format. The transform job should create separate /input and /output directories under the specified S3 URI, each containing manifests of the source objects.
-
Timing: There can be a delay in the propagation of captured data to S3. Ensure you've allowed enough time for the data to be fully captured and stored before attempting to create the monitoring schedule.
-
Ground Truth Data: For model quality monitoring, you need to ingest Ground Truth labels and merge them with the captured prediction data. Make sure you've properly ingested and merged this data.
-
Baseline Creation: Verify that you've created a proper baseline for your model quality monitoring. This baseline should compare predictions from your model with Ground Truth labels in a baseline dataset.
If you've checked all these points and are still encountering issues, you may want to review your monitoring job configuration and ensure all parameters are set correctly. It might also be helpful to check CloudWatch logs for any additional error messages or insights into why the captured data isn't being found.
Sources
Capture data from batch transform job - Amazon SageMaker AI
Data capture - Amazon SageMaker
Model quality - Amazon SageMaker
Model Monitor FAQs - Amazon SageMaker AI
Relevant content
asked 3 years ago
