Skip to content

Error: No captured data found despite enabling data capture

0

Hi, I am using a batch transform job on AWS SageMaker and trying to create a model quality monitoring schedule - however I get the error No capture data found. I have checked the data capture location where the manifest jsonl file sits and have checked the prefix s3 path and the dummy.csv.out file is there. So can anyone help with with I might be missing? Thanks

asked 2 years ago221 views

1 Answer
1

It sounds like you've encountered an issue with data capture for your batch transform job in Amazon SageMaker. Let's go through a few potential reasons for this error and some steps you can take to resolve it:

  1. Data Capture Configuration: Ensure that you've properly enabled data capture for your batch transform job. When creating the transform job, you need to specify a BatchDataCaptureConfig with the correct destination S3 URI.

  2. Inference ID Generation: For model quality monitoring, it's crucial to set the "generate_inference_id" parameter to True in your BatchDataCaptureConfig. This appends an inference ID and transform job start time to each record, which is necessary for merging with Ground Truth data.

  3. S3 Permissions: Verify that SageMaker has the necessary permissions to access the S3 bucket where your captured data is stored.

  4. Data Format: Check that the captured data is in the correct format. The transform job should create separate /input and /output directories under the specified S3 URI, each containing manifests of the source objects.

  5. Timing: There can be a delay in the propagation of captured data to S3. Ensure you've allowed enough time for the data to be fully captured and stored before attempting to create the monitoring schedule.

  6. Ground Truth Data: For model quality monitoring, you need to ingest Ground Truth labels and merge them with the captured prediction data. Make sure you've properly ingested and merged this data.

  7. Baseline Creation: Verify that you've created a proper baseline for your model quality monitoring. This baseline should compare predictions from your model with Ground Truth labels in a baseline dataset.

If you've checked all these points and are still encountering issues, you may want to review your monitoring job configuration and ensure all parameters are set correctly. It might also be helpful to check CloudWatch logs for any additional error messages or insights into why the captured data isn't being found.
Sources
Capture data from batch transform job - Amazon SageMaker AI
Data capture - Amazon SageMaker
Model quality - Amazon SageMaker
Model Monitor FAQs - Amazon SageMaker AI

answered 2 years ago

EXPERT

reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.