SageMaker Model Monitor (ModelQuality) says “Job inputs had no data” even though DataCapture and Ground Truth exist (IDs match)

Question

I’m setting up a ModelQuality monitoring schedule for a binary classification endpoint. The schedule runs, but the report says:

Job inputs had no data

I can confirm both DataCapture and Ground Truth data exist in S3 for the analysis window, and they refer to the same invocations.

What exists in S3

Data capture (under the usual SageMaker DataCapture layout): s3://mlops-bikeshare-387706002632-ca-central-1/datacapture/endpoint=bikeshare-staging-config-20250923-065304/bikeshare-staging/AllTraffic/2025/09/23/18/02-45-448-7c294b52-b6d7-486c-aadf-38286a00ffe5.jsonl Example line (redacted):


{
  "captureData": {
    "endpointInput": {
      "observedContentType": "application/json",
      "mode": "INPUT",
      "data": "{\"inputs\": {\"dataframe_split\": {\"columns\": [...], \"data\": [[...]]}}}",
      "encoding": "JSON"
    },
    "endpointOutput": {
      "observedContentType": "application/json",
      "mode": "OUTPUT",
      "data": "{\"predictions\":[0.17401574552059174]}\n",
      "encoding": "JSON"
    }
  },
  "eventMetadata": {
    "eventId": "52907414-e646-4d3e-bd9e-33d957fd0244",
    "inferenceId": "2025-09-23-17-35_ca4e4962-2169-4559-8f60-b05ab7809750",
    "inferenceTime": "2025-09-23T18:02:45Z"
  },
  "eventVersion": "0"
}

Ground truth (hourly JSONL produced from my parquet joins): s3://mlops-bikeshare-387706002632-ca-central-1/monitoring/ground-truth/2025/09/23/17/labels-2025092317.jsonl Example line (redacted):

{
  "groundTruthData": { "data": "0", "encoding": "CSV" },
  "groundTruthMetadata": {
    "eventId": "d8f02eeb-73ca-46d1-b929-1293c2599393",
    "inferenceId": "2025-09-23-17-25_ca4e4962-2169-4559-8f60-b05ab7809750",
    "inferenceTime": "2025-09-23T17:49:15Z"
  },
  "eventVersion": "0"
}

(Each ground-truth line is built by copying identifiers from DataCapture; every label line can be matched back to a DataCapture line.)

How I create the schedule:

sm.create_model_quality_job_definition(
    JobDefinitionName="bikeshare-model-quality-jd",
    ModelQualityAppSpecification={"ImageUri": image_uri, "ProblemType": "BinaryClassification"},
    ModelQualityJobInput={
        "EndpointInput": {
            "EndpointName": "bikeshare-staging",
            "LocalPath": "/opt/ml/processing/input_data",
             "S3InputMode": "File",
             "ProbabilityAttribute": "predictions",
            "ProbabilityThresholdAttribute": 0.15,
            "StartTimeOffset": "-PT5H",
            "EndTimeOffset": "-PT1H",
        },
        "GroundTruthS3Input": {"S3Uri": f"s3://{bucket}/monitoring/ground-truth"},
    },
    ModelQualityJobOutputConfig={
        "MonitoringOutputs": [
            {
                "S3Output": {
                    "S3Uri": reports_prefix,
                    "LocalPath": "/opt/ml/processing/output",
                    "S3UploadMode": "EndOfJob",
                }
            }
        ]
    },
    JobResources={"ClusterConfig": {"InstanceCount": 1, "InstanceType": "ml.m5.large", "VolumeSizeInGB": 30}},
    NetworkConfig={"EnableNetworkIsolation": False},
    RoleArn=role_arn,
    StoppingCondition={"MaxRuntimeInSeconds": 3300},
)

sm.create_monitoring_schedule(
    MonitoringScheduleName="bikeshare-model-quality",
    MonitoringScheduleConfig={
        "ScheduleConfig": {
            "ScheduleExpression": "NOW",
            "DataAnalysisStartTime": "-PT5H",
            "DataAnalysisEndTime": "-PT1H",
        },  # cron(0 0/2 ? * * *)
        "MonitoringJobDefinitionName": "bikeshare-model-quality-jd",
        "MonitoringType": "ModelQuality",
    },
)

What I already tried for the ground-truth JSON schema I tested multiple schemas (all fail with the same “no data” result):

Canonical (my best guess):

{
  "groundTruthData": {"data":"0","encoding":"CSV"},
  "eventMetadata":   {"eventId":"<id-from-capture>","inferenceId":"<...>","inferenceTime":"<...>"},
  "eventVersion":"0"
}

Using groundTruthMetadata instead of eventMetadata (body includes either only eventId, or only inferenceId, or both).

Super-minimal variants like:

{"groundTruthData":"0","inferenceId":"<...>"}

…and similar small changes to field names/casing. 3. Adjusted the analysis time window out to 12 hours to avoid any boundary issues. Still “Job inputs had no data”.

Any guidance on why the analyzer reports “Job inputs had no data” and what the strict ground-truth JSON contract is would be greatly appreciated. I can provide ProcessingJob ARNs and exact timestamps if needed.

re:Post Agent · Answer

The issue appears to be with the structure of your Ground Truth data. Based on the examples provided, there's a mismatch between your Ground Truth JSON structure and what SageMaker Model Monitor expects.

In your Ground Truth data, you're using `groundTruthMetadata` as the key for the metadata section, but SageMaker Model Monitor expects `eventMetadata` instead. The correct structure for Ground Truth data should be:

```json
{
"groundTruthData": { "data": "0", "encoding": "CSV" },
"eventMetadata": {
"eventId": "d8f02eeb-73ca-46d1-b929-1293c2599393",
"inferenceId": "2025-09-23-17-25_ca4e4962-2169-4559-8f60-b05ab7809750",
"inferenceTime": "2025-09-23T17:49:15Z"
},
"eventVersion": "0"
}
```

When ingesting Ground Truth labels and merging them with predictions, SageMaker Model Monitor uses either the `inferenceId` or `eventId` to match the Ground Truth records with the captured prediction data. The `inferenceId` is the preferred identifier, and it must match exactly between your captured data and Ground Truth data.

Additionally, ensure that:
1. The date in the Ground Truth S3 path (which represents when the Ground Truth label was collected) falls within the analysis window specified in your monitoring job.
2. The `inferenceId` values in your Ground Truth data match exactly with those in your captured prediction data.
3. The format of your Ground Truth data follows the expected jsonlines format.

If you've already tried the correct structure with `eventMetadata` instead of `groundTruthMetadata`, then check if there might be other issues such as:
- Time zone discrepancies between inference time and analysis window
- S3 permissions issues preventing the monitoring job from accessing the data
- Mismatched `inferenceId` values between captured data and Ground Truth data  
**Sources**  
[Governing the ML lifecycle at scale: Centralized observability with Amazon SageMaker and Amazon CloudWatch | AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/governing-the-ml-lifecycle-at-scale-centralized-observability-with-amazon-sagemaker-and-amazon-cloudwatch/)  
[Ingest Ground Truth labels and merge them with predictions - Amazon SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality-merge.html)  
[Model quality - Amazon SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality.html)

SageMaker Model Monitor (ModelQuality) says “Job inputs had no data” even though DataCapture and Ground Truth exist (IDs match)

Add your answer

Relevant content