I want to use the model monitor in the sagemaker pipeline to get the results of statistics.json and constraints.json

Hello,

When you say "you do not know what CSV content I should send to get the 'Regression' results", are you referring to the ContentType for your dataset?

Firstly, the line below means that the features/column names in the training dataset are not provided as the first row:

dataset_format=DatasetFormat.csv(header=False)

Please see the link [1] for more information on the above parameter. Basically, your dataset content is a comma-separated value file but for this particular scenario header = False as there is no column names provided for the training dataset.

I believe your overall question has to do with MetricsSource [2] object that would be defined as part of the ModelMetrics module, as in what will be the ContentType value used for your use case.

When it comes to MetricsSource object, if you consider the example "SageMaker Pipelines integration with Model Monitor and Clarify" from https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-pipelines/tabular/model-monitor-clarify-pipelines/sagemaker-pipeline-model-monitor-clarify-steps.ipynb, you will see that there is code that looks like the following:

    model_statistics=MetricsSource(
        s3_uri=model_quality_check_step.properties.CalculatedBaselineStatistics,
        content_type="application/json",
    ),
    model_constraints=MetricsSource(
        s3_uri=model_quality_check_step.properties.CalculatedBaselineConstraints,
        content_type="application/json",
    ),

As you can see from the above that content_type is "application/json"

The data you shared below suggest that the Pipeline has been deployed as an endpoint [3], as there is a deploy module that is supported.

But from running the example "SageMaker Pipelines integration with Model Monitor and Clarify", I was able to get the files

s3://S3-BUCKET-NAME-HERE/some-prefix/modelqualitycheckstep/statistics.json 
s3://S3-BUCKET-NAME-HERE/some-prefix/modelqualitycheckstep/constraints.json

where statistics.json contains the following:

{
  "version": 0,
  "dataset": {
    "item_count": 627,
    "evaluation_time": "2022-10-23T10:49:40.638Z"
  },
  "regression_metrics": {
    "mae": {
      "value": 1.4107242246563925,
      "standard_deviation": 0.025615074935394368
    },
    "mse": {
      "value": 3.9022604063585753,
      "standard_deviation": 0.23140761659194883
    },
    "rmse": {
      "value": 1.9754139835382798,
      "standard_deviation": 0.05901487899216817
    },
    "r2": {
      "value": 0.40614751710172436,
      "standard_deviation": 0.03121704707239033
    }
  }
}

From the above, one can see that if the MetricsSource Object is declared then the metrics published for a regression problem type.

Hope I answered the question properly, if not please reach out to AWS Support[4] (SageMaker), explain your issue/use case in detail and share relevant AWS resource names (plus CloudWatch logs).

References:

[1] https://sagemaker.readthedocs.io/en/stable/api/inference/model_monitor.html#sagemaker.model_monitor.dataset_format.DatasetFormat

[2] https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_MetricsSource.html

[3] https://sagemaker.readthedocs.io/en/stable/api/inference/pipeline.html

[4] https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-case