What does datatype_check_threshold mean in SageMaker model monitoring?

Question

According to the [docs](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-byoc-constraints.html):

> If the threshold is above the value of the specified datatype_check_threshold, this causes a failure that is treated as a violation in the violation report. If the data types in the current execution are not the same as in the baseline dataset, this threshold is used to evaluate if it needs to be flagged as a violation.... Default value is 0.1

So if datatype_check_threshold=0.1 and there is more than 10% data type is matching to requirement (corret data type), then there is no violation? Is that correct?

Thank you

sqavi · Answer

datatype_check_threshold set's the tolerance level of data type validation. So if value is set datatype_check_threshold=0.1, it means it will not be triggering a violation if 10% or less data is not matching the requirement. If the percentage of input data that does not match the expected data exceed threshold i.e. 10% in our case, an error is raised indicating a data type mismatch.
https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-byoc-constraints.html

What does datatype_check_threshold mean in SageMaker model monitoring?

Add your answer

Relevant content