Auto rollback with Guardrails if model accuracy is not good

1

While deploying model with guardrails for SageMaker Inference Endpoint::

We can create CloudWatch alarms to monitor Endpoint performance for metrics like Invocation5XXErrors, ModelLatency and we can rollback the model to previous version if the threshold is matched. [Different metrics available][https://docs.aws.amazon.com/sagemaker/latest/dg/monitoring-cloudwatch.html]

Please let me know is there any way to achieve any one of the following requirement:

  1. I want to the use metrics like "accuracy" returned by endpoint and rollback to previous version if the model performance is consistently not good in the real time.
  2. Use the metrics like "accuracy" returned by version2 and compare it with version1 or with the ground truth and rollback model if necessary
已提問 2 年前檢視次數 346 次
1 個回答
0

You should take a look at SageMaker Model Monitor capabilities and its integration with SageMaker Pipelines to achieve this.

In this method, you will continuously "monitor the model" for model drift and take downstream actions when you detect the drift. The action could be to send a notification(SNS) or re-trigger the Model training.

Some examples for using SageMaker Model Monitor can be found here

AWS
已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南