Remove a label from Custom labels in Rekognition - different results?

Question

I have a training and test data set, each with 10 or so labels applied. I created a model on that data. Let's call it Model 1.

Then, I created a new project and used the same data set, but I removed a label by removing it in all images in which it appeared, in both the test and training data. I created a second model, Model 2.

For the labels that remain in both models, however, I get different precision and recall values for labels that I did not change in any way. Does this make sense?

Answer

Hello,

Yes, this is expected behavior due to multiple factors. One major contributor could be changes in the threshold value. The Custom Labels feature implements custom logic to compute optimal thresholds for each training iteration, hence variations are reflected in the Precision and Recall calculations. Further details can be found in the summary file, which contains the threshold values computed per training run.

Reference can be found [here](https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/im-summary-file-api.html)

Remove a label from Custom labels in Rekognition - different results?

Relevant content