AWS Rekognition Custom labels - Evaluation
I am viewing the test results of a model trained through AWS Rekognition Custom labels for image classification. When filtering to False Negatives images only I get the below image. I was under the impression that the confidence should be n/a as indicated in your documentation? It is quite confusing how it gives the true label under Labels and give it a confidence score but it is a False negative. Could you please explain why is that? Please see image below for reference. Thank you
- 최신
- 최다 투표
- 가장 많은 댓글
The "No predictions for this image" outcome during training with Amazon Rekognition Custom Labels means that the model was not able to detect any of the labels present in the image with a confidence above the assumed threshold.
This can occur for a few reasons:
- The image may not contain clear examples of the labels you are trying to train. Images without obvious examples of labels can be difficult for the model to learn from.
- Your training dataset may not be large or varied enough for the model to learn distinguishing features of the labels. Adding more diverse, high-quality images to the training set can help address this.
- The labels in the image may be obscure or difficult examples that the model needs more data to learn. Focusing the training set on clearer, typical examples of labels may improve results.
- The key difference between this and a false negative is that a false negative means the model incorrectly did not detect a label that was present. A "No predictions" result means the model was not confident enough in any prediction to return a label for that image.
Hello, Apologies for any confusion caused. To clarify how the confusion metrics work:
- During training time, we run custom computations to determine the optimal threshold for each class.
- When displaying results in the console, any image marked as positive (as belonging to the class) but with a confidence score below the computed threshold is labeled a false negative.
Specifically for your case, the threshold is higher than 41.1. The SDK always returns a confidence score. Please feel free to use a threshold suitable for your use case.
These computed thresholds can be found in the summary file provided in our documentation: https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/im-summary-file.html
Thank you for your answer - What I am referring to though is the presence of a confidence score with a false negative label detected. In your documentation, it mentions that if an image is labelled as False negative then the confidence score should be n/a and not actually have a confidence score.