Facial Landmarks for partially covered face in Rekognition

Question

Hi,

we're working out how to distinguish faces that are just a little bit covered to faces that are mostly covered. The "face occluded" prop does not say anything about how much covered the face actually is.

We thought of using the facial landmarks for doing this, checking if one of the eyes/nose etc is covered. But when I tested with a face where more than half of the face is covered, then we still got coordinates for the facial landmarks that you cannot see in the picture.

Why is this? How does rekognition find these landmarks? And if anyone has any suggestions on how to solve this issue, that would be appreciated.

Answer

Hello,

That is correct, by using facial landmarks you can identify specific points on the face such as the corners of the eyes, the tip of the nose, and the corners of the mouth. However, Rekognitions ability to detect facial landmarks may be limited when the face is partially or fully covered. This is due to the algorithm not have enough visual data to accurately identify the landmarks.

Using Rekognitions detect faces for facial landmarks to distinguish between faces that are partially covered and faces that are mostly covered will not be sufficient alone.

As far as the next steps, I would try using Rekognitions custom labels. This will allow you to make a custom model to your specific use case while still using Rekognitions existing capabilities. By uploading a few images for training, you will be able to create your custom image analysis model for detecting partially covered faces.

What are custom labels?
https://aws.amazon.com/rekognition/custom-labels-features/

How to get started? 
https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/getting-started.html

Facial Landmarks for partially covered face in Rekognition

Relevant content