Alternate way to - Lambda Linear Processing the Rekognition API calls

Question

**Problem Statement:**
While processing the images from S3->SQS->Lambda->Rekognition APIs, due to lambda concurrency, parallel processing will be executed and I'm getting duplicate data, if a person is present in multiple pictures.

**Example**:
Image_1 -> has person A, B, C and 1st lambda consumer will starts processing the Image_1.

Parallelly,
Image_2 -> also has person A and 2nd concurrent consumer lambda will starts processing the image_2.

As these both are processing in parallel, if this the Person-A doesn't exist in my collection, then these both processing will createUser (for the same person which is a duplicate)

So, to avoid this I've set the Lambda concurrency to ```1```. Which is processing the images linearly one by one. But it is taking huge time for processing high number of images.

**How to increase speed without duplicating the data?**

Rekognition Workflow:

1. Call IndexFaces
2. Call SearchUsers (with FaceId)
3. If existing user, Call AssociateFaces and associate FaceId to the UserId
4. Else, Call CreateUser, and create a user
5. Associate Faces to the User.

Uri · Answer

Save the FaceIDs that you already created in a DynamoDB table. When you need to create a user, try creating it first in DDB using conditional writes. If it succeeded, you will call CreateUser. If you fail, you will try to associate the face with the user. If it fails because the user is not yet created, wait a few seconds, and try again. This way only one instance will create the user.

Alternate way to - Lambda Linear Processing the Rekognition API calls

Relevant content