S3 static loop issue

0

Hi AWS,

I was going through a question from AWS CQ session i.e. An AWS Lambda function has been configured to create a thumbnail image whenever an object is created in a particular S3 bucket. However, it is getting stuck in an infinite loop. How this can be avoided ?.

I was assuming it happened because of the throttling, however the answer is Configure a Prefix on the Amazon S3 Event. Can you please give some more insights and share any link which covers this.

profile picture
Arjun
preguntada hace un año271 visualizaciones
3 Respuestas
1

When placing the image back into the same bucket it came from, it needs to be differentiated. Therefore placing the thumbnail into a different folder.

Only monitor file creation in the folder the image is uploaded to. The lambda function needs to store the images it creates in a different folder (prefix)

See prefix on when creating event notifications

On the S3 bucket, Goto properties -> Event Notifications -> Create event notification Configure the prefix here (folder name)

https://docs.aws.amazon.com/AmazonS3/latest/userguide/notification-how-to-filtering.html

This means the lambda function will only trigger on the image upload and not the thumbnails created by lambda.

profile picture
EXPERTO
respondido hace un año
0

Hi, this is happening because if you don't take any precaution in the structure of your bucket and the way it is managed by the lamdbda, an initial image will trigger the lambda to generate a thumbnail written back to the bucket. The write of the thumbnail will again trigger the lambda to generate a thumbnail of the thumbnail ad infinitum.

So, you either need to have a folder in your bucket or some naming convention for the thumbnail allowing you to distinguish within the lambda if it was triggered by an initial image or a thumbnail. If lambda starts on a thumbail, it should just return without doing anything. That will stop recursivity.

profile pictureAWS
EXPERTO
respondido hace un año
  • This method triggers on false positives which is costly of compute resource and money. Prefix on event notifications on the bucket means the lambda function will only trigger on the correct images.

0

In short, avoid triggering the lambda from an s3 bucket, which is the same bucket accessed by the lambda to PUT the generated thumbnail, causing infinite loop (and costs).

profile picture
EXPERTO
respondido hace un año

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas