By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Invoke Lambda functions from S3 uploads at high scale

0

Customers who wants to upload files to S3, trigger Lambda functions, and write the file name in a database. This scenario can cause a 10,000 uploads per second.

There is no 100% guarantee that for every file uploaded, a Lambda function will be invoked.

What would be a good way to deal with the peaks and write the file names to a database?

Topics
StorageServerlessCompute
Tags
Amazon Simple Storage ServiceAWS Lambda
Language
English
Nir_Sh
asked 4 years ago395 views
1 Answer
0
Accepted Answer

Recently S3 changed their notification mechanism and it is now at least once delivery of events. If you want to make sure that all events are handled, send them to SQS and trigger a Lambda from the queue. in addition to notifications you can compare the S3 Inventory to what's in the database to catch missed events.

profile pictureAWS
EXPERT
Uri
answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content