By using AWS re:Post, you agree to the Terms of Use

Invoke Lambda functions from S3 uploads at high scale

0

Customers who wants to upload files to S3, trigger Lambda functions, and write the file name in a database. This scenario can cause a 10,000 uploads per second.

There is no 100% guarantee that for every file uploaded, a Lambda function will be invoked.

What would be a good way to deal with the peaks and write the file names to a database?

1 Answer
0
Accepted Answer

Recently S3 changed their notification mechanism and it is now at least once delivery of events. If you want to make sure that all events are handled, send them to SQS and trigger a Lambda from the queue. in addition to notifications you can compare the S3 Inventory to what's in the database to catch missed events.

profile picture
EXPERT
answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions