How to handle failed lambda functions
I have a S3 bucket with files. A lambda function is called after an upload event is coming from my s3 bucket. Within my lambda (python based) function, I upload this file to my personal server outside of aws via a rest call.
Problem: My rest endpoint is not always online. (For example system restart, updated etc or any other problems)· But I don't want to loose any files. Is there a nice aws way to retry to upload infinitely? (For example every hour?)
Thank you :)
You have different options. The easiest I think would be to send the S3 events to an SQS queue with a VisibilityTimeout of 1 hour and a retention of 14 days. You will trigger the function from the queue. The function will try to upload the file to the API. If it failed, it will return a value indicating that the message processing failed. That message will become visible again in an hour and will be retried. This will happen up to 14 days, the maximum retention time for the queue.
If 14 days is not enough, you could also set up a DLQ with many retries. If it failed all those retries the message will be sent to the DLQ. You will then be able to re-drive them back to the original queue for another round.
There are other options for instance using StepFunctions. You will invoke a new state machine for each file. The state machine will call a Lambda function to upload the file. If upload is OK, state machine ends. If not, it gets into a loop with a Wait state. It can run for up to a year.
You could do it in a two step process: use the S3 event/lambda to create a record in a database (e.g. dynamodb) and then have another lambda that checks the database for files that need to be uploaded and does the upload. You can schedule that lambda to run every hour from a cloudwatch/eventbridge event.
Another thing to consider is using a deadletter queue on your current lambda and then replay the events on the deadletter queue via a scheduled lambda.
A lambda function to delete old archive files in s3 bucketasked 3 years ago
Lambda Container-Based Function Needs --privileged and --device /dev/fuseAccepted Answerasked 12 days ago
Invoke Lambda functions from S3 uploads at high scaleAccepted Answerasked 2 years ago
Pass Cloudwatch Event Data to Lambda Functionasked 21 days ago
Bucket causes error for Lambda Trigger - Invalid Argumentasked 6 months ago
403 creating java lambda functionasked 5 months ago
How to handle failed lambda functionsasked 3 months ago
S3 object and lambda function in step machine with the asynchronous express workflowsAccepted Answerasked 5 months ago
Why is my S3 Lambda Trigger not executed for larger objects uploaded to S3 with KMS keyasked 5 months ago
Can we get output from a Lambda function as CloudFormation Parameter?asked 2 years ago