Why is my Lambda function retrying valid Amazon SQS messages and placing them in my dead-letter queue?

Short description

If your Lambda function is throttled, returns an error, or doesn't respond when reading an Amazon SQS message batch, then the messages return to your queue. After the visibility timeout occurs, your Lambda function receives the message batch again. If your function fails to process valid messages multiple times, then Amazon SQS sends the messages to your dead-letter queue, if you've configured one.

To prevent valid messages from being placed in a dead-letter queue, your function code must be idempotent and capable of handling messages multiple times. For more information, see How can I prevent an Amazon SQS message from invoking my Lambda function more than once?

Resolution

Verify that your Lambda function's code is idempotent

For idempotency best practices and example function logic, see How do I make my Lambda function idempotent?

Verify that your Amazon SQS queue's visibility timeout is at least six times longer than your Lambda function's timeout setting

Set your source queue's visibility timeout to at least six times longer than your function's timeout. The extra time allows your function to retry processing a batch if the function is throttled while processing a previous batch.

For more information, see Setting the visibility timeout in the Amazon SQS Developer Guide.

Note: If your function doesn't receive messages because the queue's visibility timeout isn't long enough, then the messages won't be recorded in your Amazon CloudWatch Logs.

Verify that the maxReceiveCount attribute is set to at least five on your source queue's redrive policy

Set the maxReceiveCount on the source queue's redrive policy to at least five. If your function returns an error, or can't be invoked because it's at maximum concurrency, processing might succeed with additional attempts. A maxReceiveCount of at least five gives your messages more chances to be processed before they're sent to your dead-letter queue.

For more information, see How do dead-letter queues work? and Avoid inconsistent message processing in the Amazon SQS Developer Guide.

Identify and resolve any errors that your Lambda function returns

Follow the instructions in How do I troubleshoot Lambda function failures? Your function automatically removes messages from your queue only if the function doesn't return an error.