Dead letter queues, SQS events and network issues

1

Regarding DLQs, according to https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html: "The maxReceiveCount is the number of times a consumer tries receiving a message from a queue without deleting it before being moved to the dead-letter queue. Setting the maxReceiveCount to a low value such as 1 would result in any failure to receive a message to cause the message to be moved to the dead-letter queue. Such failures include network errors and client dependency errors."

I'm confused by the wording "consumer tries receiving". In the situation where a lambda is receiving messages from a source SQS queue via an event trigger, what is the "consumer" ... the lambda, or the AWS internal service which polls the source queue and sends any found messages to the lambda?

What types of network errors would cause AWS to move a message from the source queue to the DLQ?

Thanks in advance.

已提問 4 個月前檢視次數 173 次
1 個回答
0
已接受的答案

In the context of Amazon SQS and AWS Lambda, the "consumer" refers to the AWS Lambda function that is triggered by the SQS event and processes the messages from the queue.

When a message is received from the SQS queue, the Lambda function attempts to process it. If the processing is successful and the message is deleted from the queue, the process ends. However, if the Lambda function fails to process the message (due to an error in the function, for example), the message remains in the queue and the "ReceiveCount" for that message is incremented, until maxReceiveCount is reached, then going to DLQ.

Network errors that could cause a message to be moved from the source queue to the DLQ include issues with the network connectivity between the AWS services (SQS and Lambda), or within the Lambda function's execution environment. It's important to note that the SQS service itself is designed to be highly available and resilient to network errors, so network errors within SQS itself are rare.

If your lambda process multiple messages at a time, you should implement 'partial batch responses' so that not all messages are put back in the queue if one fails. Look for 'Implementing partial batch responses' in AWS Lambda SQS Doc, you will find some code samples.

profile pictureAWS
已回答 4 個月前
profile picture
專家
已審閱 4 個月前
  • Thank you for your reply. When you say "If the processing is successful and the message is deleted from the queue", is the message deleted manually by the lambda, or automatically by AWS when the lambda returns successfully (ie, does not throw an exception)?

  • When you implement partial batch responses, every message that are NOT in the "batchItemFailures" array are considered as proceed and removed from the queue. If your function throws an exception, the entire batch is considered a complete failure and messages are requeued after the visibility timeout.

  • One other question. Re "if the Lambda function fails to process the message (due to an error in the function, for example), the message remains in the queue and the "ReceiveCount" for that message is incremented, until maxReceiveCount is reached, then going to DLQ."

    What causes the repeated incrementing of the ReceiveCount? Repeated attempts to get the lambda to process the message?

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南