跳至內容

Some Messages Are Being Polled , Others Are Missed in AWS SQS

0

Hi team ,

I'm using an SQS Standard Queue, and I'm running into an issue where not all messages are returned when I poll manually from the AWS console.

Here’s what’s happening: 1.On aws console I clicked on Poll for messages 2. I send messages to the queue when the polling is in progress .

some messages are retrieved, while others messages are never getting polled . same behaviour is observed in my springboot application for an SQSListener . the delivery delay is set to 0 seconds .

I wonder if someone could provide further guidance on this issue ? Thank you very much.

已提問 3 個月前檢視次數 134 次
2 個答案
0

Hello.

How many messages are in the SQS queue?
By default, ReceiveMessage retrieves 10 messages.
https://docs.aws.amazon.com/ja_jp/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html

Retrieves one or more messages (up to 10), from the specified queue. Using the WaitTimeSeconds parameter enables long-poll support. For more information, see Amazon SQS Long Polling in the Amazon SQS Developer Guide.

In the management console, you can adjust the "Maximum message count" as shown in the image below.
a

Additionally, as described in the AI automated response of re:Post Agent, using long polling can reduce the chance of missed messages.
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html

專家
已回答 3 個月前
0

This behavior is expected with Amazon SQS, especially when using short polling (which is the default method). When you poll for messages using short polling, Amazon SQS samples only a subset of its servers based on a weighted random distribution and returns messages from just those servers. This means a particular ReceiveMessage request might not return all your messages.

For standard queues with fewer than 1,000 messages, subsequent polling requests will eventually return all your messages. If you keep consuming from your queues, Amazon SQS will eventually sample all of its servers, and you'll receive all your messages.

To improve message retrieval, consider these options:

  1. Use long polling instead of short polling by setting the WaitTimeSeconds parameter to a value greater than 0 (up to 20 seconds). Long polling queries all servers for messages and waits until at least one message is available before sending a response, which eliminates the "missed messages" issue.

  2. For your SpringBoot application's SQSListener, ensure it's configured to use long polling.

  3. Check your visibility timeout settings. If messages are being processed but not deleted, they'll become invisible for the duration of the visibility timeout and won't appear in subsequent polls until that timeout expires.

  4. If you're using a FIFO queue (though you mentioned standard queue), be aware that message group ID can affect message retrieval patterns.

The behavior you're experiencing is part of SQS's distributed architecture design and not an error. Long polling is generally recommended as it reduces empty responses and ensures more consistent message retrieval.
Sources
Amazon SQS short and long polling - Amazon Simple Queue Service
Troubleshoot messages not returned for an Amazon SQS ReceiveMessage API call - Amazon Simple Queue Service
Unable to fetch data from SQS queue. | AWS re:Post

已回答 3 個月前
AWS
專家
已審閱 3 個月前
專家
已審閱 3 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。