내용으로 건너뛰기

Some Messages Are Being Polled , Others Are Missed in AWS SQS

0

Hi team ,

I'm using an SQS Standard Queue, and I'm running into an issue where not all messages are returned when I poll manually from the AWS console.

Here’s what’s happening: 1.On aws console I clicked on Poll for messages 2. I send messages to the queue when the polling is in progress .

some messages are retrieved, while others messages are never getting polled . same behaviour is observed in my springboot application for an SQSListener . the delivery delay is set to 0 seconds .

I wonder if someone could provide further guidance on this issue ? Thank you very much.

질문됨 3달 전140회 조회
2개 답변
0

Hello.

How many messages are in the SQS queue?
By default, ReceiveMessage retrieves 10 messages.
https://docs.aws.amazon.com/ja_jp/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html

Retrieves one or more messages (up to 10), from the specified queue. Using the WaitTimeSeconds parameter enables long-poll support. For more information, see Amazon SQS Long Polling in the Amazon SQS Developer Guide.

In the management console, you can adjust the "Maximum message count" as shown in the image below.
a

Additionally, as described in the AI automated response of re:Post Agent, using long polling can reduce the chance of missed messages.
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html

전문가
답변함 3달 전
0

This behavior is expected with Amazon SQS, especially when using short polling (which is the default method). When you poll for messages using short polling, Amazon SQS samples only a subset of its servers based on a weighted random distribution and returns messages from just those servers. This means a particular ReceiveMessage request might not return all your messages.

For standard queues with fewer than 1,000 messages, subsequent polling requests will eventually return all your messages. If you keep consuming from your queues, Amazon SQS will eventually sample all of its servers, and you'll receive all your messages.

To improve message retrieval, consider these options:

  1. Use long polling instead of short polling by setting the WaitTimeSeconds parameter to a value greater than 0 (up to 20 seconds). Long polling queries all servers for messages and waits until at least one message is available before sending a response, which eliminates the "missed messages" issue.

  2. For your SpringBoot application's SQSListener, ensure it's configured to use long polling.

  3. Check your visibility timeout settings. If messages are being processed but not deleted, they'll become invisible for the duration of the visibility timeout and won't appear in subsequent polls until that timeout expires.

  4. If you're using a FIFO queue (though you mentioned standard queue), be aware that message group ID can affect message retrieval patterns.

The behavior you're experiencing is part of SQS's distributed architecture design and not an error. Long polling is generally recommended as it reduces empty responses and ensures more consistent message retrieval.
Sources
Amazon SQS short and long polling - Amazon Simple Queue Service
Troubleshoot messages not returned for an Amazon SQS ReceiveMessage API call - Amazon Simple Queue Service
Unable to fetch data from SQS queue. | AWS re:Post

답변함 3달 전
AWS
전문가
검토됨 3달 전
전문가
검토됨 3달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

관련 콘텐츠