Saltar al contenido

Some Messages Are Being Polled , Others Are Missed in AWS SQS

0

Hi team ,

I'm using an SQS Standard Queue, and I'm running into an issue where not all messages are returned when I poll manually from the AWS console.

Here’s what’s happening: 1.On aws console I clicked on Poll for messages 2. I send messages to the queue when the polling is in progress .

some messages are retrieved, while others messages are never getting polled . same behaviour is observed in my springboot application for an SQSListener . the delivery delay is set to 0 seconds .

I wonder if someone could provide further guidance on this issue ? Thank you very much.

preguntada hace 3 meses146 visualizaciones
2 Respuestas
0

Hello.

How many messages are in the SQS queue?
By default, ReceiveMessage retrieves 10 messages.
https://docs.aws.amazon.com/ja_jp/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html

Retrieves one or more messages (up to 10), from the specified queue. Using the WaitTimeSeconds parameter enables long-poll support. For more information, see Amazon SQS Long Polling in the Amazon SQS Developer Guide.

In the management console, you can adjust the "Maximum message count" as shown in the image below.
a

Additionally, as described in the AI automated response of re:Post Agent, using long polling can reduce the chance of missed messages.
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html

EXPERTO
respondido hace 3 meses
0

This behavior is expected with Amazon SQS, especially when using short polling (which is the default method). When you poll for messages using short polling, Amazon SQS samples only a subset of its servers based on a weighted random distribution and returns messages from just those servers. This means a particular ReceiveMessage request might not return all your messages.

For standard queues with fewer than 1,000 messages, subsequent polling requests will eventually return all your messages. If you keep consuming from your queues, Amazon SQS will eventually sample all of its servers, and you'll receive all your messages.

To improve message retrieval, consider these options:

  1. Use long polling instead of short polling by setting the WaitTimeSeconds parameter to a value greater than 0 (up to 20 seconds). Long polling queries all servers for messages and waits until at least one message is available before sending a response, which eliminates the "missed messages" issue.

  2. For your SpringBoot application's SQSListener, ensure it's configured to use long polling.

  3. Check your visibility timeout settings. If messages are being processed but not deleted, they'll become invisible for the duration of the visibility timeout and won't appear in subsequent polls until that timeout expires.

  4. If you're using a FIFO queue (though you mentioned standard queue), be aware that message group ID can affect message retrieval patterns.

The behavior you're experiencing is part of SQS's distributed architecture design and not an error. Long polling is generally recommended as it reduces empty responses and ensures more consistent message retrieval.
Sources
Amazon SQS short and long polling - Amazon Simple Queue Service
Troubleshoot messages not returned for an Amazon SQS ReceiveMessage API call - Amazon Simple Queue Service
Unable to fetch data from SQS queue. | AWS re:Post

respondido hace 3 meses
AWS
EXPERTO
revisado hace 3 meses
EXPERTO
revisado hace 3 meses

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.