By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

waittime for sqs polling

0

HI team,

I am running a node application on ECS Fargate, which involves consuming messages from an SQS queue through long polling. The application uses a batch size of 100 for parallel processing across multiple consumers.

If:

  • The batch size is set to 100
  • Long polling wait time is configured at 2 seconds
  • There are only 2 messages present in the queue

I'm seeking guidance on configuring SQS to ensure that these 2 available messages are consumed immediately, without the application waiting for the 2-second polling interval.

I want the application to consume by batch of 100, and if there are no 100 messages available the application should not wait 2 seconds to consume available messages.

Thank you!

Topics
ServerlessContainersApplication IntegrationMicroservices
Tags
ServerlessAWS FargateAmazon Simple Queue ServiceMicroservices
Language
English
JessDL
asked 8 months ago520 views
1 Answer
0

When reading from SQS you can't define a window size and the maximum buffer size that you can configure is 10 messages. Just use LongPolling with 10 messages. When the messages will become available. it will return the messages. It will not wait for 20 seconds.

The only caveat is that if there is a very small number of messages in the queue, due to the distributed nature of SQS, you may not get the messages immediately, regardless of the batch size. When you have many messages, it will not happen.

profile pictureAWS
EXPERT
Uri
answered 8 months ago
  • JessDL
    8 months ago

    thank you for your answer, if i have a batch size with 10 messages and let's say in the queue there are only 4 messages will consume those 4 messages immediately? or long polling will wait till it gets a total of 10 messages or the wait time for long polling been elapsed

  • Uri EXPERT
    8 months ago

    The batch size is the maximum messages to retrieve, not the number of messages to wait for. This means that if the there are less messages in the queue, you will get them. However, as I wrote before, in some edge cases, when there is low traffic in the queue and small number of messages, you may wait for 20 seconds and get an empty batch, even though there are messages in the queue. If you have more than one consumer, the chance of this happening, reduces.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content