Multiple SQS messages of the same group in one batch

Question

Hello,

I am getting multiple messages from a same SQS FIFO group in a single receive request with MaxNumberOfMessages set to > 2. Is this correct behavior? I don't understand, the API contract says:

*When receiving messages from a FIFO queue with multiple message group IDs, Amazon SQS first attempts to return as many messages with the same message group ID as possible. This allows other consumers to process messages with a different message group ID. When you receive a message with a message group ID, **no more messages for the same message group ID are returned unless you delete the message** or it becomes visible.*

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues-understanding-logic.html

My code dispatches the messages directly to the workers without any group checking logic. It looks like I either need to implement such feature, or simply make all SQS receive requests with maxMessages set to 1 which sounds like a waste of resources. I am confused, because on the very same documentation page that says above, there is this box:

*It is possible to receive **up to 10 messages in a single call** using the MaxNumberOfMessages request parameter of the ReceiveMessage action. These messages retain their FIFO order and **can have the same message group ID**.*

That breaks the above contract which literally says: "unless you delete the message or it becomes visible". In my testing, I see those messages immediately returned in a single receive call, yet they are set to expire in 30 seconds. I don't understand, is there some kind of log that could tell me why a message was delivered that fast? I am probably missing something in here.

Thanks for help

Accepted Answer

This is exactly how it should work. SQS FIFO maintains the order of all messages with the same group ID. To make sure that the order is preserved, if one consumer got a message with a specific group ID, no **other** consumer will be able to get messages with the ID until the message is deleted or becomes visible again. Otherwise, the messages can be processed out of order.

Nothing prevents a **single consumer** to get more than one message for the same group ID as the consumer knows the order of the messages (the order of the messages in the array) and it can process them in order.

If your code sends the messages to be processed in an asynchronous manner, you will break the order. You should process the messages one at a time. In order to process the messages quicker you should have multiple consumers, each will get messages from different groups IDs and this is how you will maintain the order and process the messages quicker.

Multiple SQS messages of the same group in one batch

Relevant content