Why doesn't my Amazon SQS FIFO queue return all messages or messages in other message groups?

4 minute read

My Amazon Simple Queue Service (Amazon SQS) First-In-First-Out (FIFO) queue doesn't return some messages when I call the ReceiveMessage API.


When you use FIFO queues, you can't request to receive messages from a specific message group ID. You also can't filter the ReceiveMessage API call to return only messages within one message group at a time.

When you specify a MaxNumberOfMessages parameter in the ReceiveMessage API call, SQS returns as many messages with the same message group ID as possible. The ReceiveMessage API call can also return messages from other available message groups.

For FIFO queues, when you receive messages that belong to a particular message group ID, note these guidelines:

  • You must delete the messages in the current message group ID before you can receive more messages within the same message group ID.
  • Messages that are received but not yet deleted are in the inflight state. Inflight messages block delivery of available messages within the same message group.
  • A common reason that messages are in the inflight state is because the consumer client fails unexpectedly. Another reason is when the downstream AWS Lambda consumer function encountered exceptions or reached its maximum concurrency.
  • The ChangeMessageVisibility API can extend or shorten the visibility timeout period for inflight messages.
  • You can wait until the message visibility timeout period expires to allow message redelivery.

For all FIFO queues and Amazon SQS quota limits, note these guidelines:

  • FIFO queues have a maximum of 20,000 inflight messages.
    Note: Amazon SQS doesn't return error messages that exceed quota limits.
  • FIFO queues can process up to 300 transactions per second (TPS) without high throughput activated. Requests over 300 TPS get the "ThrottlingException" error even if messages in the queue are available.
  • SQS FIFO queue messages are stored in partitions. Each partition supports up to 300 TPS for send, receive, and delete operations. For more information, see Optimizing partition utilization.
  • It's possible for FIFO queues with high throughput to share one partition with multiple message groups. If a message group exceeds 300 TPS for its partition, then messages from other groups on that partition aren't delivered because of throttling.
  • FIFO queue high throughput transactions per second varies dependent on the AWS Region. For more information, see Quotas related to messages.

To reduce Amazon SQS throughput limit on FIFO queues, note these guidelines:

A FIFO queue looks through the first 20,000 messages to determine the available message group. If message groups in the first 20,000 messages are blocked because of inflight messages, then messages from other groups beyond the first 20,000 won't return. For more information, see Avoid having a large backlog of messages with the same group ID.

Example A

A FIFO queue has a total of 20,001 messages. The first 20,000 messages belong to message group 1, and the last message belongs to message group 2. When you try to receive messages from the queue, you receive a message only from group 1. Any consecutive ReceiveMessage API calls result in empty receives. This happens because FIFO looks through only the messages that belong to group 1, and that group is blocked by the current call.

Example B

A FIFO queue has a total of 20,000 messages. The first 19,999 messages belong to message group 1, and the last message belongs to message group 2. When you try to receive messages from the queue, the first ReceiveMessage call gets a message that belongs to group 1. The second ReceiveMessage call gets a message that belongs to group 2. Any additional ReceiveMessage calls result in empty receives because both groups are now blocked by the current calls.

Related information

Amazon SQS FIFO (First-In-First-Out) queues

Using the Amazon SQS message group ID

AWS OFFICIALUpdated 6 months ago