Unexpected Concurrent Message Processing After AWS RabbitMQ Upgrade on July 16

The concurrent processing behavior you're observing is typically caused by RabbitMQ's pre-fetch mechanism. You can use the RabbitMQ pre-fetch value to optimize how your consumers consume messages. RabbitMQ implements the channel pre-fetch mechanism provided by AMQP 0-9-1 by applying the pre-fetch count to consumers as opposed to channels. The pre-fetch value is used to specify how many messages are being sent to the consumer at any given time. **By default, RabbitMQ sets an unlimited buffer size for client applications.

Recommended Solution

To restore sequential message processing, you need to explicitly set a pre-fetch value of 1 in your consumer configuration. This will ensure that RabbitMQ only sends one message at a time to each consumer.

Here's how to implement this using the RabbitMQ Java client library:

ConnectionFactory factory = new ConnectionFactory();
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();

// Set pre-fetch count to 1 for sequential processing
channel.basicQos(1, false);

QueueingConsumer consumer = new QueueingConsumer(channel);
channel.basicConsume("my_queue", false, consumer);

Additionally, note that:

If your client applications have configured to automatically acknowledge delivery of messages to consumers, setting a pre-fetch value will have no effect.

Make sure your consumers are using manual acknowledgment (not auto-ack) for the pre-fetch setting to work properly.

Source: Configure pre-fetching