Hello AWS Community,
I am currently working on a project where maintaining the order of messages in an SQS queue is critical. I'm using an SQS FIFO queue with the expectation that messages will be processed in the exact order they are sent. However, I'm encountering an issue where the order of the messages in the queue is not being preserved as expected.
For example, if I send five messages in the sequence [1, 2, 3, 4, 5], they do not always appear in the queue in this order. Instead, I see them in orders such as [2, 3, 4, 5, 1] or even with duplicates like [1, 1, 2, 3, 4].
Here is a snippet of my code where I send messages to the SQS FIFO queue:
queueProducerUtility
import { Producer } from "sqs-producer";
import { SQSClient } from "@aws-sdk/client-sqs";
import { v4 as uuidv4 } from "uuid";
const queueUrl = process.env.QUEUE_URL;
const config = {
region: process.env.REGION,
credentials: {
accessKeyId: process.env.ACCESS_KEY_ID,
secretAccessKey: process.env.SECRET_ACCESS_KEY,
},
};
// create simple producer
const producer = Producer.create({
queueUrl: queueUrl,
region: "us-east-1",
sqs: new SQSClient(config),
});
const sendMessageToSQS = async (message) => {
const response = await producer.send({
id: uuidv4(),
body: JSON.stringify(message.body.content),
groupId: message.headers.customerChannelIdentifier,
deduplicationId: uuidv4(), // typically a hash of the message body
});
return response;
};
export default sendMessageToSQS;
messageController
import RESPONSE from "../common/http-status-codes.mjs";
import AWSService from "../services/awsService.mjs";
import producer from "../utilities/queueProducerUtility.mjs";
import { Logger } from "../logger/logger.mjs";
const logger = Logger("Message-Controller");
const pushMessageToQueue = async (req, res) => {
try {
const _awsService = new AWSService(req);
console.log("Mesaage Content : Producer ", req.body.body.content);
// const response = await producer(req.body);
const sqsResponse = await _awsService.sendMessageToSQS();
RESPONSE.OK(sqsResponse, res);
} catch (error) {
logger.error("Error in pushing message to Queue:", error);
}
};
export default {
pushMessageToQueue,
};
Yes i am using await so that it complete its execution before sending message again. you can check my code snippets
the issue is i am using group_Id that is used for scalling. After removing that all is working fine. thanks
SQS FIFO maintains order within all messages with the same message group ID, so you must include one and use the same for all messages that should be in order.