Need Help Ensuring Message Order in AWS SQS FIFO Queue

0

Hello AWS Community,

I am currently working on a project where maintaining the order of messages in an SQS queue is critical. I'm using an SQS FIFO queue with the expectation that messages will be processed in the exact order they are sent. However, I'm encountering an issue where the order of the messages in the queue is not being preserved as expected.

For example, if I send five messages in the sequence [1, 2, 3, 4, 5], they do not always appear in the queue in this order. Instead, I see them in orders such as [2, 3, 4, 5, 1] or even with duplicates like [1, 1, 2, 3, 4].

Here is a snippet of my code where I send messages to the SQS FIFO queue:

queueProducerUtility

import { Producer } from "sqs-producer";
import { SQSClient } from "@aws-sdk/client-sqs";
import { v4 as uuidv4 } from "uuid";

const queueUrl = process.env.QUEUE_URL;
const config = {
  region: process.env.REGION,
  credentials: {
    accessKeyId: process.env.ACCESS_KEY_ID,
    secretAccessKey: process.env.SECRET_ACCESS_KEY,
  },
};

// create simple producer
const producer = Producer.create({
  queueUrl: queueUrl,
  region: "us-east-1",
  sqs: new SQSClient(config),
});

const sendMessageToSQS = async (message) => {
  const response = await producer.send({
    id: uuidv4(),
    body: JSON.stringify(message.body.content),
    groupId: message.headers.customerChannelIdentifier,
    deduplicationId: uuidv4(), // typically a hash of the message body
  });
  return response;
};

export default sendMessageToSQS;

messageController

import RESPONSE from "../common/http-status-codes.mjs";
import AWSService from "../services/awsService.mjs";
import producer from "../utilities/queueProducerUtility.mjs";
import { Logger } from "../logger/logger.mjs";

const logger = Logger("Message-Controller");

const pushMessageToQueue = async (req, res) => {
  try {
          const _awsService = new AWSService(req);
          console.log("Mesaage Content : Producer  ", req.body.body.content);
          // const response = await producer(req.body);
          const sqsResponse = await _awsService.sendMessageToSQS();
        RESPONSE.OK(sqsResponse, res);
  } catch (error) {
          logger.error("Error in pushing message to Queue:", error);
  }

};

export default {
  pushMessageToQueue,
};
1개 답변
1
수락된 답변

How do you call pushMessageToQueue? Do you wait for it to complete before calling it again?

SQS FIFO preserves the order of messages as they are put into the queue, so you must make sure that you actually put them in order. For that you need to do it one after the other.

With regards to duplicates, you can use message deduplication to ensure that even if the producer tried adding the same message twice, it will appear in the queue only once.

profile pictureAWS
전문가
답변함 6달 전
profile picture
전문가
검토됨 6달 전
  • Yes i am using await so that it complete its execution before sending message again. you can check my code snippets

  • the issue is i am using group_Id that is used for scalling. After removing that all is working fine. thanks

  • SQS FIFO maintains order within all messages with the same message group ID, so you must include one and use the same for all messages that should be in order.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠