MessageDeduplicationId became the MessageId after being moved to the DLQ

0

Let's say the original message had:

  • MessageId: A
  • MessageDeduplicationId: B
    After being moved to the DLQ, MessageDeduplicationId became A. The main and DLQ is a FIFO queue.
    Here's the message from DLQ:
    {
    MessageId: 'a19e29fe-92a1-47fa-895f-7839b7e05868',
    ReceiptHandle: 'AQEBmbrbeMDcyADV/jd/xnCNihL1rB57ultwRTjzE_ps0aDqs1hAt39EzZL_ZovlWTAPR1EoYmGpO0ixEEvL8vQEMhYzqa71JClNe4sSMMUwv8gImsjknSbr_L5BuN5ob6Y6qolRgCM/GFBjXc/VHygjNQePyBdafg8RVASwgS3Yk6JrmhKm6BoduEk/k7J5f1aT3MC88w_Mvgc03oCBrmSqZ5tmTr330MWyUH505pFZZXE0EAtutXpqCNVCgea7cA7tpaWj4F/5IToO1/KyInpfKxYoDYrDGO56lYaiL33NCJM=',
    MD5OfBody: 'c81e728d9d4c2f636f067f89cc14862c',
    Body: '2',
    Attributes: {
    SenderId: 'AROAZ3AT2SL3K4UXABBDR:sqs_dlq_replay_function',
    ApproximateFirstReceiveTimestamp: '1622803955869',
    ApproximateReceiveCount: '6',
    SentTimestamp: '1622803940765',
    SequenceNumber: '18862182151616497664',
    MessageDeduplicationId: 'a19e29fe-92a1-47fa-895f-7839b7e05868',
    MessageGroupId: '3'
    }
    }

Is this an expected behavior? I'm using LocalStack and the MessageDeduplicationId is kept as-is after moving to the DLQ. I don't know AWS is correct, or LocalStack is correct?

Edited by: ducdnguyen on Jun 4, 2021 4:54 AM

질문됨 3년 전961회 조회
3개 답변
0

Thank you for the report, we are looking into this behavior.

AWS
답변함 3년 전
0

We have confirmed that this is an intended, but not document behavior of the dead-leter queue (DLQ) mechanism when using SQS FIFO queues.

SQS supports DLQ mechanism, which sends an un-processed message to DLQ. Such process is very similar to SendMessage, so most of SendMessage constraints apply, too.

FIFO queues support deduplication id, to ensure exactly-once processing. We also use this field internally when sending a message to DLQ. As such, it needs to be a globally unique id, to make sure that the DLQ deduplication will not prevent storing of two independent messages that happen to share a deduplication id.

For example, both queue 1 and 2 are set to same DLQ. We enqueue message A to queue 1, and message B to queue 2. Both messages have same deduplication id. These messages may be sent to DLQ in the same time. If we keep the original deduplication id, only one of them will make it to DLQ - as the other will be recognized as duplicate.
To avoid this, SQS chooses to use message id, a unique id to identify them during such process, so both A and B can be enqueued into DLQ.

We will follow up with an update to the official documentation to explain this mechanism.

If you plan to use the original message deduplication id when consuming from DLQ, we suggest storing it as extra message attribute or as part of the message body.

AWS
답변함 3년 전
0

Thank you.
I had another thread going on with the AWS Support channel for this same behavior.
Just received the answer as well, and all answered my question.

답변함 3년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠