MessageDeduplicationId became the MessageId after being moved to the DLQ

0

Let's say the original message had:

  • MessageId: A
  • MessageDeduplicationId: B
    After being moved to the DLQ, MessageDeduplicationId became A. The main and DLQ is a FIFO queue.
    Here's the message from DLQ:
    {
    MessageId: 'a19e29fe-92a1-47fa-895f-7839b7e05868',
    ReceiptHandle: 'AQEBmbrbeMDcyADV/jd/xnCNihL1rB57ultwRTjzE_ps0aDqs1hAt39EzZL_ZovlWTAPR1EoYmGpO0ixEEvL8vQEMhYzqa71JClNe4sSMMUwv8gImsjknSbr_L5BuN5ob6Y6qolRgCM/GFBjXc/VHygjNQePyBdafg8RVASwgS3Yk6JrmhKm6BoduEk/k7J5f1aT3MC88w_Mvgc03oCBrmSqZ5tmTr330MWyUH505pFZZXE0EAtutXpqCNVCgea7cA7tpaWj4F/5IToO1/KyInpfKxYoDYrDGO56lYaiL33NCJM=',
    MD5OfBody: 'c81e728d9d4c2f636f067f89cc14862c',
    Body: '2',
    Attributes: {
    SenderId: 'AROAZ3AT2SL3K4UXABBDR:sqs_dlq_replay_function',
    ApproximateFirstReceiveTimestamp: '1622803955869',
    ApproximateReceiveCount: '6',
    SentTimestamp: '1622803940765',
    SequenceNumber: '18862182151616497664',
    MessageDeduplicationId: 'a19e29fe-92a1-47fa-895f-7839b7e05868',
    MessageGroupId: '3'
    }
    }

Is this an expected behavior? I'm using LocalStack and the MessageDeduplicationId is kept as-is after moving to the DLQ. I don't know AWS is correct, or LocalStack is correct?

Edited by: ducdnguyen on Jun 4, 2021 4:54 AM

已提问 3 年前961 查看次数
3 回答
0

Thank you for the report, we are looking into this behavior.

AWS
已回答 3 年前
0

We have confirmed that this is an intended, but not document behavior of the dead-leter queue (DLQ) mechanism when using SQS FIFO queues.

SQS supports DLQ mechanism, which sends an un-processed message to DLQ. Such process is very similar to SendMessage, so most of SendMessage constraints apply, too.

FIFO queues support deduplication id, to ensure exactly-once processing. We also use this field internally when sending a message to DLQ. As such, it needs to be a globally unique id, to make sure that the DLQ deduplication will not prevent storing of two independent messages that happen to share a deduplication id.

For example, both queue 1 and 2 are set to same DLQ. We enqueue message A to queue 1, and message B to queue 2. Both messages have same deduplication id. These messages may be sent to DLQ in the same time. If we keep the original deduplication id, only one of them will make it to DLQ - as the other will be recognized as duplicate.
To avoid this, SQS chooses to use message id, a unique id to identify them during such process, so both A and B can be enqueued into DLQ.

We will follow up with an update to the official documentation to explain this mechanism.

If you plan to use the original message deduplication id when consuming from DLQ, we suggest storing it as extra message attribute or as part of the message body.

AWS
已回答 3 年前
0

Thank you.
I had another thread going on with the AWS Support channel for this same behavior.
Just received the answer as well, and all answered my question.

已回答 3 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则