MessageDeduplicationId became the MessageId after being moved to the DLQ

0

Let's say the original message had:

  • MessageId: A
  • MessageDeduplicationId: B
    After being moved to the DLQ, MessageDeduplicationId became A. The main and DLQ is a FIFO queue.
    Here's the message from DLQ:
    {
    MessageId: 'a19e29fe-92a1-47fa-895f-7839b7e05868',
    ReceiptHandle: 'AQEBmbrbeMDcyADV/jd/xnCNihL1rB57ultwRTjzE_ps0aDqs1hAt39EzZL_ZovlWTAPR1EoYmGpO0ixEEvL8vQEMhYzqa71JClNe4sSMMUwv8gImsjknSbr_L5BuN5ob6Y6qolRgCM/GFBjXc/VHygjNQePyBdafg8RVASwgS3Yk6JrmhKm6BoduEk/k7J5f1aT3MC88w_Mvgc03oCBrmSqZ5tmTr330MWyUH505pFZZXE0EAtutXpqCNVCgea7cA7tpaWj4F/5IToO1/KyInpfKxYoDYrDGO56lYaiL33NCJM=',
    MD5OfBody: 'c81e728d9d4c2f636f067f89cc14862c',
    Body: '2',
    Attributes: {
    SenderId: 'AROAZ3AT2SL3K4UXABBDR:sqs_dlq_replay_function',
    ApproximateFirstReceiveTimestamp: '1622803955869',
    ApproximateReceiveCount: '6',
    SentTimestamp: '1622803940765',
    SequenceNumber: '18862182151616497664',
    MessageDeduplicationId: 'a19e29fe-92a1-47fa-895f-7839b7e05868',
    MessageGroupId: '3'
    }
    }

Is this an expected behavior? I'm using LocalStack and the MessageDeduplicationId is kept as-is after moving to the DLQ. I don't know AWS is correct, or LocalStack is correct?

Edited by: ducdnguyen on Jun 4, 2021 4:54 AM

posta 3 anni fa961 visualizzazioni
3 Risposte
0

Thank you for the report, we are looking into this behavior.

AWS
con risposta 3 anni fa
0

We have confirmed that this is an intended, but not document behavior of the dead-leter queue (DLQ) mechanism when using SQS FIFO queues.

SQS supports DLQ mechanism, which sends an un-processed message to DLQ. Such process is very similar to SendMessage, so most of SendMessage constraints apply, too.

FIFO queues support deduplication id, to ensure exactly-once processing. We also use this field internally when sending a message to DLQ. As such, it needs to be a globally unique id, to make sure that the DLQ deduplication will not prevent storing of two independent messages that happen to share a deduplication id.

For example, both queue 1 and 2 are set to same DLQ. We enqueue message A to queue 1, and message B to queue 2. Both messages have same deduplication id. These messages may be sent to DLQ in the same time. If we keep the original deduplication id, only one of them will make it to DLQ - as the other will be recognized as duplicate.
To avoid this, SQS chooses to use message id, a unique id to identify them during such process, so both A and B can be enqueued into DLQ.

We will follow up with an update to the official documentation to explain this mechanism.

If you plan to use the original message deduplication id when consuming from DLQ, we suggest storing it as extra message attribute or as part of the message body.

AWS
con risposta 3 anni fa
0

Thank you.
I had another thread going on with the AWS Support channel for this same behavior.
Just received the answer as well, and all answered my question.

con risposta 3 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande