Does AWS SNS FIFO deduplication apply only within the same MessageGroupId?

Question

My understanding is:

AWS preserves the order of messages within each MessageGroupId.
Messages are only considered duplicates if they have the same MessageDeduplicationId and are sent within the 5-minute deduplication window within the same MessageGroupId; messages with the same MessageDeduplicationId but different MessageGroupIds are not treated as duplicates.

Can someone please confirm if deduplication is only enforced within each MessageGroupId, and not across different groups?

For example, if I send two messages with the same MessageDeduplicationId but different MessageGroupIds, will both messages be delivered within 5-minute deduplication window?

re:Post Agent · Answer

The scope of deduplication in AWS SNS FIFO topics depends on the `FifoThroughputScope` attribute setting:

1. When `FifoThroughputScope` is set to `Topic`, message deduplication applies across the entire topic, regardless of message group IDs. In this case, if you send two messages with the same MessageDeduplicationId but different MessageGroupIds within the 5-minute deduplication interval, only the first message will be delivered, and the second will be considered a duplicate.

2. When `FifoThroughputScope` is set to `MessageGroup`, message deduplication applies only within each individual message group. In this scenario, if you send two messages with the same MessageDeduplicationId but different MessageGroupIds within the 5-minute deduplication interval, both messages will be delivered because they belong to different message groups.

So your understanding is partially correct - deduplication can be enforced either across the entire topic or only within each MessageGroupId, depending on how the `FifoThroughputScope` attribute is configured. The default behavior depends on how you've set up your FIFO topic.

Every message sent to a FIFO topic must have a unique MessageDeduplicationId within the applicable deduplication scope and interval. If a message with a particular MessageDeduplicationId is sent successfully, subsequent messages with the same MessageDeduplicationId within the deduplication scope and 5-minute interval will be accepted but not delivered.  
**Sources**  
[Amazon SNS message deduplication for FIFO topics - Amazon Simple Notification Service](https://docs.aws.amazon.com/sns/latest/dg/fifo-message-dedup.html)  
[PublishBatchRequestEntry - Amazon Simple Notification Service](https://docs.aws.amazon.com/sns/latest/api/API_PublishBatchRequestEntry.html)  
[Publish - Amazon Simple Notification Service](https://docs.aws.amazon.com/sns/latest/api/API_Publish.html)

AWS-Dev · Answer

Thanks for the update. 
if FifoThroughputScope is not defined during SNS creation then it gets defaults to Topic ?

Does AWS SNS FIFO deduplication apply only within the same MessageGroupId?

Add your answer

Relevant content