Skip to content

Fanout Amazon SNS to Amazon SQS fair queues

1

Recently AWS has published a doc describing the new feature of fairness in regular SQS, called fair queues (link). Basically it uses the MessageGroupId field for performing fairness. When sending regular messages to a queue we could put that id as part of the request, but how could I use this feature when messages are sent to my regular queues via regular SNS (fanout).

Unsuccessful Attempts:

  • Published a msg with MessageGroupId to my regular SNS -> got http error by the SNS (because it is not fifo SNS).
  • Created fifo SNS and published a msg with MessageGroupId to it -> SQS received it, without the MessageGroupId.
3 Answers
1
Accepted Answer

For anyone interested, this feature was released on July 31st, 10 days after SQS fair queues were announced.

See Amazon SNS standard topics now support Amazon SQS fair queues.

AWS

answered a year ago

  • I think this should be the accepted answer, as the feature is now supported.

1

I was extremely excited to hear about these new fair queues but sad that it does not have support in SNS. Without that it is useless to us. Please address this feature gap quickly. All that is needed is for MessageGroupId to be forwarded to the standard queues. The topic should ideally be standard as well, and it should accept a MessageGroupId even if it doesn't use it for anything other than propagation.

answered a year ago

0

Since you are talking about queue fairs, I assume you are using SQS Standard queues. No, MessageGroupId is not propagated between SNS FIFO topics and SQS Standard queues. It is mentioned in the docs. See: Amazon SNS message grouping for FIFO topics :

When you publish messages to an Amazon SNS FIFO topic, you set the message group ID. The group ID is a mandatory token that specifies that a message belongs to a specific message group. The SNS FIFO topic passes the group ID to the subscribed Amazon SQS FIFO queues. There is no limit to the number of group IDs in SNS FIFO topics or SQS FIFO queues. Message group ID is not passed to Amazon SQS standard queues.

However, now with fair queues we have a valid use case to have the MessageGroupId propagated in this scenario. I will create the feature in the service roadmap.

AWS

answered a year ago

AWS
SUPPORT ENGINEER

reviewed a year ago

  • Thank you! Would be great to get notified here about ETA / completion of the new feature :)

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.