Is it possible to proactively clear all SQS messages by MessageGroupId?

0

I am managing our per-customer sub-queues in an SQS FIFO by setting MessageGroupId to the customer id. When one of these customer account is deleted on our end I want to clear out all pending SQS messages with that account's MessageGroupId to save consumer cycles in our workers. Is that possible? Thanks!

asked 2 years ago1056 views
2 Answers
0

Have you thought about creating an SQS queue for each customer? That would be a much easier way to manage a task like this in the long term, however; I understand that this may result in application changes. There are deleteMessage and deleteMessageBatch actions, but this requires you to know the message ID. The batch delete also only supports up to 10 messages at a time so depending on the amount of messages in the queue this could add up.

answered 2 years ago
  • My thinking was that if I created a separate FIFO for each customer then I would have to pro-actively poll each one of those queues which wouldn't scale to e.g. 150K customers. Especially since I have two queues that need to be consumed separately and in order: incoming + outgoing. But maybe you can suggest a way around that limitation?

0

There is no API in SQS to do that. You can either create a separate queue per customer, which may be problematic from management point of view and consumers, or, you can add code to the consumer that will verify, before actuall processing the message, that the customer is still active.

profile pictureAWS
EXPERT
Uri
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions