Can AWS Managed Streaming for Apache Kafka be used as message queues?

Yes, AWS Managed Streaming for Apache Kafka (MSK) can be used as a message queue system, and it can fulfill your requirements for a push-based notification microservice without the need for a scheduler.

Amazon MSK is a fully managed service that allows you to build and run applications using Apache Kafka to process streaming data. While Kafka is often associated with stream processing, it can also be effectively used as a message queue system.

Kafka uses a publish-subscribe model where producers send messages to topics, and consumers read from these topics. This model allows for push-based message delivery, which aligns with your requirement of not wanting to use a scheduler to pull messages at regular intervals.

In Kafka, consumers can be grouped into consumer groups. When you have multiple consumers in a group reading from the same topic, Kafka automatically distributes the messages across these consumers. This behavior is similar to how a traditional message queue works, where each message is processed by only one consumer in the group.

To use MSK as a message queue for your notification microservice:

1. Create a Kafka topic for your notifications.
2. Set up your microservice as a Kafka consumer, subscribing to this topic.
3. Configure your consumer to use Kafka's consumer group feature.
4. Implement your notification logic within the consumer.

With this setup, when messages are produced to the topic, they will be automatically pushed to your consumer without the need for a scheduler. Your consumer will receive messages as soon as they are available, providing the real-time behavior you're looking for in your notification service.

It's worth noting that while MSK provides the Kafka infrastructure, you'll need to implement the consumer logic in your application code. This gives you flexibility in how you process the messages but also requires more setup compared to a simpler queue service like SQS.

In summary, AWS MSK (based on Apache Kafka) can indeed be used as a message queue system and provides the push-based behavior you're looking for in your notification microservice.
Sources
Amazon Managed Streaming for Apache Kafka (MSK) in AWS GovCloud (US) - AWS GovCloud (US)
Community | Topic or a Queue - What exactly is Apache Kafka?
Community | Topic or a Queue - What exactly is Apache Kafka?