Can I use Kafka in KRaft mode on AWS MSK?

Question

I would like to use Kafka in KRaft mode instead of Zookeeper on AWS, since [Kraft mode seems to be the future of Kafka](https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka4.0). As I understand, basic solution from AWS for Kafka is AWS MSK. But in MSK documentation I see that they mention that they use Zookeeper under the hood of their infrastructure - https://aws.amazon.com/msk/faqs/ :

> ... With a few clicks in the console, you can create an Amazon MSK
> cluster. From there, Amazon MSK replaces unhealthy brokers,
> automatically replicates data for high availability, manages Apache
> ZooKeeper nodes, automatically deploys hardware patches as needed,
> manages the integrations with AWS services, makes important metrics
> visible through the console, and supports Apache Kafka version
> upgrades so you can take advantage of improvements to the open-source
> version of Apache Kafka.  ...

So is there a way to use AWS MSK with Kafka in KRaft mode?

I suppose, that I can run Kafka in KRaft mode during the [Create Topic step in AWS MSK guide](https://docs.aws.amazon.com/msk/latest/developerguide/create-topic.html), but I am not sure that the whole infrastructure will be set up and work the proper way after that.

I am new with AWS MSK and will be appreciate for your help. Probably I need to use some other solutions from AWS for this task. Global idea is to use Kafka in my Java based application with microservices deployed on AWS.

Answer

**Amazon MSK** currently uses **Zookeeper** for quorum management and there is no option currently to use **KRaft** on Amazon MSK. Please, see [Supported Apache Kafka versions reference](https://docs.aws.amazon.com/msk/latest/developerguide/supported-kafka-versions.html) for more details on this.

Regarding the part when you mentioned the [Create Topic step in AWS MSK guide](https://docs.aws.amazon.com/msk/latest/developerguide/create-topic.html), it does not use Kraft mode. Many of the command-line tools that previously used the *--zookeeper* flags have been updated to use the *--bootstrap-server* option. More details on the deprecation of the *--zookeeper* flag on Apache Kafka can be found on [this KIP](https://cwiki.apache.org/confluence/display/KAFKA/KIP-555%3A+Deprecate+Direct+Zookeeper+access+in+Kafka+Administrative+Tools).

Answer

On AWS MSK, you can use Kafka in KRaft mode. For managing Kafka clusters, AWS MSK supports both ZooKeeper and KRaft modes.

To create a Kafka cluster in KRaft mode, you can follow the same steps as creating a cluster with ZooKeeper, but with a few modifications.

To do this

Amazon MSK service > Create cluster > Basic configuration > Select Kafka Version > Cluster configuration > select the "KRaft" option for "Configuration type" > Configure the rest of the options for your Kafka cluster

Once your Kafka cluster in KRaft mode is up and running, you can create and manage topics using the Kafka command-line tools or the AWS MSK API.

Can I use Kafka in KRaft mode on AWS MSK?

관련 콘텐츠