MSK Upgrade - any concerns?

Question

We currently have a MSK cluster running Apache Kafka 2.4.1.1. There are three kafka.m5.large brokers and every topic has the replication factor set to 3.

We received a notification that this is becoming EOL 8 June and will ned to be updated to version 3.5.1 or higher and that Amazon will auto-upgrade.

Looking at the upgrade instructions, it appears to be just a straight forward click a button and Amazon will take of it.

Is there anything I should be concerned about breaking, clients not being compatible with 3.5.1 or downtime?

Answer

There are no known compatibility issues from the client perspective. But there is a functional change that may impact current behavior of producers, if/when you decide to update the client's libs as well. For messages with no key presented, the default partitioner in Kafka 3.3 has been changed, and it is not round-robin. Again, this is not a change on Kafka brokers, it's a change in client libs.

Also, I would recommend to run upgrade Kafka version API during off-peak window, making sure you don't have another maintenance scheduled, and having about 2 hours for the operation to complete.

MSK Upgrade - any concerns?

Relevant content