Does Updating an Active MSK cause data loss
Hi there, I would like to update an active MSK configuration from 1 to 2. Would that cause any downtime on the current partition as the current data is crucial?
Hi there, are you looking to update MSK cluster configuration and apply a new revision to the cluster? Any updates or version upgrade wouldn't cause any data loss. When an update is triggered, cluster will undergo a rolling reboot i.e change/update will be applied on one broker at a time and rebooted.
During this time one broker will not be available and all the partition leadership this broker contains will be reassigned to other active brokers in the cluster at that time and those active brokers continue to accept client IO. As a best practice to avoid any disruption in client IO make sure to have RF of topics atleast equal to number of AZs and minISR = RF-1.
Thanks for the answer. The update was completed but partition counts didn't increase when I ran this command ./kafka-topics.sh --describe --bootstrap-server b-2.non-prod-***:9092 --topic. The config file should the increment though. Any idea on what am missing?
Hi again, based on question I'm assuming the config change that was made is related to number of partitions. Correct me if I'm wrong. If that is indeed the change you made, it applies only to the newly 'auto created' topics after the change has been made. This config doesn't change partition count of the existing topics. If you would like to change the partition count of existing topic, you can do so by running below command. Hope that answers!
./kafka-topics.sh --bootstrap-server <brokers> --alter --topic <topic> --partitions <new count>
Relevant questions
Clarifying MSK data transfer pricing within region or AZ
Accepted Answerasked 3 years agoMSK Connect - Exposing Connector Specific Metrics
asked 5 months agoCopying data from IBM MQ to Amazon MSK
asked 8 months agoData loss while MSK is in an HEALING state.
asked 2 months agoWhat changes an MSK cluster's "current_version"?
asked 2 months agoMSK Custom Configuration using Cloudformation
asked 4 months agoMSK ebs snapshot
asked 2 years agoDoes Updating an Active MSK cause data loss
Accepted Answerasked 6 months agoHow big is the risk when updating the Kafka version?
asked 2 years agoMax plugins per MSK connector = 1, but plugins is an array?
asked 8 months ago