How to monitor Kafka (MSK) maintenance status?

Question

We're taking metrics from Kafka as described in [official guide](https://docs.aws.amazon.com/msk/latest/developerguide/metrics-details.html)

And we setup some alerts - for example, we have KafkaOfflinePartitions, KafkaUnderReplicatedPartitions and performance degradation

But during maintenance window AWS starts to update Kafka servers one-by-one. This triggers alerts described above

Is it possible to get status of maintenance window with some metrics, so I'll be able to mute this alerts during maintenance? Or what is best practice to deal with such situations

Harita · Answer

You can see the status of your cluster in two ways:

* Go to the AWS Management Console and check cluster status.
* Use the "describe-cluster-v2" command or the "DescribeClusterV2" operation to describe the cluster.

If your cluster's status shows "MAINTENANCE", it means that AWS is currently doing regular maintenance work on the cluster.

For a full list and explanation of all the possible cluster status values, you can check the documentation at the following link:
https://docs.aws.amazon.com/msk/latest/developerguide/msk-cluster-states.html

rePost-User-5855472 · Answer

Hello,

You may also monitor the `ActiveControllerCount ` Cloudwatch metrics in MSK and it  is observed at stats:avg

Ideally value for the ActiveControllerCount should be 1/count of brokers

For EG: Having 3 brokers in MSK 
ActiveControllerCount should be 1/3= 0.33

Thank you.

How to monitor Kafka (MSK) maintenance status?

Relevant content