AWS MSK cloudwatch logs - disable DEBUG


I've upgraded AWS MSK service from 3.4.0 to 3.5.1, without changing any other configurations. After upgrade I see a lot of DEBUG message in CloudWatch Logs for AWS MSK Partition:

[2023-10-10 11:16:11,797] DEBUG [Partition haas.syslog-9 broker=1] Recorded replica 2 log end offset (LEO) position 0 and log start offset 0. (kafka.cluster.Partition)

This kind of logs appeared after upgrade, and I don't see how to disable it. For 1h, it generates more than 1mln logs.

Standard broker info logs present as well.

My MSK logs configuration - Basic monitoring enabled, Open monitoring with Prometheus and JMX exporter.

No custom cluster configuration provided.

Any suggestion how to disable such kind of logs? Regards, Oleksandr

1 Answer

Hello there,

It seems someone might have enabled the DEBUG logging. However, you can disable at broker level using the below commands:

bin/ --bootstrap-server localhost:9092 --describe --entity-type broker-loggers --entity-name 0 
// show all the log levels for broker 0
bin/ --bootstrap-server localhost:9092 --alter --add config "kafka.server.ReplicaManager=WARN,kafka.server.KafkaApis=DEBUG" --entity-type broker-loggers --entity-name 0
// set some log levels for broker 0

I hope this helps.

answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions