Storage Capacity on MSK maxed out
A new cluster with 3 brokers having EBS storage of 2 GB each was created which worked fine for a day . on the second day, the producer stopped sending messages and while checking the logs, found this error in the logs - 'Give up sending metadata request since no node is available'. We even tried to connect to the cluster but was receiving this exception. kafka.errors.NoBrokersAvailable: NoBrokersAvailable. Checked the Cloudwatch metrics and found that the storage capacity has reached a maximum limit. changed the values of configuration to retention.ms to 86400000 ms (24 hours) and retention.bytes to 500000000 bytes (500 MB) in order to release memory and also increased the EBS storage to 15 GB but the brokers were still not available.
Hello there, when storage capacity on MSK is already maxed out, deleting topics, adjusting retention etc wouldn't take any effect as the brokers will be in unresponsive state. In this scenario only option would be to increase the storage and once the brokers are healthy retention kicks in. I see you mentioned storage is increased to 15gb. Are the brokers still not available? Can you please make sure if storage update went through successfully by describing the cluster or checking whether 'EBS storage volume per broker' shows 15gb in MSK console.
Hi, when you re-size the EBS, what time take this proccess? because I re-size 500 GB to 1TB (13 hours have passed since the modification) and I can't send data to MSK. . I need reboot brocker? or It's automatically
Storage Capacity on MSK maxed outasked a month ago
Redshift ran out of storage during data CopyAccepted AnswerEXPERTasked 2 years ago
Why is there a cost of IOPS-month provisioned for activated ebs on stopped ec2?asked 4 months ago
Timeout sending messagesasked 3 years ago
Scaling a clusterasked 3 years ago
Storage Burst Balance - Cloudwatch Alarm - EBS, EFS, etc.asked 7 months ago
FSx unable to increase storage capacityasked 2 years ago
Automatically update brokerstring in MSKAccepted AnswerEXPERTasked 2 years ago
Does FSx for OpenZFS allow updating SSD storage capacity after creation?Accepted Answerasked 7 months ago
AWS Outposts pricing - Foot note on EBS Pricing for 33, 55 & larger TiersAccepted Answerasked 2 years ago