By using AWS re:Post, you agree to the Terms of Use
/RDS instance running out of EBS byte balance/

RDS instance running out of EBS byte balance

0

Hi Hivemind,

I am running a data-crunching workload on DB.t3. medium RDS instance (with a 600GiB gp2 volume and PostGres Engine).

I have observed that the EBS byte balance quickly reaches zero and remains at zero throughout the workload execution. Could someone let me know the effect of this on the workload? Will it degrade the rate of the read /write operations and eventually slow down the DB queries?

Please note that IOPS(at 4000), EBS IO Balance (at 100), and CPU credit balance (at 600) remain healthy throughout the execution.

2 Answers
0

The burst balance being exhausted will limit the read/write operations of the database to the baseline performance of the volume- this could absolutely impair an I/O intensive workload.

There is a CloudWatch metric called Burst Balance you can use to monitor the burst credits. This is what you should be looking at as I suspect it gets exhausted very quickly and does not replenish until the workload I/O falls below the baseline IOPS of the volume. If you increase the volume size to 1 TiB or greater, the volume IOPS will exceed the burst IOPS and eliminate burst as as a bottleneck.

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html#CHAP_Storage.IO.Credits

answered 3 months ago
  • The question was about EBS Byte balance, not Burst balance. We are having the same problem, but I am wracking my head trying to understand what that metric measures exactly, and hence what to do to prevent its depletion.

0

Description of the metric can be found here - https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-metrics.html

Information about EBS optimized instances is here - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.html#current. Using the describe-instance-types in that command, I see that the maximum bandwidth for t3.medium is 2085 Mbps. If you're getting EBSByteBalance% as zero, you're somehow consuming a lot of throughput, but not a lot of IOPS. When the byte balance is zero, you should be getting throttled on bandwidth and getting significantly less than the max 2085 Mbps

I would recommend summing up NetworkReceiveThroughput and NetworkTransmitThroughput metrics and monitor them alongside EBSByteBalance%. This would help with the confirmation.

On a side note, I'm not sure how you're getting 4000 IOPS. According to the docs, you should be getting between 1800 and 3000 IOPS - https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html.

answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions