Keyspaces: Why do i get WriteThrottleEvents when there is plenty of write capacity available?
When attempting to import data into keyspaces, even with provisioned WCU of 40000, I am only able to import a few thousand records per second. The graphs show that it is only using a small amount of the available capacity, but also shows that the WriteThrottleEvents are through the roof. What causes writes to become throttled even though the provisioned capacity is barely being used? The errors returned to the client are all WriteTimeout.
Thank you for using Apache Cassandra.
In AWS Keyspaces(for Apache Cassandra), the throttling exception on the table gets translated to timeout exceptions at the client side. Similar to DynamoDB, even in AWS Keyspaces, data is distributed across multiple partitions at the backend and each partition has a throughput limit of 3000 RCU or 1000 WCU. Throttling in Keyspaces table occurs due to :
- In case of table with provisioned capacity mode : exceeding the provisioned capacity allocated for the table or exceeding the partition level throughput limit.
- In case of table with on demand capacity mode: consuming more than double the previous traffic peak for the table or exceeding the partition level throughput limit
- If the throttling is due to exceeding the table’s provisioned throughput, then increase the provisioned capacity accordingly based on your table’s traffic or enable autoscaling on the table.
If the throttling is due to exceeding the partition level throughput limit, then:
- Increasing the provisioned capacity won’t help.
- Have a look at your access patterns and distribute read and write operations evenly across the backend partitions of your table and not exceeding the partition level throughput limit of 3000 IOPS.
- Implement error retries based on exponential back off strategy - If not using AWS SDK, then you will have to implement a retry strategy which retries these throttled requests multiple times (10 times) based on exponential backoff strategy.
If you still have further concern, we would recommend you to open a case with Premium Support we require details that are non-public information. Please open a support case with AWS using the following link
Cloudformation Import of RDS resources modifying settingsasked 4 months ago
Why does AWS DynamoDB separate RCU and WCUasked 5 months ago
MIgrating from self-managed Cassandra to KeyspacesAccepted Answerasked 2 years ago
Pricing questionasked 2 years ago
How do I test my code to handle UnprocessedItems on a BatchWriteItem?asked 3 months ago
VM Export of AMI is not Completingasked a year ago
'unable to complete the operation against any hosts' regular errorasked a year ago
Keyspaces: Why do i get WriteThrottleEvents when there is plenty of write capacity available?asked 4 months ago
Missing table size metrics in Cloudwatch for Amazon Keyspacesasked 2 years ago
prepare() server internal error when using TTLasked 2 years ago