Kinesis Stream Throttling

Question

Hi All, 
I have a simple putRecordsRequest to Kinesis using the AWS SDK. I am running this code as a load generator in conjunction with TPS Generator to perform some load testing. When doing the load tests at high tps such as 100K, I get a LimitExceededExpectation: Rate exceeded for stream ingest-event-rishprak-dev-iad. Furthermore, looking at the cloudwatch dashboards for this stream, nearly all of the PutRecords are throttled - 99.7% PutRecords throttled. I have already increased the shards provisioned for the stream to 200 so that write capacity is 200k records per second. Could anyone provide me any insight as to why I am getting throttled in this stream? 
ARN is arn:aws:kinesis:us-east-1:407205872689:stream/ingest-event-rishprak-dev-iad

Chris_G · Answer

Hello,

Unfortunately we cannot troubleshoot your specific instance through re:Post, you will need to submit a ticket to our support team to troubleshoot your current Kinesis Data Stream.

If you're receiving the error message `LimitExceededException`, this is not related to the `PutRecords` method.  This error message is associated with other methods such as:
* `createStream`
* `describeLimits`
* `describeStream`
* `describeStreamSummary`

If you were running into a throttling/throughput issue, you might receive an error `ProvisionedThroughputExceededException`.

Can you confirm if you're utilizing partitions and if you're performing any additional methods outside of `PutRecords`?

Kinesis Stream Throttling

Add your answer

Relevant content