How much data are you trying to move through the data stream to the firehose to S3 (per second)?
Kinesis Data Stream size, write, etc limits (and how to increase shards) is linked > here < Hopefully it's something straightforward like capacity, and can be resolved with an extra shard.
Troubleshooting on the Firehose service is > here < I'd recommend looking at the data freshness metric section - that talks about throttling.
Thank you for your answer. Your answer seems to be correct.
My Kinesis Data stream reported the throttling issue on reads. And this warning was not important because Kinesis Firehose could retry. After a short wait, the data was correctly recieved.
Hi there, the issue occured might due to you have multiple consumers for the Kinesis data stream, for the limitation, the Kinesis Data Fireshose use GetRecords API to retrieve data from data stream, for GetRecords API, it has 5 TPS for hard limit, which means you cannot invoke the API 5 times per second.
If the issue persist, you could try to do the following methods to optimize:
- recude the consumer count
- try to slow down the polling rate for other consumers (since you cannot control the pulling rate for Firehose)
- enable Enhanced fanout for your other consumers
Why does the Kinesis Firehose output "Throttling error encountered when calling Kinesis".asked 3 months ago
How to set document id when delivering data from Kinesis Data Firehose to Opensearch indexasked 5 months ago
Kinesis Analytics Application output to SNS?asked 10 days ago
Kinesis Transformation Buffering from Data StreamAccepted Answerasked 3 months ago
How to set the starting position for a Kinesis Delivery Stream
How to create a kinesis firehose delivery stream with dynamic partitions enabled using python cdk?asked 2 months ago
Picking the correct Opensearch index date from the Kinesis Delivery Stream
Kinesis Firehose Delivery Stream - S3 - JSONasked 22 days ago
Kinesis Stream Throttlingasked a month ago
Multiple Kinesis Data Analytics apps to use the same Kinesis firehose delivery stream as sourceAccepted Answerasked 2 years ago