- Newest
- Most votes
- Most comments
Not sure if applicable, but have you considered using EventBridge pipes? You can either filter or use batching: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-batching-concurrency.html
Hi Craig.
You may be able to use EventBrige Pipes for your use case. If you can apply filters to your ingested records, using pipes can help you distribute the records to different destinations (multiple Kinesis Firehose) which would help coping with the hard limits.
I hope this helps.
Besides EventBridge pipes mentioned in other answers, one other thing to consider is to send data from Event Bridge to Kinesis Data Stream first and from there to Kinesis Firehose. KDS has much higher rate limits and you can design your shards for the proper write bandwidth. At the integration point between KDS and KFH, it is KFH that will pull from KDS so theoretically you should not hit any rate limits.
Relevant content
- Accepted Answerasked a year ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated a year ago