AWS Builder Center: Learn, Build and Connect with builders in the AWS community
AWS Builder Center is the official home for builders on AWS. Share and read what others are working on, follow people who inspire you, explore training and workshops, and find tools to support what you're building.
[Kinesis Data Stream] [Enrichment] Question on Kinesis Data Stream Enrichment Solution
0
Hi Team,
Our Question:
Is there a feature that offered by Kinesis that allows us to enrich records in the same Kinesis stream? (get the plain record -> enrich it -> put it back into the same Kinesis, or enrich the records in place without take it out of the stream).
Background:
We are try to come up with a solution to enrich data in Kinesis Stream by join the incoming plain data with some static data.
After research I see there are many ways to do so, such as use Flink, Lambda, or EventBridge Pipes.
However, Flink is not an option for us. We use Flink to do aggregation task already, we don't want to centralize all tasks in a same service, plus it may be costly and difficult to scale in our use case.
For Lambda and EventBridge, the problem is that we have to create a new Kinesis stream to deliver the enriched data to Flink.
Is there any way to enrich the records in place without create a new Kinesis stream?
I understand you wish to get an approach to enrich data without creating a new kinesis stream . However unfortunately currently it is not possible to enrich the records in place without creating a new Kinesis stream .
