Browse through the questions and answers listed below or filter and sort to narrow down your results.
2
answers
0
votes
2
views
asked 2 months ago
How to Integrate a Kinesis Data Analytics Flink Application with a Self-Managed Kafka Cluster Running on Amazon EC2 (not Amazon MSK)?
I am interested to build a real time application with data residing on their Kafka cluster. The Kafka is running on AWS, using K8S not using MSK. What are the step required to integrate Kinesis Data Analytics Flink application with their Kafka cluster?
Accepted AnswerAmazon Kinesis Data Analytics
1
answers
0
votes
15
views
asked 2 years ago
KDA (Flink) to S3
A customer is building a streaming solution that processes messages from their customers. Data in the stream is sent from multiple customers - each JSON in the stream has a customerId that tells you what customer originally sent the data.
They want to store a copy of this data on S3, but partitioned by customerId. We *could* do this with multiple Firehoses (one for each customer). Is this possible to do with Flink - so use Flink to pull from the Kinesis stream and send to multiple destinations on S3, into files based on customer Id? I'm guessing this is possible - is it easy and can we use any existing libraries?
Thanks
Accepted AnswerAmazon Kinesis Data Analytics
1
answers
0
votes
12
views
asked 2 years ago