Kinesis Data Analytics with MSK and Lambda

0

Hi, We are exploring with KDA for our streaming data ingestion pipeline for ML purpose. From https://aws.amazon.com/blogs/machine-learning/using-streaming-ingestion-with-amazon-sagemaker-feature-store-to-make-ml-backed-decisions-in-near-real-time/ it seems possible to connect KDA with MSK and use Lambda as destination to ingest data to Sagemaker Feature Store (or any custom location). However checking with KDA documentation it seems connecting with MSK is only possible with KDA studio or as a Flink app which does not provide a Lambda destination directly - instead would require another Kinesis stream in between, whereas KDA SQL application does support connecting with Lambda but not support reading from MSK. Our use case is very similar to the blog post above and we prefer to use Python as much as possible. What can be a viable path here?

Thanks, Rob

1回答
0

Hi,

Thank you for reaching out to us. Please note that I checked the documentation and could see that there is no mention of Kinesis Analytics application to MSK. The diagram shows the Kinesis Analytics application is connected to Lambda. Please note that KDA SQL application does not support writing to MSK directly. So the ways to write to MSK are to use Kinesis flink or if using SQL application , have lambda as destination and then lambda write to MSK

AWS
サポートエンジニア
回答済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ