Kinesis Data Analytics with MSK and Lambda

0

Hi, We are exploring with KDA for our streaming data ingestion pipeline for ML purpose. From https://aws.amazon.com/blogs/machine-learning/using-streaming-ingestion-with-amazon-sagemaker-feature-store-to-make-ml-backed-decisions-in-near-real-time/ it seems possible to connect KDA with MSK and use Lambda as destination to ingest data to Sagemaker Feature Store (or any custom location). However checking with KDA documentation it seems connecting with MSK is only possible with KDA studio or as a Flink app which does not provide a Lambda destination directly - instead would require another Kinesis stream in between, whereas KDA SQL application does support connecting with Lambda but not support reading from MSK. Our use case is very similar to the blog post above and we prefer to use Python as much as possible. What can be a viable path here?

Thanks, Rob

1 個回答
0

Hi,

Thank you for reaching out to us. Please note that I checked the documentation and could see that there is no mention of Kinesis Analytics application to MSK. The diagram shows the Kinesis Analytics application is connected to Lambda. Please note that KDA SQL application does not support writing to MSK directly. So the ways to write to MSK are to use Kinesis flink or if using SQL application , have lambda as destination and then lambda write to MSK

AWS
支援工程師
已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南