Kinesis Data Analytics with MSK and Lambda

0

Hi, We are exploring with KDA for our streaming data ingestion pipeline for ML purpose. From https://aws.amazon.com/blogs/machine-learning/using-streaming-ingestion-with-amazon-sagemaker-feature-store-to-make-ml-backed-decisions-in-near-real-time/ it seems possible to connect KDA with MSK and use Lambda as destination to ingest data to Sagemaker Feature Store (or any custom location). However checking with KDA documentation it seems connecting with MSK is only possible with KDA studio or as a Flink app which does not provide a Lambda destination directly - instead would require another Kinesis stream in between, whereas KDA SQL application does support connecting with Lambda but not support reading from MSK. Our use case is very similar to the blog post above and we prefer to use Python as much as possible. What can be a viable path here?

Thanks, Rob

1 回答
0

Hi,

Thank you for reaching out to us. Please note that I checked the documentation and could see that there is no mention of Kinesis Analytics application to MSK. The diagram shows the Kinesis Analytics application is connected to Lambda. Please note that KDA SQL application does not support writing to MSK directly. So the ways to write to MSK are to use Kinesis flink or if using SQL application , have lambda as destination and then lambda write to MSK

AWS
支持工程师
已回答 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则