Kinesis Data Analytics with MSK and Lambda

0

Hi, We are exploring with KDA for our streaming data ingestion pipeline for ML purpose. From https://aws.amazon.com/blogs/machine-learning/using-streaming-ingestion-with-amazon-sagemaker-feature-store-to-make-ml-backed-decisions-in-near-real-time/ it seems possible to connect KDA with MSK and use Lambda as destination to ingest data to Sagemaker Feature Store (or any custom location). However checking with KDA documentation it seems connecting with MSK is only possible with KDA studio or as a Flink app which does not provide a Lambda destination directly - instead would require another Kinesis stream in between, whereas KDA SQL application does support connecting with Lambda but not support reading from MSK. Our use case is very similar to the blog post above and we prefer to use Python as much as possible. What can be a viable path here?

Thanks, Rob

1 réponse
0

Hi,

Thank you for reaching out to us. Please note that I checked the documentation and could see that there is no mention of Kinesis Analytics application to MSK. The diagram shows the Kinesis Analytics application is connected to Lambda. Please note that KDA SQL application does not support writing to MSK directly. So the ways to write to MSK are to use Kinesis flink or if using SQL application , have lambda as destination and then lambda write to MSK

AWS
INGÉNIEUR EN ASSISTANCE TECHNIQUE
répondu il y a 2 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions