Kinesis Data Analytics with MSK and Lambda

0

Hi, We are exploring with KDA for our streaming data ingestion pipeline for ML purpose. From https://aws.amazon.com/blogs/machine-learning/using-streaming-ingestion-with-amazon-sagemaker-feature-store-to-make-ml-backed-decisions-in-near-real-time/ it seems possible to connect KDA with MSK and use Lambda as destination to ingest data to Sagemaker Feature Store (or any custom location). However checking with KDA documentation it seems connecting with MSK is only possible with KDA studio or as a Flink app which does not provide a Lambda destination directly - instead would require another Kinesis stream in between, whereas KDA SQL application does support connecting with Lambda but not support reading from MSK. Our use case is very similar to the blog post above and we prefer to use Python as much as possible. What can be a viable path here?

Thanks, Rob

1 Risposta
0

Hi,

Thank you for reaching out to us. Please note that I checked the documentation and could see that there is no mention of Kinesis Analytics application to MSK. The diagram shows the Kinesis Analytics application is connected to Lambda. Please note that KDA SQL application does not support writing to MSK directly. So the ways to write to MSK are to use Kinesis flink or if using SQL application , have lambda as destination and then lambda write to MSK

AWS
TECNICO DI SUPPORTO
con risposta 2 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande