Streaming events from Kinesis Data Stream in account A to Kinesis Data Firehose in account B

0

Hello all,

I have a Kinesis Data Stream in account A, and I need to stream the events from it to Kinesis Firehose in account B. From what I see, there is no direct integration (even using IAM roles with assume role policy).

Is it possible to use EventBridge pipes to do the following:

KDS (Account A) -> EventBridge pipe (Account A) -> KFS (Account B)

UPDATE

  1. Account A can not directly write to targets in Account B.
  2. Lambda might be to costly
  3. Can EventBridge Pipes be used here?
2개 답변
1

Hi,

As you have observed, it's not possible to set a Kinesis Delivery Stream to consume from a Kinesis Data Stream in a different account. I believe a similar constraint would prevent the Event Bridge Pipes method.

To keep the serverless approach, you can use AWS Lambda to consume and process from the data stream, as native integration is provided that handles polling, checkpointing etc. You may find this blog useful - https://aws.amazon.com/blogs/big-data/best-practices-for-consuming-amazon-kinesis-data-streams-using-aws-lambda/

There is also the option to deploy the Kinesis Data Firehose in Account A and deliver to a target in Account B, for an example walkthrough of this, see https://repost.aws/knowledge-center/kinesis-firehose-cross-account-streaming

Hope that helps!

AWS
전문가
Greg K
답변함 8달 전
profile picture
전문가
검토됨 한 달 전
0

Hello,

Yes, there is no direct integration, but, couple of things can be done - write a lambda function in Account A listening for updates on the Stream and then write to Firehose in Account B (via cross account assume role). OR If Account A can directly write to Account B S3 bucket (if that is the end goal) then that is also simpler.

You may also want to check this link for more insights - https://docs.aws.amazon.com/managed-flink/latest/java/examples-cross.html

AWS
답변함 8달 전
profile picture
전문가
검토됨 한 달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠