Exploring AWS Solutions: Stream processing for 1000+ devices – Kinesis Data Streams, Firehose or Alternative?

0

I'm designing a system that will listen to over 1000 streams (1 for each device), and should be able to process the streamed data to take action when needed. I need the processing and taking action to be near-real-time, but not actually real time.

All devices streams start from a RPC call, so the system must:

  1. Send the RPC call to attach it's session to the event stream.
  2. Active listen to the stream, if certain events are received trigger a lambda (or make an API call, tbd)
  3. Send keep-alive calls to maintain the session, every X seconds.
  4. Active listen to the stream 24/7/365 No long term storage required.

I don't know if I can use Firehose for this. Kinesis Data Stream seem to be a better match, however I don't know:

  1. if it can make the RPC calls (start and keep alive), or if I would need another service in between.
  2. If I would charged with 1000 streams per hour + data ingested? If so that would make it not financially feasible.

Finally, I'm also open to other services ideas

1回答
0

You can use Amazon Kinesis Data Streams for ingesting and processing high-velocity data streams.

However, Kinesis Data Streams doesn’t directly handle RPC calls or session management. Consider using AWS Lambda to process incoming data from Kinesis.

For RPC calls and session management, you’ll need an additional service (e.g., an API Gateway).

Keep in mind the cost implications of using Kinesis Data Streams with 1000 streams.

profile picture
エキスパート
回答済み 5ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ