Exploring AWS Solutions: Stream processing for 1000+ devices – Kinesis Data Streams, Firehose or Alternative?

0

I'm designing a system that will listen to over 1000 streams (1 for each device), and should be able to process the streamed data to take action when needed. I need the processing and taking action to be near-real-time, but not actually real time.

All devices streams start from a RPC call, so the system must:

  1. Send the RPC call to attach it's session to the event stream.
  2. Active listen to the stream, if certain events are received trigger a lambda (or make an API call, tbd)
  3. Send keep-alive calls to maintain the session, every X seconds.
  4. Active listen to the stream 24/7/365 No long term storage required.

I don't know if I can use Firehose for this. Kinesis Data Stream seem to be a better match, however I don't know:

  1. if it can make the RPC calls (start and keep alive), or if I would need another service in between.
  2. If I would charged with 1000 streams per hour + data ingested? If so that would make it not financially feasible.

Finally, I'm also open to other services ideas

No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions