Aggregate data from real time IoT device to DynamoDB

0

Hi,

Problem: I have an IoT Thing that will be sending MQTT messages at a certain frequency, let's assume 10 seconds. First, I want to save this to a DynamoDB via IoT Rule (no problem here) Then, I need to aggregate this data to a 5 minutes frequency and save it in another DynamoDB table. To put in other words, I want to resample my stream of data from 10 seconds to 5 minutes. The thing itself will be sending the timestamps, so it has to be based on the thing's timestamp. Assume the message has timestamp; project_id; power; and the power must be the average of the messages in the last 5 minutes.

It is desired that the resampled timeseries has whole numbers, e.g. 2023-08-31 12:00:00.00000Z; 2023-08-31 12:05:00.00000Z And that a message is never left out, due to lag for example.

What is the best architecture to achieve this? All my ideas so far don't have the robustness I'm looking for.

asked 8 months ago266 views
1 Answer
1

Take a look over this Lab which goes into the details of using DynamoDB Streams and Lambda for aggregation.

Event Driven Architecture

You can also use Kinesis Data Streams and Kinesis Data Analytics:

https://aws.amazon.com/managed-service-apache-flink/

profile pictureAWS
EXPERT
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions