Aggregate data from real time IoT device to DynamoDB

0

Hi,

Problem: I have an IoT Thing that will be sending MQTT messages at a certain frequency, let's assume 10 seconds. First, I want to save this to a DynamoDB via IoT Rule (no problem here) Then, I need to aggregate this data to a 5 minutes frequency and save it in another DynamoDB table. To put in other words, I want to resample my stream of data from 10 seconds to 5 minutes. The thing itself will be sending the timestamps, so it has to be based on the thing's timestamp. Assume the message has timestamp; project_id; power; and the power must be the average of the messages in the last 5 minutes.

It is desired that the resampled timeseries has whole numbers, e.g. 2023-08-31 12:00:00.00000Z; 2023-08-31 12:05:00.00000Z And that a message is never left out, due to lag for example.

What is the best architecture to achieve this? All my ideas so far don't have the robustness I'm looking for.

질문됨 9달 전277회 조회
1개 답변
1

Take a look over this Lab which goes into the details of using DynamoDB Streams and Lambda for aggregation.

Event Driven Architecture

You can also use Kinesis Data Streams and Kinesis Data Analytics:

https://aws.amazon.com/managed-service-apache-flink/

profile pictureAWS
전문가
답변함 9달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인