Hi,
Problem:
I have an IoT Thing that will be sending MQTT messages at a certain frequency, let's assume 10 seconds.
First, I want to save this to a DynamoDB via IoT Rule (no problem here)
Then, I need to aggregate this data to a 5 minutes frequency and save it in another DynamoDB table.
To put in other words, I want to resample my stream of data from 10 seconds to 5 minutes.
The thing itself will be sending the timestamps, so it has to be based on the thing's timestamp.
Assume the message has timestamp; project_id; power; and the power must be the average of the messages in the last 5 minutes.
It is desired that the resampled timeseries has whole numbers, e.g. 2023-08-31 12:00:00.00000Z; 2023-08-31 12:05:00.00000Z
And that a message is never left out, due to lag for example.
What is the best architecture to achieve this? All my ideas so far don't have the robustness I'm looking for.