IoT Events - Calculating Moving Averages from Inputs

0

Hi, I have a built a detector model in IoT Events that takes input data from environmental sensors, and if the sensor values breach a threshold, an email alert is sent through Lambda. A timer is then set, and if the sensor values continue to breach the threshold 24 hours later, the email alert is resent. This solution is simple and works well. However, the problem is that the sensor input values can spike wildly. This can lead to a false alarm as after the first email is sent, the sensor input values can rapidly fall back down under the threshold.

I would like to create a moving average calculation that looks back over a predefined period of time and calculates the average for the environmental sensor inputs. For example, calculate the average of the sensor values over the last 12 hours. If the moving average value is above the threshold, send the initial email alert. Is it possible to calculate a moving average like this within IoT Events?

As I mentioned, the email alert is generated by a Lambda function, and the IoT Events model is triggered by an IoT Core Rule.

Thanks!

1 Answer
1
Accepted Answer

Hi. It's possible, but probably not the best solution. IoT Events doesn't really have a good mechanism to store a buffer of N samples to calculate an N-period moving average. You could achieve it with the setVariable action and a very complex detection model. I would instead recommend you involve other AWS services to pre-process the raw inputs or recommend you use an alternate low-pass filtering technique such as exponential smoothing.

Exponential smoothing only needs the current input and the last smoothed value and is thus quite easy to implement in IoT Events. You might also consider debouncing, some kind of outlier rejection or all manner of variations or combinations.

The HVAC example seems like a good fit for your use case, and it makes use of exponential smoothing (combined with anomaly/outlier rejection):

{
  "eventName": "calculateAverage",
  "condition": "$input.temperatureInput.sensorData.temperature < $variable.anomalousHigh && $input.temperatureInput.sensorData.temperature > $variable.anomalousLow",
  "actions": [
	{
	  "setVariable": {
		"variableName": "averageTemperature",
		"value": "((($variable.averageTemperature * ($variable.sensorCount - 1)) + $input.temperatureInput.sensorData.temperature) / $variable.sensorCount)"
	  }
	}
  ]
}

In this case the smoothing constant is 1 / $variable.sensorCount. You can of course use a different smoothing constant to tune for your filtering needs.

profile pictureAWS
EXPERT
Greg_B
answered 2 years ago
  • Hi Greg,

    Thanks for the thoughtful response. I loaded the HVAC detector model into IoT Events, and it looks pretty similar to what I am trying to achieve. From what I read about exponential smoothing constants, the value needs to be between 0 and 1. The higher the value, the more spikes will be present in the data. In my case, I am only testing with one set of sensors on a Raspberry Pi (3 sensors total per device). So I am not sure if the sensor count concept would apply here. In the HVAC example, it presumes a total of 12 sensors. Do you have any ideas for what an appropriate smoothing constant would be for this scenario? In the future, there will be N number of Raspberry Pis, but for now its only one with three sensors attached. Thanks!

  • The best value depends very much on your particular time series data. You said previously that the data "spikes wildly". If you used only an exponential smoother to solve it, you would probably need a lowish smoothing constant to smooth out these spikes. However, "spikes wildly" suggests to me that you should have some other kind of filtering, such as outlier rejection. Possibly that is all you need; maybe no exponential smoothing (nor moving average) is needed.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions