Amazon Forecast - How to choose Winning autoML algorithm

Question

Our customer tried AutoML on Amazon Forecast and they got the result below.
Winning autoML algorithm was NPTS.
How was NPTS selected?

NPTS 
wQL[0.5]/MAPE 0.6676
RMSE 23.2451
wQL[0.9] 0.4503
wQL[0.1] 0.1988

ARIMA
wQL[0.5]/MAPE 0.6184
RMSE 16.5769
wQL[0.9] 0.5333
wQL[0.1] 0.4487

Prophet
wQL[0.5]/MAPE 1.4359
RMSE 43.3114
wQL[0.9] 1.2045
wQL[0.1] 0.5908

ETS
wQL[0.5]/MAPE 0.6380
RMSE 18.0232
wQL[0.9] 31.9275
wQL[0.1] 31.7771

Accepted Answer

Hi,

Per the [How it Works - Predictors](https://docs.aws.amazon.com/forecast/latest/dg/howitworks-predictor.html) page of the doc:

> AutoML optimizes the average of the weighted P10, P50 and P90 quantile losses, and returns the algorithm with the lowest value.

For more information on which metrics are used and how they are calculated see  [Evaluating Predictor Accuracy](https://docs.aws.amazon.com/forecast/latest/dg/metrics.html)

Amazon Forecast - How to choose Winning autoML algorithm

相关内容