−11.01 | <0.0001 | −11.09 | <0.0001 |
SVR (20) | Validation | C=0.005206 |
0.026924 | 0.019478 | 0.014985 |
SVR (20) | Test | C=0.005206 |
0.036014 | 0.024916 | 0.016682 |
KNN (20) | Validation | Power of Minkowski metric=2 |
0.026328 | 0.020331 | 0.016199 |
KNN (20) | Test | Power of Minkowski metric=2 |
0.039305 | 0.025935 | 0.017202 |
XGBoost (27) | Validation | Max depth:7 |
0.027622 | 0.020678 | 0.016553 |
XGBoost (27) | Test | Max depth:7 |
0.038848 | 0.027218 | 0.019782 |
LGBM (43) | Validation | Number of leaves:58 |
0.025905 | 0.018803 | 0.014339 |
LGBM (43) | Test | Number of leaves:58 |
0.038870 | 0.026283 | 0.016467 |
LSTM (20) | Validation | H1 | 0.026565 | 0.019741 | 0.014537 |
LSTM (20) | Test | H1 | 0.036705 | 0.024918 | 0.016772 |
2 | S+NS SVR (0.504057), S+NS LightGBM (0.495943) | 0.03873 | 0.025882 | 0.017714 |
3 | S+NS SVR (0.337508), S+NS LightGBM (0.332075), S LightGBM (0.330417) | 0.038593 | 0.025959 | 0.017321 |
4 | S+NS SVR (0.255711), S+NS LightGBM (0.251594), S LightGBM (0.250338), S KNN (0.242357) | 0.038436 | 0.025665 | 0.017152 |
5 | S+NS SVR (0.206542), S+NS LightGBM (0.203217), S LightGBM (0.202202), S KNN (0.195756), S LSTM (0.192283) | 0.037734 | 0.025267 | 0.016599 |
6 | S+NS SVR (0.173976), S+NS LightGBM (0.171176), S LightGBM (0.170321), S KNN (0.164891), S LSTM (0.161965), S SVR (0.157671) | 0.03681 | 0.024751 | 0.01743 |
7 | S+NS SVR (0.150427), S+NS LightGBM (0.148006), S LightGBM (0.147266), S KNN (0.142572), S LSTM (0.140042), S SVR (0.136329), S+NS KNN (0.135359) | 0.036871 | 0.024897 | 0.016953 |
9 | S+NS SVR (0.119825), S+NS LightGBM (0.117896), S LightGBM (0.117307), S KNN (0.113568), S LSTM (0.111553), S SVR (0.108595), S+NS KNN (0.107822), S XGBoost (0.103175), S+NS XGBoost (0.100259) | 0.036898 | 0.024757 | 0.01645 |
10 | S+NS SVR (0.109125), S+NS LightGBM (0.107368), S LightGBM (0.106832), S KNN (0.103426), S LSTM (0.101591), S SVR (0.098897), S+NS KNN (0.098193), S XGBoost (0.093961), S+NS XGBoost (0.091305), S+NS LSTM (0.089302) | 0.036899 | 0.024915 | 0.016466 |
RMSE | 0.036106 | 0.038053 | 0.036746 | 0.037284 | 0.050244 | |
MAE | 0.024094 | 0.026068 | 0.024681 | 0.026295 | 0.034908 | |
MedAE | 0.015307 | 0.016645 | 0.01647 | 0.017959 | 0.022378 |
2 | SVR (0.504057), LightGBM (0.495943) | 0.038730 | 0.025882 | 0.017714 |
3 | SVR (0.346773), LightGBM (0.341191), KNN (0.312036) | 0.038314 | 0.026003 | 0.016682 |
4 | SVR (0.268786), LightGBM (0.264459), KNN (0.241861), XGBoost (0.224895) | 0.038301 | 0.025793 | 0.016876 |
2 | LightGBM (0.508099), KNN (0.491901) | 0.038571 | 0.025784 | 0.017147 |
3 | LightGBM (0.342575), KNN (0.331655), LSTM (0.32577) | 0.037403 | 0.025111 | 0.015704 |
4 | LightGBM (0.260092), KNN (0.251801), LSTM (0.247333), SVR (0.240775) | 0.036181 | 0.024366 | 0.01599 |
SVR (27) | Validation | C=0.005317, epsilon=0.092179 | 0.025632 | 0.019126 | 0.015488 |
SVR (27) | Test | C=0.005317, epsilon=0.092179 | 0.041904 | 0.025875 | 0.017279 |
KNN (40) | Validation | Power of Minkowski metric=1 |
0.027021 | 0.020110 | 0.013813 |
KNN (40) | Test | Power of Minkowski metric=1 |
0.039313 | 0.026863 | 0.018946 |
XGBoost (74) | Validation | Max depth:3 |
0.028021 | 0.021604 | 0.020396 |
XGBoost (74) | Test | Max depth:3 |
0.040685 | 0.026906 | 0.016939 |
LGBM (80) | Validation | Number of leaves:32 |
0.025840 | 0.019361 | 0.014083 |
LGBM (80) | Test | Number of leaves:32 |
0.037284 | 0.026295 | 0.017959 |
LSTM (20) | Validation | H2 | 0.028334 | 0.021702 | 0.018201 |
LSTM (20) | Test | H2 | 0.039593 | 0.028891 | 0.020576 |
1. | |
a. | the possibly largest group of hyperparameters will be selected according to best practice mentioned in literature, |
b. | one-step-ahead prediction will be done, providing Xi as train set and Yi as test set, and then one model with the lowest RMSE will be chosen, with parameters Hi. |
As a result, set {H1, H2, H3} is obtained. | |
2. | |
3. | Hj will be chosen, where Aj = min{A1, A2, A3}. It is the best set of hyperparameters, which is believed to assure stable fit in future forecasts. |
RMSE | 0.039305 | 0.038848 | 0.036705 | 0.038870 | |||
MAE | 0.025935 | 0.027218 | 0.024918 | 0.026283 | |||
MedAE | 0.017202 | 0.019780 | 0.016772 | 0.016467 |