Verification of the Weather Research and Forecasting Model (WRF) for the Domain of Silesian Voivodeship and Upper Silesia Metropolitan Region

In recent years, in the situation of climate change and its effects on the quality of life of the population, one of the problems to be solved has been the adoption of these changes in various areas of socio-economic life at the local level. In the developed plans for adaptation to climate change, which are becoming one of the local strategic documents of socio-economic development of the cities/municipalities, the impact of climate change on air quality and its effects on the health and life of residents is announced. The basis of the air quality forecasting system is the forecast of meteorological parameters at the local level, based on global forecasting, which is used by the WRF (Weather Research and Forecasting model). Such a solution was adopted in the construction of the air quality forecasting system and health effects for the inhabitants of the Silesian Voivodeship - InfoSMOG-MED [1], in which the forecast of selected meteorological parameters from the WRF model determines, on the one hand, the demand for heat in the housing stock, and this in turn affects the consumption of fuels in the sources installed in municipal housing resources used for this purpose, which generate emissions of pollutants into the atmospheric air, taking into account their efficiency, depending on the type and age of the source.

The amount of emissions from emission areas (from industrial, municipal and road transport sources), constituting input data to the pollution spread model, taking into account the values of forecasted meteorological parameters, gives an air quality forecast with simultaneous information on health threats for residents with diseases of the cardiovascular system, respiratory system, diabetes diseases and diseases of the youngest inhabitants of the population Silesian Voivodeship (paediatric). The structure of the above-described air pollution forecasting system is shown on the below scheme (Figure 1.).

Figure 1.

The WRF model has been elaborated on the basis of the Global Forecast System (GFS) - provided by the Weather Prediction Center (WPC). The applied WRF model (version 4.2) generates 156 meteorological parameters [2], from which 14 were selected. These are:

The worm-up (spin-up) time was 6 hours and the forecast length was 24 hours.

The selection of the domain size was based on the following requirements. The 10 km grid size for the area of Poland was selected as the resolution compatible with the EMEP grid system, both for emission inventory and concentration assessment of air pollution. The 2km grid size for the area of Silesian Voivodeship was selected for the calculation of the forecast of the air pollution concentrations, used in the InfoSMOG system. The 1km grid size for Upper Silesia Metropolitan Region was selected for the calculation of meteorological parameters for determining the boundary conditions of personal physical training in free air for the selected population. The size of the areas and grids were adjusted to the capacities of the computational matrix. WRF 3.9.1.1 version was used, and the initial resolution was 0,5 degree.

The WRF model has been used also for the prediction of the meteorological parameters in the SMART project [3] for the determination of the activities and training rationalization system for different population groups with respect to the forecast of meteorological parameters.

As well for the InfoSMOG-MED as for the SMART project the results of the accuracy of the meteorological parameter prediction are very important because it is connected with the daily and even hourly activities of the inhabitants in the particular locations connected with the air quality and in the case of the SMART project even based on the additional recommendations of the doctors of the physiotherapist on the daily activities patients.

Thus, the main objective of the study undertaken was to evaluate the statistical errors of forecasted values of meteorological parameters between modelled data with adjustment derived from data from synoptic stations (IMGW and METAR) relative to model data without adjustment, with respect to 3 spatial domains, and to assess to what extent a reduction in the model’s computational grid will reduce the error of forecasted values of meteorological parameters.

Figure 2.

Figure 3.

The results of the six scenarios of the data comparisons in the 3 computing domains are presented in this article.

Statistical studies of errors in forecasted values of meteorological parameters were carried out for 3 computing domains. The spatial extent of the domains is as follows: Polish domain no.1 circa 1 825 196 km² (Fig. 2) –D1; Silesian Voivodeship domain no. 2 circa 56 450 km² – D2; Upper Silesia Metropolitan Region domain no. 3 circa 8 824 km² – D3 (Fig. 3). Additionally European domain circa 15 556 357 km² have been used in one scenario.

In domain D1 the grid resolution was 10km x 10km, in the domain D2 2 km x 2 km and in the domain D3 1 km x 1 km. The six scenarios for the statistical analyses of the temperature and wind speed data have been established. Comparisons were made between:

Scenario 1 refers to domain D1 contained 547 992 data pairs for the statistical comparisons, scenario 2 refers to domain D2 contained 154 008 data pairs for the statistical compare and scenario 3 refers to domain D3 contained 215 784 data pairs for the statistical comparisons. Scenario 4 refers to an area covering a common part of domains D2 and D3, which contained 134 136 data pairs for the statistical comparison.

Scenario 5 refers to domain 3 contained 215 784 data pairs for the statistical comparison. Scenario 6 refers to a single point of the meteorological station while in terms of time, it covered a period of 8 months with the 728 pairs for the statistical comparison.

The following designations have been used in the statistical analyses:

For a more complete picture of the variables subjected to comparative analysis, basic statistics were calculated such as minimum, maximum, mean, standard deviation, median and median deviation of MAD (median absolute deviation from the median).

Description of the error estimators

Six error estimators were used in the aforementioned scenarios. The calculation formulas of them are shown in the table below (Table 1).

The following error estimators were used in this study: mean error (ME), mean absolute error (MAE), mean square error (MSE), root mean square error (RMSE), percent bias (PBIAS) and Nash-Sutcliffe efficiency (NSE).

MAE, MSE and RMSE are commonly used error estimators in model evaluation. Their purpose is that they specify the error in units of the variable being evaluated, which facilitates the analysis of the results (Moriasi at el. 2007) [4]. Values of the MAE, MSE and RMSE parameters equal to 0 indicate a perfect fit between the data being evaluated and the model data. It is considered that parameter values of MAE, MSE and RMSE less than half of the standard deviation of the analysed variables can be considered low and that each of them is appropriate for the evaluation of the model (Singh at el. 2004) [5]. The value of the error estimator ME determines the global trend in the analysed data. ME values greater than zero indicate an underestimation of the model results, and values less than zero indicate an overestimation of the model values.

Percent bias (PBIAS) is a measure of the mean trend (tendency) of the evaluated data relative to the benchmark data (e.g. simulated data to observed data or adjusted data to unadjusted data). The optimal value of the PBIAS estimator is zero. Low values of PBIAS indicate an accurate simulation of the model results. Positive values of PBIAS indicate an underestimation of the model values, while negative values indicate an overestimation of the model data.

Nash-Sutcliffe efficiency (NSE) measures the relative magnitude of the residual variance (noise) compared to the measured variance of the data (information) (Nash at el. 1970) [6]. NSE indicates how well a plot of observed versus simulated data fits a 1:1 line. NSE ranges from ∞ to 1.0 inclusive, with NSE = 1 being the optimal value. Values between 0.0 and 1.0 are generally seen as acceptable levels of performance, while values <0.0 indicate that the average observed value is a better predictor than the simulated value, indicating unacceptable performance (Moriasi at el. 2007).

The temporal extent of the analyses

Calculations of differences between modelling results according to scenarios 1, 2, 3, 4, and 5 were carried out for four selected days representing four climatological seasons: 2021-03-26 (spring), 2021-06-14 (summer), 2021-09-11 (autumn) and 2021-12-26 (winter). Calculations of differences between the results of modelling and measurements for the Silesian Planetarium station were made for data from the period of 1.07.2021-28.02.2022.

The conducted research and analyses showed that the verification of the Weather Research and Forecasting Model (WRF) for the domain of Silesian Voivodeship and Upper Silesia Metropolitan Region in the range of the air temperature and wind speed can be assessed as positive and it can be stated that system used for forecast meteorological parameters in the projects InfoSMOG-MED and SMART using the WRF model in terms of air temperature and wind speed forecast with sufficient accuracy, reproducing real conditions. However, when the seasonal changes in error level were compared it can be seen that, on average, the values of three error estimators (MAE, RMSE and NSE) for the five scenarios were most favourable for days representing the summer and autumn seasons and least favourable for days representing the spring and winter seasons. The last one is particularly unfavourable. This pattern is corroborated by the results of NSE.

Comparing the effect of data adjustment on the final result for 3 domains (scenarios 1, 2, 3) it can be seen that the highest influence pertains to domains no. 2 and 3. This means that for the domain D1 (nationwide) the correction of model data with measurement data can be omitted due to small differences between the model data without correction and the model data with correction. In the case of domains D2 (Silesian Voivodeship) and D3 (Upper Silesia Metropolitan Region), it is necessary to correct model data with measured data from synoptic and METAR stations.

The average of the aggregated errors (MEA, MSE and RMSE) for the four seasons is 1.05, 1.65 and 1.64 °C for the D1, D2 and D3 domains, respectively.

It shows a slight difference in aggregate error values between the D2 and D3 domains, which indicates that the density of the calculation grid in the model does not improve the quality of the air temperature forecast while increasing the time required to perform the forecast calculations. This conclusion is supported by an analysis of the differences in scenario four.

The differences in the values of the forecasted temperature (obtained on the basis of modelled data nationwide and European domains) for scenario 5 and domain D3 are small. The use of European data for correction of the model results to a small extent overestimating the forecasted temperature values in relation to the forecast values using Polish measured data for correction.

In scenario 6 the measured data were compared with modelled data for the grid in which the Planetarium climate station is located. The percentage error of PBIAS obtained for the temperature was on the level of about 10%, while the average of aggregated errors (MAE, MSE and RMSE) was on the level 2.8 °C. It shows that the modelled data are underestimated.

When it comes to wind speed, it can be seen the greater differences in relation to air temperature. For scenarios 1, 2, 3 and domains D1, D2, D3, the error increases with the increase of the density of the model calculation grid. The average of the aggregate errors (MEA, MSE and RMSE) calculated for the four seasons was 0.69 m/s (domain D1), 1.2 m/s (D2) and 1.53 m/s (D3), respectively. In the case of the third domain, the differences between the data with and without correction are so large that the value of the NSE indicator is less than 0, i.e. -0.321.

Particularly large discrepancies were obtained for the winter season and also, to a lesser extent, for the spring season. For winter, the PBIAS value is - 42.19%, which means that without correction, the modelling results are very overstated. The errors calculated for scenario 4 are small for both the corrected data and the non-adjusted model data. The differences obtained in the 5th scenario are moderate and are around 0.5m/s (average of aggregated errors).

Scenario 6 shows a high level of discrepancy between modelled and measurement data for wind speed. The NSE is approximately -16.2 and the PBIS is at the level -217.5%. The results obtained indicate that the selected station is not suitable for correcting the model data for wind speed.

The final conclusion is that continuation of the research is necessary on the improve the wind speed forecast data during the autumn and winter seasons and improve the wind speed forecast data in a particular location due to forecasted data for the calculated grid.

It also can be concluded that reducing the mesh of the model calculation does not reduce the error of the forecasted values of meteorological parameters in this case air temperature and wind speed.