Quality control of time-series seawater temperature and wave data adapted to the regional conditions of the Baltic Sea

A spike test, which determines the relationship between adjacent measurements, is a good verification indicator for flagging suspicious records from the database. The Copernicus Marine Environment monitoring service (Copernicus Team, 2020) estimated and presented the spike test results, which are valid for the entire Baltic Sea area (Table 2). In the case of wave parameters, the test formula remains unchanged, and the test result depends on the analysed parameter.

Considering the data resolution and limitations of the reanalysis data, the authors attempted to identify new spike test result limits or confirm those proposed by Copernicus. A result for the maximum wave height parameter was also introduced, which was not included in Copernicus.

The data come from the open sea zones (Tables 3 and 4) and the shallow water zones (Tables 5 and 6). In both locations, the period was divided into a non-storm season (April–August) and a stormy season (IX–III), allowing us to specify the test result values in seasonal terms.

In the non-storm season, the test results were close to those proposed by Copernicus. Based on the data from the Petrobaltic point, the test result value for significant wave height remains unchanged compared to the one proposed by Copernicus, that is, 3 m. The mean wave period increased from 4 s to 5 s, and the peak wave period doubled from 10 s to 20 s. In the storm season, the resulting values were 5.5 m for significant wave height, 10 s for the mean wave period, and 17.4 s for the peak wave period.

The point in the Pomeranian Bay is not classified as open water of the Baltic Sea, which means it is impossible to refer to the values proposed by Copernicus. In addition, the measuring device measures parameters such as significant wave height and peak wave period, and the following limit values have been proposed: Hs—0.55 m and Tp—4 s in the stormless season and 0.8 m and 4.5 s in the stormy season.

The set of temperature measurements tested using the spike test formula (1) includes 190,967 results measured at eight coastal stations from 1946 to 2019. The test limit value for temperature results is 6°C.

At the same time, it should be noted that this tested formula skips the first and last measurements from the analysed time interval due to the lack of possibility of comparison to neighbouring values. Therefore, 190,951 values were tested out of all results. Only one outlier measurement was identified using the test limit indicated in the SeaDataNet guidelines (6°C).

It is strongly recommended that measurements be carried out at 1-day intervals. In the case of longer time intervals, differences between measurements may result from seasonal pattern temperature variations rather than from incorrect values.

Considering such a large number of water temperature measurements at coastal stations from 1946 to 2019, it may be surprising that only a single outlier was identified using the spike test. On the one hand, this may indicate a very good and careful system of collecting, verifying, and recording results. On the other hand, it is necessary to consider whether adopting the globally indicated limit values of this test for regional results is appropriate.

The global limit value for the spike test in the guidelines is 6°C.

After calculating the test statistics for the temperature results according to the formula (1), the 99th percentile was calculated as 1.5. The test results indicate that 99% of the differences between adjacent temperature measurements are ≤1.5°C. This result was mathematically rounded to 2, and this value was adopted as the test limit value for temperature measurements in the southern Baltic Sea (Table 7). Simultaneously, the indicated limit value is used to flag suspicious values whose correctness should be subjected to expert assessment.

The second test conducted on 190,951 temperature results indicated 703 outlier measurements.

As a result of the test based on the mean and four times the standard deviation, 154 records were flagged at the Petrobaltic point, constituting a total of 0.32% of all records in the analysed data. The largest number, that is, 101 records, was flagged for the significant wave height parameter in the stormless season and 41 in the stormy season. Both the mean and peak wave periods in the storm season were characterised by a small number of flagged records (2–5), while in the non-storm season, they were four and one, respectively, representing only 0.012% (Table 8).

In the Pomeranian Bay, the mean and four times the standard deviation test showed a higher percentage of flagged measurements compared to the data set from the Petrobaltic point. In the storm season, the values for Hs and Tp were 0.71% and 0.80%, respectively, and in the stormless season 0.21% and 1.02%, respectively (Table 9).

The set of temperature measurements, tested using Dixon’s 4σ formula (2), includes 190,967 results measured at eight coastal stations from 1946 to 2019. Due to seasonal temperature, the range of meteorological frequencies of the year is extendedFor each station, the mean and standard deviation were calculated for the results measured in one of the four meteorological seasons. As a result of the test, eight outlier measurements were identified (Table 10).

The observations allow us to conclude that a positive feature of this test is its universal nature, which means that it can be properly applied to any parameter.

This test is also used to identify gross errors. Therefore, low temperatures in summer and high temperatures in winter are particularly noteworthy in the above list. However, there are no measurements from the spring and autumn seasons.

The set of temperature measurements at open sea stations includes 40,651 results measured from 1959 to 2019. At both coastal and offshore stations, the seasonality of meteorological seasons was taken into account when calculating average values and standard deviation. . With the adopted assumptions, the test identified 23 outlier measurements among temperature measurements at open sea stations (Table 11).

In the case of open sea stations, a much greater temporal and spatial diversity of outliers can be observed. Low temperatures during the summer season continue to dominate and concern mainly measurements made <50 m.

The Dixon test was distinguished by the high detection of significant wave height measurements compared to the mean wave period. At the Petrobaltic point, 142 significant wave height records and 6 records each for the mean and peak wave periods were identified. Using the Dixon test at the Pomeranian Bay point, it was possible to flag 143 phallic height records and 143 mean wave period records.

The test was performed on 190,967 temperature results at coastal stations. Taking into account the limitation of the test resulting from the maximum number of tested results (40), each month in which measurements were made over the years 1946–2019 was analysed separately. The results were categorised into ranges: from 3 to 7 measurements per month, from 7 to 12, and above 12 measurements to use the right Q-Dixon test formula to calculate the critical values of Q₁ and Q_n and to assign the right critical value of Q_kr. This test showed the presence of 282 outliers in the analysed data set.

The Hampel test at the Pomeranian Bay point detected 143 outliers for significant wave height and the same number for the mean wave period, constituting 0.81% of the total data set. At the Petrobaltic point, representing deep water conditions, there were many fewer flagged measurements compared to shallow water conditions, that is, the Pomeranian Bay, and they amounted to 68 measurements each for significant wave height and mean wave period.

The Hampel test was conducted for all temperature measurements taken at coastal stations. As in the case of the Q-Dixon test, temperature measurements were analysed separately for each month. Among 190,967 temperature results, this test identified 7,649 outlier measurements. It is characterised by the highest sensitivity to the presence of outlier measurements in the analysed data sets.

By dint of the quartile test, it was possible to flag the largest number of records in the shallows of the Pomeranian Bay, that is, 259 measurement records for significant wave height and the same number for the mean wave period. In deep water conditions, this test showed 121 records for both significant wave height and mean wave period.

The quartile test was also applied to all temperature measurements at coastal stations, analysing each month separately. This test showed the presence of 931 outlier measurements.

The test was applied to 40,651 temperature results taken between 1959 and 2019 at 18 open sea stations. At the test limit value (9°C), one outlier measurement was identified (station P5, 1965-04-06, level 20). At the same time, it should be noted that this test, in its formula, skips the measurement made at the first and last levels due to the lack of possibility of comparison to the values at adjacent levels. Therefore, it was possible to test 31,590 of the 40,651 total results.

Also, in the case of this test, measures were taken to adapt it to the conditions of the southern Baltic Sea. In the first step, the formula of that test was changed in this way: the measurement from the first level was compared only with the measurement made at the lower level, while the measurement made at the last level was compared with the measurement made at the higher level. In the case of measurements made at intermediate levels, the previous formula was used. The changed test formula is as follows: (7) Measurement at the first levelTest value=|V1-V2|Measurement from the second level-previous fromulaTest value=|V2-V3+V12|Measurement at the last levelTest value= |V3-V2| \matrix{{{\rm{Measurement}}\,{\rm{at}}\,{\rm{the}}\,{\rm{first}}\,{\rm{level}}} \hfill & {{\rm{Test}}\,{\rm{value}} = \left| {V_1 - V_2 } \right|} \hfill \cr {{\rm{Measurement}}\,{\rm{from}}\,{\rm{the}}\,{\rm{second}}\,{\rm{level}} - {\rm{previous}}\,{\rm{fromula}}} \hfill & {{\rm{Test}}\,{\rm{value}} = \left| {V_2 - {{V_3 + V_1 } \over 2}} \right|} \hfill \cr {{\rm{Measurement}}\,{\rm{at}}\,{\rm{the}}\,{\rm{last}}\,{\rm{level}}} \hfill & {{\rm{Test}}\,{\rm{value}} = \,\left| {V_3 - V_2 } \right|} \hfill \cr } where V2

measurement from level n

After calculating the test statistics for the temperature results according to formula (7), the 99th percentile was calculated as 4.7. The test results indicate that 99% of the differences between adjacent temperature measurements are ≤4.7°C. This result was mathematically rounded to 5, and this value was adopted as the test limit value for temperature measurements performed at southern Baltic open sea stations (Table 12). At the same time, the indicated limit value is used to flag suspicious values whose correctness should be subjected to expert assessment.

The second test, with the limit lowered to 5°C, conducted on 40,651 temperature results indicated 304 outlier measurements.

The conducted tests (spike, Dixon, Hampel, and quartile) for significant wave height and wave period in two different locations, representing deep water (Petrobaltic) and shallow water (Pomeranian Bay) conditions, enabled the assessment of the applicability of the analysed tests in the southern Baltic Sea (Fig. 5).

Figure 5

Number of flagged measurement records using the analysed tests at both locations.

A mutual comparison of the convergence of detection and flagging of identical measurement records by all tests for significant wave height (Table 13) and mean wave period (Table 14) allowed for the estimation of the applicability and reliability of the analysed tests. The highest number of overlapping flagged wave height records was the Spike Test and the Hampel Test, which detected only three less measurement records. The Dixon test in both locations (Petrobaltic, Pomeranian Bay) flagged 226 records of significant wave height measurements, of which as many as 117 records were flagged by the quartile test

The Hampel test showed 211 outlying measurements of significant wave height, of which 207 out of 211 also include a quartile test.

The spike test for the mean wave period flagged 484 records, with 377 of the same measurements were flagged by the quartile test The Dixon test, like the Quartile test, contains the most flagged records The Hampel test flagged 211 records, of which the quartile test flagged 209 of the same measurements. Notably, 377 of the same records out of 380 were flagged by the quartile test and the spike test (Table 14).

The example period, December 2018–January 2019, shows (Fig. 6) the identification of records flagged by one or more tests. It should be emphasised that flagging values are not rejected or removed values, but they only narrow the data set and designate data records that will be further subjected to expert evaluation. All extremely low values, where the difference from the previously recorded measurement exceeds 1 m, and in some cases even 5 m, were flagged by four tests: spike, Hampel, Dixon, and quartile. Between January 2 and 4, 2019, some records were flagged by three tests (spike, Hampel, quartile), two tests (spike and quartile), and one test (spike).

Figure 6

Significant wave height measurements with marking of flagged measurements from December 2018 to January 2019 for the Petrobaltic point.

Figure 6 shows the station’s location in the coastal zone. The bar chart indicates the type of test and the number of outlier measurements, while the size of the circle shows the number of measurements made at a given station in the analysed period from 1946 to 2019.

Analysing the above figure, it can be concluded that most outlier measurements were found based on the Hampel test. Based on this test, the number of outlier measurements identified ranges from 695 at the Gdańsk station to 1,277 at the Władysławowo station. The least outlier measurements were found based on the Dixon’s 4σ test, only eight. This test’s results coincided with the Hampel test results only in three cases and with the results of the spike test in two cases. Among the stations presented in Fig. 7, the Kołobrzeg station is noteworthy, located on the central coast, where the largest number of outlier measurements indicated using the spike test were observed with the limit lowered to 2°C. This shows a wide variation in the temperature results measured here. The final data verification stage is an expert assessment of results indicated as outlier measurements. The phenomenon of upwelling should be considered among the factors that may influence the results of temperature measurements measured in the central coast region.

Figure 7

Location of coastal stations taking into account the results of the data verification tests used.

Upwelling is the phenomenon of rising ocean or deep-sea waters, usually from a depth below the thermocline to the surface due to the influence of strong along-shore winds. These winds, according to Ekman’s theory, cause surface water to flow away from the shore and replace it with water from lower layers (Leppäranta and Myrberg, 2009). On the Polish coast, the areas most exposed to the occurrence of upwelling are its central part, particularly around Kołobrzeg, Łeba, and the Hel Peninsula. The temperature difference between the outflow of bottom water masses reaches up to 10°C. This phenomenon is especially visible in summer and occurs in various forms. However, the dominant ones are the so-called filaments (Gurova et al., 2013), which create longitudinal ribbons/tongues with a concentric structure, where the lowest water temperature values characterise the centre of the tongue due to the strongest frontal uplift of deep-sea water masses. This phenomenon is often sudden, causing significant daily fluctuations in water temperature. To assess the impact of upwelling on the water temperature, the measurement value from 1 August 2019 for the Kołobrzeg coastal station (13.3°C), which was flagged with the spike test, the quartile test, and the Hampel test, was analysed in detail. This measurement’s difference compared to the previous day’s value is 3.3°C.

The spatial distribution of water temperature from 31 July to 1 August 2019, based on reanalysis data (Fig. 8), confirms the phenomenon of upwelling in the analysed period.

Figure 8

Surface water temperature from 31 July to 1 August 2019 in the southern Baltic Sea, based on (CMEMS, 2023).

The temperature of the upwelling tongue is approximately 14°C–15°C, which is equal to the temperature of the bottom water (Fig. 9). This indicates the upwelling of deep-sea water masses into higher layers.

Figure 9

Surface water temperature from 31 July to 1 August 2019 in the southern Baltic Sea, based on (CMEMS, 2023).

Such a detailed analysis, therefore, gives grounds to question the mentioned temperature measurement result as an outlier measurement (Table 15).

In the next step, the impact of outliers on the basic statistical values of the analysed set of temperature measurement results at coastal stations (190,967), such as the mean median and standard deviation, was analysed (Table 16).

With such a large data set, identifying and eliminating a relatively small number of outlier measurements from the set do not significantly impact the results of statistical analyses. However, the results of the Hampel test have a noticeable impact on the statistical values.

In the next step, the impact of outlier measurements was analysed for a small data set limited to one month. For this purpose, changes in statistical values in temperature measurements made at the Kołobrzeg station in August 2006 were analysed (Table 17). This is one of many examples where the spike, Hampel, Q-Dixon, and quartile tests indicated the same values as outliers.

By analysing the selected range of measurements for a single station, a clear impact of outlier measurements on the size of statistical analyses can already be observed. In that case, the identification and elimination of outliers have a significant impact, for example, on the standard deviation result. Eliminating a few outliers from the set of analysed values contributes to lowering the standard deviation value, making the set of results more homogeneous, which has a significant impact on monthly, annual, and long-term assessments and analyses.

The chart (Fig. 10) shows the identification of records flagged by one or more tests at the Kołobrzeg station in 2006. It should be emphasised that flagging values are not rejected or removed values, but only narrow the data set and designate data records that will be further subjected to expert evaluation. Outliers were identified using the spike, Hampel, Dixon, and quartile tests. It can be seen that in August, there is a measurement flagged by four tests, while in September and November, by two tests. In the other months, we can see measurements detected by one test.

Figure 10

Temperature measurements with flagged measurements at the Kołobrzeg station in 2006.

The percentage of flagged data varies depending on station location (Fig. 11). The bar diagrams indicate the type of test and the number of outlier measurements. At the same time, the size of the circle corresponds to the number of measurements made at a given station in the analysed period from 1959 to 2019.

Figure 11

Location of open sea stations taking into account the results of the data verification tests used.

In more than half of the analysed open sea stations, outlier measurements were found due to the gradient test using the test limit indicated in the guidelines (9°C). However, the percentage of these measurements is marginal compared to the total number of measurements performed at these stations. It ranges from 1 measurement at stations B13, R4, K, and L7 located in the transition zone to 2 measurements at stations P63, P2, P3, P110, P5, and P1 located mainly in the transitional and deep-water zones. The situation changes when the gradient test limit is lowered from 9°C to 5°C.

As a result of this procedure, the occurrence of outlier measurements is observed in almost all stations in the Polish marine areas of the southern Baltic Sea, except station B15. The largest percentage of outlier measurements related to the number of measurements taken is observed at stations located in the deep-water zone, such as P5, P3, P116, P140, and P63. In the case of open sea stations, it should be that in the summer, the Baltic Sea heats up quite quickly, but the heat is transferred into the water area slowly. As a result, a horizontal thermal stratification is created, where warm waters remain above the cold ones, and a thermocline forms between these waters, a layer where temperature decreases rapidly with depth. This phenomenon may affect the verification test results at a globally set limit value. Also, in the case of open sea stations, the impact of outlier measurements identified by the gradient test on the result of calculating statistical values was analysed (Table 18).

As in the case of coastal stations, eliminating outlier measurements has a small but noticeable impact on the results of statistical analyses.