<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>National statistical institutes currently investigate how to improve the output quality of official statistics based on machine learning algorithms. A key issue is concept drift, that is, when the joint distribution of independent variables and a dependent (categorical) variable changes over time. Under concept drift, a statistical model requires regular updating to prevent it from becoming biased. However, updating a model asks for additional data, which are not always available. An alternative is to reduce the bias by means of bias correction methods. In the article, we focus on estimating the proportion (base rate) of a category of interest and we compare two popular bias correction methods: the misclassification estimator and the calibration estimator. For prior probability shift (a specific type of concept drift), we investigate the two methods analytically as well as numerically. Our analytical results are expressions for the bias and variance of both methods. As numerical result, we present a decision boundary for the relative performance of the two methods. Our results provide a better understanding of the effect of prior probability shift on output quality. Consequently, we may recommend a novel approach on how to use machine learning algorithms in the context of official statistics.</p>
</abstract>

National statistical institutes currently investigate how to improve the output quality of official statistics based on machine learning algorithms. A key issue is concept drift, that is, when the joint distribution of independent variables and a dependent (categorical) variable changes over time. Under concept drift, a statistical model requires regular updating to prevent it from becoming biased. However, updating a model asks for additional data, which are not always available. An alternative is to reduce the bias by means of bias correction methods. In the article, we focus on estimating the proportion (base rate) of a category of interest and we compare two popular bias correction methods: the misclassification estimator and the calibration estimator. For prior probability shift (a specific type of concept drift), we investigate the two methods analytically as well as numerically. Our analytical results are expressions for the bias and variance of both methods. As numerical result, we present a decision boundary for the relative performance of the two methods. Our results provide a better understanding of the effect of prior probability shift on output quality. Consequently, we may recommend a novel approach on how to use machine learning algorithms in the context of official statistics.

Improving the Output Quality of Official Statistics Based on Machine Learning Algorithms

Journal of Official Statistics

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

National statistical institutes currently investigate how to improve the output quality of official statistics based on machine learning algorithms. A key...

Improving the Output Quality of Official Statistics Based on Machine Learning Algorithms

Publié en ligne: 14 juin 2022

Pages: 485 - 508

Reçu: 01 déc. 2020

Accepté: 01 juin 2021

DOI: https://doi.org/10.2478/jos-2022-0023

Mots clés
Output quality, concept drift, prior probability shift, misclassification bias

© 2022 Q.A. Meertens et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Improving the Output Quality of Official Statistics Based on Machine Learning Algorithms

Publié en ligne: 14 juin 2022

Pages: 485 - 508

Reçu: 01 déc. 2020

Accepté: 01 juin 2021

DOI: https://doi.org/10.2478/jos-2022-0023

Mots clésOutput quality, concept drift, prior probability shift, misclassification bias

© 2022 Q.A. Meertens et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Mots clés
Output quality, concept drift, prior probability shift, misclassification bias