<abstract xmlns="http://www.w3.org/1999/xhtml">

<sec>
<h3>Purpose</h3>
<p>The purpose of this study is to develop and compare model choice strategies in context of logistic regression. Model choice means the choice of the covariates to be included in the model.</p>
</sec>
<sec>
<h3>Design/methodology/approach</h3>
<p>The study is based on Monte Carlo simulations. The methods are compared in terms of three measures of accuracy: specificity and two kinds of sensitivity. A loss function combining sensitivity and specificity is introduced and used for a final comparison.</p>
</sec>
<sec>
<h3>Findings</h3>
<p>The choice of method depends on how much the users emphasize sensitivity against specificity. It also depends on the sample size. For a typical logistic regression setting with a moderate sample size and a small to moderate effect size, either BIC, BICc or Lasso seems to be optimal.</p>
</sec>
<sec>
<h3>Research limitations</h3>
<p>Numerical simulations cannot cover the whole range of data-generating processes occurring with real-world data. Thus, more simulations are needed.</p>
</sec>
<sec>
<h3>Practical implications</h3>
<p>Researchers can refer to these results if they believe that their data-generating process is somewhat similar to some of the scenarios presented in this paper. Alternatively, they could run their own simulations and calculate the loss function.</p>
</sec>
<sec>
<h3>Originality/value</h3>
<p>This is a systematic comparison of model choice algorithms and heuristics in context of logistic regression. The distinction between two types of sensitivity and a comparison based on a loss function are methodological novelties.</p>
</sec>
</abstract>

Purpose
The purpose of this study is to develop and compare model choice strategies in context of logistic regression. Model choice means the choice of the covariates to be included in the model.


Design/methodology/approach
The study is based on Monte Carlo simulations. The methods are compared in terms of three measures of accuracy: specificity and two kinds of sensitivity. A loss function combining sensitivity and specificity is introduced and used for a final comparison.


Findings
The choice of method depends on how much the users emphasize sensitivity against specificity. It also depends on the sample size. For a typical logistic regression setting with a moderate sample size and a small to moderate effect size, either BIC, BICc or Lasso seems to be optimal.


Research limitations
Numerical simulations cannot cover the whole range of data-generating processes occurring with real-world data. Thus, more simulations are needed.


Practical implications
Researchers can refer to these results if they believe that their data-generating process is somewhat similar to some of the scenarios presented in this paper. Alternatively, they could run their own simulations and calculate the loss function.


Originality/value
This is a systematic comparison of model choice algorithms and heuristics in context of logistic regression. The distinction between two types of sensitivity and a comparison based on a loss function are methodological novelties.

A comparison of model choice strategies for logistic regression

Journal of Data and Information Science

This work is licensed under the Creative Commons Attribution 4.0 International License.

{"article-title":"A comparison of model choice strategies for logistic regression"}

Purpose
The purpose of this study is to develop and compare model choice strategies in context of logistic regression. Model choice means the choice of the...

A comparison of model choice strategies for logistic regression

Article Category: Research Papers

Published Online: Feb 06, 2024

Page range: 37 - 52

Received: Oct 11, 2023

Accepted: Dec 22, 2023

DOI: https://doi.org/10.2478/jdis-2024-0001

Keywords
Model choice, Logistic regression, Logit regression, Monte Carlo simulations, Sensitivity, Specificity

© 2024 Markku Karhunen, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

A comparison of model choice strategies for logistic regression

Article Category: Research Papers

Published Online: Feb 06, 2024

Page range: 37 - 52

Received: Oct 11, 2023

Accepted: Dec 22, 2023

DOI: https://doi.org/10.2478/jdis-2024-0001

KeywordsModel choice, Logistic regression, Logit regression, Monte Carlo simulations, Sensitivity, Specificity

© 2024 Markku Karhunen, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Model choice, Logistic regression, Logit regression, Monte Carlo simulations, Sensitivity, Specificity