A comparison of model choice strategies for logistic regression
Markku Karhunen
Orcid profile 
Article Category: Research Papers
Published Online: Feb 06, 2024
Page range: 37 - 52
Received: Oct 11, 2023
Accepted: Dec 22, 2023
DOI: https://doi.org/10.2478/jdis-2024-0001
Keywords
© 2024 Markku Karhunen, published by Sciendo
This work is licensed under the Creative Commons Attribution 4.0 International License.
Purpose
The purpose of this study is to develop and compare model choice strategies in context of logistic regression. Model choice means the choice of the covariates to be included in the model.
Design/methodology/approach
The study is based on Monte Carlo simulations. The methods are compared in terms of three measures of accuracy: specificity and two kinds of sensitivity. A loss function combining sensitivity and specificity is introduced and used for a final comparison.
Findings
The choice of method depends on how much the users emphasize sensitivity against specificity. It also depends on the sample size. For a typical logistic regression setting with a moderate sample size and a small to moderate effect size, either BIC, BICc or Lasso seems to be optimal.
Research limitations
Numerical simulations cannot cover the whole range of data-generating processes occurring with real-world data. Thus, more simulations are needed.
Practical implications
Researchers can refer to these results if they believe that their data-generating process is somewhat similar to some of the scenarios presented in this paper. Alternatively, they could run their own simulations and calculate the loss function.
Originality/value
This is a systematic comparison of model choice algorithms and heuristics in context of logistic regression. The distinction between two types of sensitivity and a comparison based on a loss function are methodological novelties.