1. bookVolume 38 (2022): Issue 2 (June 2022)
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
access type Open Access

Data Collection Expert Prior Elicitation in Survey Design: Two Case Studies

Published Online: 14 Jun 2022
Volume & Issue: Volume 38 (2022) - Issue 2 (June 2022)
Page range: 637 - 662
Received: 01 Mar 2021
Accepted: 01 Jan 2022
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
Abstract

Data collection staff involved in sampling designs, monitoring and analysis of surveys often have a good sense of the response rate that can be expected in a survey, even when this survey is new or done at a relatively low frequency. They make expectations of response rates, and, subsequently, costs on an almost continuous basis. Rarely, however, are these expectations formally structured. Furthermore, the expectations usually are point estimates without any assessment of precision or uncertainty.

In recent years, the interest in adaptive survey designs has increased. These designs lean heavily on accurate estimates of response rates and costs. In order to account for inaccurate estimates, a Bayesian analysis of survey design parameters is very sensible.

The combination of strong intrinsic knowledge of data collection staff and a Bayesian analysis is a natural next step. In this article, prior elicitation is developed for design parameters with the help of data collection staff. The elicitation is applied to two case studies in which surveys underwent a major redesign and direct historic survey data was unavailable.

Keywords

Bethlehem, J., F. Cobben, and B. Schouten. 2011. Handbook of Nonresponse in Household Surveys. John Wiley and Sons.10.1002/9780470891056 Search in Google Scholar

Burger, J., K. Perryck, and B. Schouten. 2017. “Robustness of adaptive survey designs to inaccuracy of design parameters.” Journal of Official Statistics 33(3): 687–708. DOI: https://doi.org/10.1515/JOS-2017-0032.10.1515/jos-2017-0032 Search in Google Scholar

Brownstein, N.C., T.A. Louis, A. O’Hagan, and J. Pendergast. 2019. “The role of expert judgment in statistical inference and evidence-based decision-making.” The American Statistician 73(1): 56–68. DOI: https://doi.org/10.1080/00031305.2018.1529623.10.1080/00031305.2018.1529623647472531057338 Search in Google Scholar

Coffey, S., B.T. West, J. Wagner, and M.R. Elliott. 2020. “What do you think? Using expert opinion to improve predictions of response propensity under a bayesian framework.” Methods, Data, Analyses 14(2). DOI: https://doi.org/10.12758/mda.2020.05. Search in Google Scholar

Gelman, A., H.S. Stern, J.B. Carlin, D.B. Dunson, A. Vehtari, and D.. Rubin. 2013. Bayesian data analysis. Chapman and Hall CRC.10.1201/b16018 Search in Google Scholar

Gosling, J.P., J.E. Oakley, and A. O’Hagan. 2007. “Nonparametric elicitation for heavy-tailed prior distributions.” Bayesian Analysis 2(4): 693–718. DOI: https://doi.org/10.1214/07-BA228.10.1214/07-BA228 Search in Google Scholar

Ibrahim, J.G., and M.H. Chen. 2000. “Power prior distributions for regression models.” Statistical Science: 40–60. DOI: https://doi.org/10.1214/ss/1009212673.10.1214/ss/1009212673 Search in Google Scholar

Ibrahim, J.G., M.H. Chen, Y. Gwon, and F. Chen. 2015. “The power prior: Theory and applications.” Statistics in Medicine 34(28): 3724–3749. DOI: https://doi.org/10.1002/-sim.6728. Search in Google Scholar

Kreuter, F. 2013. “Facing the Nonresponse Challenge.” Annals of the American Academy of Political and Social Science 645(1): 23–35. DOI: https://doi.org/10.1177/0002716212456815.10.1177/0002716212456815 Search in Google Scholar

Moore, J.C., G.. Durrant, and P.W.F. Smith. 2018. “Data set representativeness during data collection in three UK social surveys: generalizability and the effects of auxiliary covariate choice.” Journal of the Royal Statistical Society. 181(1): 229–248. DOI: https://doi.org/10.1111/rssa.12256.10.1111/rssa.12256 Search in Google Scholar

O’Hagan, A., C.E. Buck, A. Daneshkhah, J.R. Eiser, P.H. Garthwaite, D.J. Jenkinson, J.E. Oakley, and T. Rakow. 2006. Uncertain judgements: Eliciting experts’ probabilities. New York: Wiley.10.1002/0470033312 Search in Google Scholar

Oakley, J.E., and A. O’Hagan. 2007. “Uncertainty in prior elicitations: A nonparametric approach.” Biometrika 94(2): 427–441. DOI: https://doi.org/10.1093/biomet/asm031.10.1093/biomet/asm031 Search in Google Scholar

Rietbergen, C., I. Klugkist, K.J.M. Janssen, K.G.M. Moons, and H.J.A. Hoijtink. 2011. “Incorporation of historical data in the analysis of randomized therapeutic trials.” Contemporary Clinical Trials 32(6): 848–855. DOI: https://doi.org/10.1016/j.cct.2011.06.002.10.1016/j.cct.2011.06.00221729767 Search in Google Scholar

Schouten, B., F. Cobben, and J. Bethlehem. 2009. “Indicators for the representativeness of survey response.” Survey Methodology 35(1): 101–113. Available at: https://www150.statcan.gc.ca/n1/pub/11-522-x/2008000/article/10976-eng.pdf. Search in Google Scholar

Schouten, B., N. Mushkudiani, N. Shlomo, G. Durrant, P. Lundquist, and J. Wagner. 2018. “A Bayesian analysis of design parameters in survey data collection.” Journal of Survey Statistics and Methodology 6(4): 431-464. DOI: https://doi.org/10.1093/jssam/smy012.10.1093/jssam/smy012 Search in Google Scholar

Schouten, B., A. Peytchev, and J. Wagner. 2017. Adaptive survey design. CRC Press.10.1201/9781315153964 Search in Google Scholar

Schouten, B., and N. Shlomo. 2017. “Selecting Adaptive Survey Design Strata with Partial R-indicators.” International Statistical Review 85(1): 143–163. DOI: https://doi.org/10.1111/insr.12159.10.1111/insr.12159 Search in Google Scholar

Veen, D., D. Stoel, M. Zondervan-wijnenburg, and R. van de Schoot. 2017. “Proposal for a five-step method to elicit expert judgment.” Frontiers in Psychology 8: 2110. DOI: https://doi.org/10.3389/fpsyg.2017.02110.10.3389/fpsyg.2017.02110572334029259569 Search in Google Scholar

West, B.T., J. Wagner, S. Coffey, and M.R. Elliott. 2021. “Deriving priors for Bayesian prediction of daily response propensity in responsive survey design: historical data analysis vs. literature review”. Journal of Survey Statistics and Methodology. DOI: https://doi.org/10.1093/jssam/smab036.10.1093/jssam/smab036 Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo