1. bookVolume 37 (2021): Issue 4 (December 2021)
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
access type Open Access

Evaluating the Utility of Linked Administrative Data for Nonresponse Bias Adjustment in a Piggyback Longitudinal Survey

Published Online: 26 Dec 2021
Page range: 837 - 864
Received: 01 Oct 2020
Accepted: 01 May 2021
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
Abstract

Nearly all panel surveys suffer from unit nonresponse and the risk of nonresponse bias. Just as the analytic value of panel surveys increase with their length, so does cumulative attrition, which can adversely affect the representativeness of the resulting survey estimates. Auxiliary data can be useful for monitoring and adjusting for attrition bias, but traditional auxiliary sources have known limitations. We investigate the utility of linked-administrative data to adjust for attrition bias in a standard piggyback longitudinal design, where respondents from a preceding general population cross-sectional survey, which included a data linkage request, were recruited for a subsequent longitudinal survey. Using the linked-administrative data from the preceding survey, we estimate attrition biases for the first eight study waves of the longitudinal survey and investigate whether an augmented weighting scheme that incorporates the linked-administrative data reduces attrition biases. We find that adding the administrative information to the weighting scheme generally leads to a modest reduction in attrition bias compared to a standard weighting procedure and, in some cases, reduces variation in the point estimates. We conclude with a discussion of these results and remark on the practical implications of incorporating linked-administrative data in piggyback longitudinal designs.

Keywords

American Association for Public Opinion Research. 2016. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys, (9th edition). Lanexa: AAPOR. Available at: https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf (accessed October 2021). Search in Google Scholar

Antoni, M., and A. Bethmann. 2019. “PASS-ADIAB – Linked Survey and Administrative Data for Research on Unemployment and Poverty.” Journal of Economics and Statistics 239(4): 747–756. DOI: https://doi.org/10.1515/jbnst-2018-0002.10.1515/jbnst-2018-0002 Search in Google Scholar

Antoni, M., K. Drasch, C. Kleinert, B. Matthes, M. Ruland, and A. Trahms. 2010. Working and Learning in a Changing World, part I: Overview of the study, FDZ Methodenreport No. 5/2010 (en). Nürnberg: Forschungsdatenzentrum (FDZ) der Bundesagentur für Arbeit im Institut für Arbeitsmarkt- und Berufsforschung (IAB). Available at: http://doku.iab.de/fdz/reporte/2010/MR_05-10_EN.pdf (accessed October 2021). Search in Google Scholar

Antoni, M., and S. Seth. 2011. ALWA-ADIAB – Linked individual Survey and Administrative Data for Substantive and Methodological Research, FDZ Methodenre-port No. 12/2011 (en). Nürnberg: Forschungsdatenzentrum (FDZ) der Bundesagentur für Arbeit im Institut für Arbeitsmarkt- und Berufsforschung (IAB). Available at: http://doku.iab.de/fdz/reporte/2011/MR_12-11_EN.pdf (accessed October 2021). Search in Google Scholar

Bee, C.A., G.M.R. Gathright, and B.D. Meyer. 2015. “Bias from Unit Non-Response in the Measurement of Income in Household Surveys.” Paper presented at the Joint Statistical Meetings of the American Statistical Association, August 9, 2015, Seattle, USA. Available at: http://www.solejole.org/16068.pdf (accessed February 2020). Search in Google Scholar

Behrens, K., and K. Wiese. 2019. “BIK-Regionen.” In Regionale Standards: Ausgabe 2019, edited by K. Beckmann, K. Behrens, H. Hoffmann, M. Pfister, K. Wiese, J.H.P. Hoffmeyer-Zlotnik, G. Rösch, P. Siegers, W. Sodeur, U. Hanefeld, R. Herter-Eschweiler, and E. Krack-Roberg: 114–126. Cologne, Germany: GESIS – Leibniz-Institut für Sozialwissenschaften. Search in Google Scholar

Blossfeld, H.P., H.G. Roßbach, and J. von Maurice (Eds.). 2011. Education as a Lifelong Process – The German National Educational Panel Study (NEPS), Zeitschrift für Erziehungswissenschaft: Sonderheft 14. Wiesbaden: Springer VS. Search in Google Scholar

Burgette, L.F., and J.P. Reiter. 2010. “Multiple Imputation for Missing Data via Sequential Regression Trees.” American Journal of Epidemiology 172(9): 1070–1076. DOI: https://doi.org/10.1093/aje/kwq260.10.1093/aje/kwq260 Search in Google Scholar

Burkam, D.T., and V.E. Lee. 1998. “Effects of Monotone and Nonmonotone Attrition on Parameter Estimates in Regression Models with Educational Data.” The Journal of Human Resources 33(2): 555–574. DOI: https://doi.org/10.2307/146441.10.2307/146441 Search in Google Scholar

Calderwood, L., and C. Lessof. 2009. “Enhancing Longitudinal Surveys by Linking to Administrative Data.” In Methodology of Longitudinal Surveys, edited by P. Lynn: 55–72. West Sussex, England: John Wiley & Sons.10.1002/9780470743874.ch4 Search in Google Scholar

Cheshire, H., D. Hussey, J. Medina, K. Pickering, N. Wood, K. Ward, K. Taylor, and C. Lessof. 2012. Financial circumstances, health and well-being of the older population in England: The 2008 English Longitudinal Study of Ageing: Wave 4 Technical Report. London: National Centre for Social Research. Available at: https://ifs.org.uk/elsa/report10/w4_tech.pdf (accessed October 2021). Search in Google Scholar

Christen. 2012. Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. Berlin, Germany: Springer. DOI: https://doi.org/10.1007/978-3-642-31164-2_2.10.1007/978-3-642-31164-2_2 Search in Google Scholar

Cohen, S.B. 2005. “Integrated survey designs: A framework for nonresponse bias reduction.” Journal of Economic and Social Measurement 30(2–3): 101–114. DOI: https://doi.org/10.3233/JEM-2005-0244.10.3233/JEM-2005-0244 Search in Google Scholar

Couper, M.P., and M.B. Ofstedal. 2009. “Keeping in Contact with Mobile Sample Members.” In Methodology of Longitudinal Surveys, edited by P. Lynn: 183–203. West Sussex, England: John Wiley & Sons.10.1002/9780470743874.ch11 Search in Google Scholar

Couper, M.P. 2017. “New Developments in Survey Data Collection.” Annual Review of Sociology 43(1): 121–145. DOI: https://doi.org/10.1146/annurev-soc-060116-053613.10.1146/annurev-soc-060116-053613 Search in Google Scholar

Davern, M.E., B.D. Meyer, and N.K. Mittag. 2019. “Creating Improved Survey Data Products Using Linked Administrative-Survey Data.” Journal of Survey Statistics and Methodology 7(3): 440–463. DOI: https://doi.org/10.1093/jssam/smy017.10.1093/jssam/smy017 Search in Google Scholar

Edwards, B., L. Branden, and M. Stange. 2011. “Piggyback Survey Respondents and Mode: Lessons Learned from Design and Operations.” In JSM Proceedings, Survey Research Methods Section, August 1, 2011: 5009–5021. Alexandria, VA: American Statistical Association. Available at: http://www.asasrms.org/Proceedings/y2011/Files/302992_69483.pdf (accessed October 2021). Search in Google Scholar

Ezzati-Rice, T.M., F. Rohde, and J. Greenblatt. 2008. Sample Design of the Medical Expenditure Panel Survey Household Component, 1998–2007. Methodology Report No. 22. March 2008. Rockville, MD, USA: Agency for Healthcare Research and Quality. Available at: http://www.meps.ahrq.gov/mepsweb/data_files/publications/mr22/mr22.shtml (accessed February 2020). Search in Google Scholar

Freedman, V.A., K. McGonagle, and P. Andreski. 2014. The Panel Study of Income Dynamics: Linked Medicare Claims Data. PSID Technical Series Paper No. 14-01. University of Michigan. Available at: https://psidonline.isr.umich.edu/publications/Papers/tsp/2014-01_PSIDMedicare.pdf (accessed October 2021). Search in Google Scholar

Groves, R., F.J. Fowler, M.P. Couper, J.M. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey Methodology, (2nd ed.). Hoboken, NJ: John Wiley & Sons. Search in Google Scholar

Hammon, A., S. Zinn, C. Aßmann, and A. Würbach. 2016. Samples, Weights, and Nonresponse: The Adult cohort of the National Educational Panel Study (Wave 2 to 6). Bamberg, Germany: Leibniz Institute for Educational Trajectories, National Educational Panel Study. NEPS Survey Paper No. 7. DOI: https://doi.org/10.5157/NEPS:SP07:1.0. Search in Google Scholar

Hammon, A. 2018. Samples, Weights, and Nonresponse: The Adult Cohort of the National Educational Panel Study (Wave 7 to 9). Bamberg, Germany: Leibniz Institute for Educational Trajectories, National Educational Panel Study. Available at: https://www.neps-data.de/Portals/0/NEPS/Datenzentrum/Forschungsdaten/SC6/9-0-0/SC6_9-0-0_W.pdf (accessed October 2021). Search in Google Scholar

Hoonhout, P., and G. Ridder. 2019. “Nonignorable Attrition in Multi-Period Panels With Refreshment Samples.” Journal of Business & Economic Statistics 37(3): 377–390. DOI: https://doi.org/10.1080/07350015.2017.1345744.10.1080/07350015.2017.1345744 Search in Google Scholar

Jacobebbinghaus, P., and S. Seth. 2007. “The German Integrated Employment Biographies Sample IEBS.” Schmollers Jahrbuch: Journal of Applied Social Science Studies 127: 335 – 342. Available at: https://www.ratswd.de/download/schmollers/2007_127/Schmollers_2007_2_S335.pdf (accessed October 2021). Search in Google Scholar

Korbmacher, J., and C. Czaplicki. 2013. “Linking SHARE Survey Data with Administrative Records: First Experiences from SHARE-Germany.” In SHARE Wave 4: Innovations & Methodology, edited by F. Malter and A. Börsch-Supan: 47–53. Munich: MEA, Max Planck Institute for Social Law and Social Policy. Search in Google Scholar

Knies, G., J. Burton, and E. Sala. 2012. “Consenting to Health Record Linkage: Evidence from a Multipurpose Longitudinal Survey of a General Population.” BMC Health Services Research 12(52). DOI: https://doi.org/10.1186/1472-6963-12-52.10.1186/1472-6963-12-52 Search in Google Scholar

Knies, G., and J. Burton. 2014. “Analysis of four studies in a comparative framework reveals: health linkage consent rates on British cohort studies higher than on UK household panel surveys.” BMC Medical Research Methodology 14(125). DOI: https://doi.org/10.1186/1471-2288-14-125.10.1186/1471-2288-14-125 Search in Google Scholar

Kreuter, F., and U. Kohler. 2009. “Analyzing Contact Sequences in Call Record Data: Potential and Limitations of Sequence Indicators for Nonresponse Adjustments in the European Social Survey.” Journal of Official Statistics 25(2): 203–226. Search in Google Scholar

Kreuter, F., G. Müller, and M. Trappmann. 2010a. “Nonresponse and Measurement Error in Employment Research: Making Use of Administrative Data.” Public Opinion Quarterly 74: 880–906. DOI: https://doi.org/10.1093/poq/nfq060.10.1093/poq/nfq060 Search in Google Scholar

Kreuter, F., K. Olson, J. Wagner, T. Yan, T.M. Ezzati-Rice, C. Casas-Cordero, M. Lemay, A. Peytchev, R.M. Groves, and T.E. Raghunathan. 2010b. “Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Nonresponse: Examples from Multiple Surveys.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 173: 389–407. DOI: https://doi.org/10.1111/j.1467-985X.2009.00621.x.10.1111/j.1467-985X.2009.00621.x Search in Google Scholar

Kreuter, F. 2013. Improving Surveys with Paradata: Analytic Uses of Process Information. Hoboken, NJ: Wiley.10.1002/9781118596869 Search in Google Scholar

Kroh, M. 2010. DIW Documentation of Sample Sizes and Panel Attrition in the German Socio Economic Panel (SOEP) (1984 until 2009). DIW Data Documentation 50. Berlin: German Institute for Economic Research. Available at: https://www.diw.de/documents/publikationen/73/diw_01.c.359697.de/diw_datadoc_2010-050.pdf (accessed October 2021). Search in Google Scholar

Lemay, M. 2009. Understanding the Mechanisms of Panel Attrition. PhD Dissertation. University of Maryland, College Park. Available at: http://hdl.handle.net/1903/9631 (accessed October 2021). Search in Google Scholar

Lepkowski, J.M., and M.C. Couper. 2002. “Nonresponse in the Second Wave of Longitudinal Household Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little: 259–272. New York, NY: John Wiley & Sons. Search in Google Scholar

Lin, I., and N.C. Schaeffer. 1995. “Using Survey Participants to Estimate the Impact of Nonparticipation.” Public Opinion Quarterly 59: 236–258. DOI: https://doi.org/10.1086/269471.10.1086/269471 Search in Google Scholar

Little, R.J.A., and S. Vartivarian. 2005. “Does Weighting for Nonresponse Increase the Variance of Survey Means?” Survey Methodology 31: 161–168. Available at: https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2005002/article/9046-eng.pdf?st=3epbdduy (accessed October 2021). Search in Google Scholar

Meyer, B.D., and N. Mittag. 2019. “Using Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness, and Holes in the Safety Net.” American Economic Journal: Applied Economics 11(2): 176–204. DOI: https://doi.org/10.1257/app.20170478.10.1257/app.20170478 Search in Google Scholar

Neukirch, T. 2002. “Nonignorable Attrition and Selectivity Biases in the Finnish Subsample of the ECHP: an Empirical Study Using Additional Register Information.” Chintex Working Paper 5. Frankfurt a. M.: J.W. Goethe-Universität. Available at: http://hdl.handle.net/10068/92455 (accessed October 2021). Search in Google Scholar

Olsen, R. 2018. “Panel Attrition.” In The Palgrave Handbook of Survey Research, edited by D. Vannette and J. Krosnick: 509–517. DOI: https://doi.org/10.1007/978-3-319-54395-6_59.10.1007/978-3-319-54395-6_59 Search in Google Scholar

Olson, J.A. 1999. “Linkages with Data from Social Security Administrative Records in the Health and Retirement Study.” Social Security Bulletin 62(2): 73–85. Available at: https://www.ssa.gov/policy/docs/ssb/v62n2/v62n2p73.pdf (accessed October 2021). Search in Google Scholar

Pankowska, P., B. Bakker, D.L. Oberski, and D. Pavlopoulos. 2018. “Reconciliation of two data sources by correction for measurement error: the feasibility of parameter reuse.” Statistical Journal of the IAOS 34(3): 317–329. DOI: https://doi.org/10.3233/SJI-170368.10.3233/SJI-170368 Search in Google Scholar

Pavlopoulos, D., and J.K. Vermunt. 2015. “Measuring temporary employment. Do survey or register data tell the truth?” Survey Methodology 41(1): 197–214. Available at: https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2015001/article/14151-eng.pdf?st=tAsC-b20 (accessed October 2021). Search in Google Scholar

Sakshaug, J.W., M. Antoni, and R. Sauckel. 2017. “The Quality and Selectivity of Linking Federal Administrative Records to Respondents and Nonrespondents in a General Population Survey in Germany.” Survey Research Methods 1: 63–80. DOI: https://doi.org/10.18148/srm/2017.v11i1.6718. Search in Google Scholar

Sakshaug, J.W., and M. Antoni. 2019. “Evaluating the Utility of Indirectly Linked Federal Administrative Records for Nonresponse Bias Adjustment.” Journal of Survey Statistics and Methodology 7: 227–249. DOI: https://doi.org/10.1093/jssam/smy009.10.1093/jssam/smy009 Search in Google Scholar

Sakshaug, J.W., and M. Huber. 2016. “An Evaluation of Panel Nonresponse and Linkage Consent Bias in a Survey of Employees in Germany.” Journal of Survey Statistics and Methodology 4(1): 71–93. DOI: https://doi.org/10.1093/jssam/smv034.10.1093/jssam/smv034 Search in Google Scholar

Sakshaug, J.W., and F. Kreuter. 2012. “Assessing the Magnitude of Non-Consent Biases in Linked Survey and Administrative Data.” Survey Research Methods 6(2): 113–122. DOI: https://doi.org/10.18148/srm/2012.v6i2.5094. Search in Google Scholar

Schaurer, I., and K. Weyandt. 2018. GESIS Panel Technical Report: Recruitment 2016 (Wave d11 and d12). Cologne, Germany: GESIS – Leibniz-Institut für Sozialwissenschaften. Search in Google Scholar

Scholz, J., A. Seshadri, and S. Khitatrakun. 2006. “Are Americans Saving ‘Optimally’ for Retirement?” Journal of Political Economy 114: 607–643. DOI: https://doi.org/10.1086/506335.10.1086/506335 Search in Google Scholar

Short, K., and E. McArthur. 1986. “Life Events and Sample Attrition in the Survey of Income and Program Participation.” In Proceedings of the Survey Research Methods Section of the American Statistical Association: 200–205. Available at: http://www.asasrms.org/Proceedings/y1986f.html (accessed October 2021). Search in Google Scholar

Sin, C.H. 2006. “The feasibility of using national surveys to derive samples of older people from different ethnic groups in Britain: Lessons from ‘piggy-backing’ on the Family Resources Survey.” International Journal of Social Research Methodology 9(1): 15–28. DOI: https://doi.org/10.1080/13645570500435264.10.1080/13645570500435264 Search in Google Scholar

Sinibaldi, J., M. Trappmann, and F. Kreuter. 2014. “Which is the better investment for nonresponse adjustment: Purchasing commercial auxiliary data or collecting interviewer observations?” Public Opinion Quarterly 78(2): 440–473. DOI: https://doi.org/10.1093/poq/nfu003.10.1093/poq/nfu003 Search in Google Scholar

Smith, T.W. 2011. “The Report of the International Workshop on Using Multi-Level Data from Sample Frames, Auxiliary Databases, Paradata and Related Sources to Detect and Adjust for Nonresponse Bias in Surveys.” International Journal of Public Opinion Research 23: 389–402. DOI: https://doi.org/10.1093/ijpor/edr035.10.1093/ijpor/edr035 Search in Google Scholar

Smith, T.W., and J. Kim. 2013. “An Assessment of the Multi-Level Integrated Database Approach.” The Annals of the American Academy of Political and Social Science 645: 185–221. DOI: https://doi.org/10.1177/0002716212463340.10.1177/0002716212463340 Search in Google Scholar

Spieß, M., and M. Kroh. 2004. “Documentation of Sample Sizes and Panel Attrition in the German Socio Economic Panel (GSOEP).” Research Note No. 28a. Berlin, Germany: German Institute for Economic Research (DIW). Available at: https://www.diw.de/documents/publikationen/73/diw_01.c.43407.de/diw_datadoc_2004-001.pdf (accessed October 2021). Search in Google Scholar

StataCorp. 2017. Stata 15 Base Reference Manual. College Station, TX: Stata Press. Search in Google Scholar

Stöckinger, C., S. Kretschmer, and C. Kleinert. 2018. Panel Attrition in NEPS Starting Cohort 6: A Description of Attrition Processes in Waves 2 to 7 with Regard to Nonresponse Bias, NEPS Survey Paper No. 35. Bamberg, Germany: Leibniz Institute for Educational Trajectories, National Educational Panel Study. Search in Google Scholar

Taylor, M.F., Brice, J., Buck, N., and Prentice-Lane, E. (Eds.). 2010. British Household Panel Survey User Manual Volume A: Introduction, Technical Report and Appendices. Colchester: University of Essex. Search in Google Scholar

Taylor, R., L. Conway, L. Calderwood, C. Lessof, H. Cheshire, K. Cox, and S. Scholes. 2007. Health, wealth and lifestyles of the older population in England: The 2002 English Longitudinal Study of Ageing: Technical Report. London: Institute of Fiscal Studies. Available at: https://ifs.org.uk/elsa/report03/w1_tech.pdf (accessed October 2021). Search in Google Scholar

Trappmann, M., T. Gramlich, and A. Mosthaf. 2015. “The Effect of Events Between Waves on Panel Attrition.” Survey Research Methods 9(1): 31–43. DOI: https://doi.org/10.18148/srm/2015.v9i1.5849. Search in Google Scholar

UNECE (United Nations Economic Commission for Europe). 2007. Register-Based Statistics in the Nordic Countries: Review of Best Practices with Focus on Population and Social Statistics. Technical Report E.07.II.E.11. New York/Geneva: United Nations. Available at: http://www.unece.org/fileadmin/DAM/stats/publications/Register_based_statistics_in_Nordic_countries.pdf (accessed February 2020). Search in Google Scholar

Valliant, R., J.A. Dever, and F. Kreuter. 2013. Practical Tools for Designing and Weighting Survey Samples. New York: Springer. DOI: https://doi.org/10.1007/978-3-319-93632-1_13.10.1007/978-3-319-93632-1_13 Search in Google Scholar

Watson, N., and M. Wooden. 2009. “Identifying Factors Affecting Longitudinal Survey Response.” In Methodology of Longitudinal Surveys, edited by P. Lynn: 157–181. West Sussex, England: John Wiley & Sons.10.1002/9780470743874.ch10 Search in Google Scholar

Weinert, S., C. Artelt, M. Prenzel, M. Senkbeil, T. Ehmke, C.H. Carstensen, and K. Lockl. 2019. “Development of Competencies Across the Life Course.” In Education as a Lifelong Process: The German National Educational Panel Study (NEPS), Edition ZfE Vol. 3, edited by H.P. Blossfeld and H.G. Roßbach: 57–81. Wiesbaden: Springer VS. DOI: https://doi.org/10.1007/978-3-658-23162-0.10.1007/978-3-658-23162-0 Search in Google Scholar

West, B.T., F. Kreuter, and M. Trappmann. 2014. “Is the Collection of Interviewer Observations Worthwhile in an Economic Panel Survey? New Evidence from the German Labor Market and Social Security (PASS) Study.” Journal of Survey Statistics and Methodology 2: 159–181. DOI: https://doi.org/10.1093/jssam/smu002.10.1093/jssam/smu002 Search in Google Scholar

West, B.T., J. Wagner, F. Hubbard, and H. Gu. 2015. “The Utility of Alternative Commercial Data Sources for Survey Operations and Estimation: Evidence from the National Survey of Family Growth.” Journal of Survey Statistics and Methodology 3: 240–264. DOI: https://doi.org/10.1093/jssam/smv004.10.1093/jssam/smv004 Search in Google Scholar

Young, A.F., A.J. Dobson, and J.E. Byles. 2001. “Health Services Research Using Linked Records: Who Consents and What is the Gain?” Australian and New Zealand Journal of Public Health 25(5): 417–420. DOI: https://doi.org/10.1111/j.1467-842X.2001.tb00284.x.10.1111/j.1467-842X.2001.tb00284.x Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo