Estimating Components of Mean Squared Error to Evaluate the Benefits of Mixing Data Collection Modes

Biemer, P.P. 1988. “Measuring Data Quality.” In Telephone Survey Methodology, edited by R.M. Groves, P.P. Biemer, L. Lyberg, J.T. Massey, W. II, Nicholls, and J. Waksberg, 341–375. New York: Wiley.Search in Google Scholar

Biemer, P.P. 2010. “Total Survey Error: Design, Implementation, and Evaluation.” Public Opinion Quarterly 74: 817–848. Doi: http://dx.doi.org/10.1093/poq/nfq058.10.1093/poq/nfq058Search in Google Scholar

Biemer, P.P. and L.E. Lyberg. 2003. Introduction to Survey Quality. Hoboken, NJ: Wiley.10.1002/0471458740Search in Google Scholar

Blumberg, S.J. and J.V. Luke. 2013. “Wireless Substitution: Early Release of Estimates from the National Health Interview Survey, July–December 2012.” National Center for Health Statistics. December. Available at: http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201306.pdf (accessed March 2017).Search in Google Scholar

Brick, J.M. and D. Williams. 2013. “Explaining Rising Nonresponse Rates in Cross-Sectional Surveys.” The ANNALS of the American Academy of Political and Social Science 645: 36–59. Doi: http://dx.doi.org/10.1177/0002716212456834.10.1177/0002716212456834Search in Google Scholar

Carley-Baxter, L.R., A. Peytchev, and M.C. Black. 2010. “Comparison of Cell Phone and Landline Surveys: A Design Perspective.” Field Methods 22(1): 3–15. Doi: http://dx.doi.org/10.1177/1525822X09360310.10.1177/1525822X09360310Search in Google Scholar

Chang, L. and J.A. Krosnick. 2009. “National surveys via RDD telephone interviewing versus the internet. Comparing sample representativeness and response quality.” Public Opinion Quarterly 73: 641–678. Doi: http://dx.doi.org/10.1093/poq/nfp075.10.1093/poq/nfp075Search in Google Scholar

Couper, M.P. 2011. “The Future of Modes of Data Collection.” Public Opinion Quarterly 75(5): 889–908. Doi: http://dx.doi.org/10.1093/poq/nfr046.10.1093/poq/nfr046Search in Google Scholar

De Leeuw, E. 2005. “To Mix or Not to Mix Data Collection Modes in Surveys.” Journal of Official Statistics 21: 233–255.Search in Google Scholar

De Leeuw, E.D. and N. Berzelak. 2017. “Survey mode or survey modes?” In The Sage Handbook of Survey Methodology, edited by C. Wolf, D. Joye, T.W. Smith, and Y.-C. Fu, 142–156. London: Sage Publications.10.4135/9781473957893.n11Search in Google Scholar

De Leeuw, E. and W. de Heer. 2002. “Trends in Household Survey Nonresponse: A Longitudinal and International Comparison.” In Survey nonresponse, edited by R. Groves, D. Dillman, J. Eltinge, and R.J.A. Little, 41–54. New York: Wiley.Search in Google Scholar

De Leeuw, E.D., J.J. Hox, and D.A. Dillman. 2008. “Mixed mode surveys: When and why.” In International Handbook of Survey Methodology, edited by E.D. de Leeuw, J.J. Hox, and D.A. Dillman, 299–316. New York/London: Erlbaum/Taylor & Francis.Search in Google Scholar

Dillman, D.A., G. Phelps, R. Tortora, K. Swift, J. Kohrell, J. Berck, and B.L. Messer. 2009. “Response Rate and Measurement Differences in Mixed-Mode Surveys using Mail, Telephone, Interactive Voice Response (IVR) and the Internet.” Social Science Research 38: 1–18. Doi: http://dx.doi.org/10.1016/j.ssresearch.2008.03.007.10.1016/j.ssresearch.2008.03.007Search in Google Scholar

Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixedmode Surveys: the Tailored Design Method (4^th Edition). Hoboken: Wiley.Search in Google Scholar

Eva, G., G. Loosveldt, P. Lynn, P. Martin, M. Revilla, W. Saris, and J. Vannieuwenhuyze. 2010. Assessing the Cost-Effectiveness of Different Modes for ESS Data Collection. London: City University.Search in Google Scholar

Fowler, F.J., P.M. Gallagher, V.L. Stringfellow, A.M. Zaslavsky, J.W. Thompson, and P.D. Cleary. 2002. “Using Telephone Interviews to Reduce Nonresponse Bias to Mail Surveys of Health Plan Members.” Medical Care 40: 190–200.10.1097/00005650-200203000-0000311880792Search in Google Scholar

Gordoni, G., P. Schmidt, and Y. Gordoni. 2012. “Measurement invariance across face-to-face and telephone modes: the case of minority-status collectivistic oriented groups.” International Journal of Public Opinion Research 24(2): 185–207. Doi: http://dx.doi.org/10.1093/ijpor/edq054.10.1093/ijpor/edq054Search in Google Scholar

Greene, J., H. Speizer, and W. Wiitala. 2008. “Telephone and Web: Mixed-Mode Challenge.” Health Services Research 43: 230–248. Doi: http://dx.doi.org/10.1111/j.1475-6773.2007.00747.x.10.1111/j.1475-6773.2007.00747.x232313918211527Search in Google Scholar

Groves, R.M. 1989. Survey Errors and Survey Costs. New York: Wiley.10.1002/0471725277Search in Google Scholar

Groves, R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70: 646–675.10.1093/poq/nfl033Search in Google Scholar

Groves, R.M. 2011. “Three Eras of Survey Research.” Public Opinion Quarterly 75(5): 861–971. Doi: http://dx.doi.org/10.1093/poq/nfl033.10.1093/poq/nfl033Search in Google Scholar

Groves, R.M., F.J. Fowler Jr., M.P. Couper, J.M. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey Methodology (Wiley Series in Survey Methods), 2^nd Ed. Hoboken, NJ: John Wiley & Sons.Search in Google Scholar

Groves, R.M. and L.J. Magilavy. 1984. “An Experimental Measurement of Total Survey Error.” In Proceedings of the Section on Survey Research Methods: American Statistical Association, 698–703. Alexandria, VA: American Statistical Association. Available at: http://ww2.amstat.org/sections/srms/Proceedings/ (accessed March 2017).Search in Google Scholar

Heerwegh, D. and G. Loosveldt. 2011. “Assessing mode effects in a national crime victimization survey using structural equation models: social desirability bias and acquiescence.” Journal of Official Statistics 27: 49–63.Search in Google Scholar

Hochstim, J.R. 1967. “A Critical Comparison of Three Strategies of Collecting Data from Households.” Journal of the American Statistical Association 62: 976–989. Doi: http://dx.doi.org/10.2307/2283686.10.2307/2283686Search in Google Scholar

Holbrook, A., M. Green, and J. Krosnick. 2003. “Telephone Versus Face-to-face Interviewing of National Probability Samples with Long Questionnaires.” Public Opinion Quarterly 67: 79–125. Doi: http://dx.doi.org/10.1086/346010.10.1086/346010Search in Google Scholar

Hox, J., E.D. de Leeuw, and T. Klausch. 2017. “Mixed mode research: Issues in design and analysis.” In Total Survey Error in Practice: Improving Quality in the Era of Big Data, edited by P.P. Biemer, E.D. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L.E. Lyberg, C. Tucker, and B.T. West, 511–530. Hoboken, NJ: John Wiley and Sons, Inc.10.1002/9781119041702.ch23Search in Google Scholar

Hox, J.J., E.D. de Leeuw, and E.A.O. Zijlmans. 2015. “Measurement equivalence in mixed mode surveys.” Frontiers in Psychology: Quantitative Psychology and Measurement. Doi: http://dx.doi.org/10.3389/fpsyg.2015.00087.10.3389/fpsyg.2015.00087431828225699002Search in Google Scholar

Kappelhof, J.W.S. 2013. “The Effect of Different Survey Designs on Nonresponse in Surveys of Non-Western Minorities in The Netherlands.” Survey Research Methods 8(2): 81–98. Doi: http://dx.doi.org/10.18148/srm/2014.v8i2.5784.Search in Google Scholar

Klausch, L.T., J.J. Hox, and B. Schouten. 2013. “Measurement effects of survey mode on the equivalence of attitudinal rating scale questions.” Sociological Methods and Research 42: 227–263. Doi: http://dx.doi.org/10.1177/0049124113500480.10.1177/0049124113500480Search in Google Scholar

Klausch, T., J.J. Hox, and B. Schouten. 2015a. “Selection Error in Single- and Mixed Mode Surveys of the Dutch General Population.” Journal of the Royal Statistical Society Series A 178: 945–961. Doi: http://dx.doi.org/10.1111/rssa.12102.10.1111/rssa.12102Search in Google Scholar

Klausch, T., B. Schouten, and J.J. Hox. 2014. “The Use of Within-subject Experiments for Estimating Measurement Effects in Mixed-mode Surveys.” Statistics Netherlands Discussion Paper, 2015/06. Available at: https://www.cbs.nl/en-gb/background/2014/11/the-use-of-within-subject-experiments-for-estimating-measurement-effects-in-mixed-mode-surveys (accessed March 2017).Search in Google Scholar

Klausch, T., B. Schouten, and J.J. Hox. 2015b. “Evaluating Bias of Sequential Mixedmode Designs Against Benchmark Surveys.” Sociological Methods and Research 1–34. Doi: http://dx.doi.org/10.1177/0049124115585362.10.1177/0049124115585362Search in Google Scholar

Kreuter, F., S. Presser, and R. Tourangeau. 2008. “Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity.” Public Opinion Quarterly 72: 847–865. Doi: http://dx.doi.org/10.1093/poq/nfn063.10.1093/poq/nfn063Search in Google Scholar

Kreuter, F., G. Müller, and M. Trappmann. 2010. “Nonresponse and Measurement Error in Employment Research: Making Use of Administrative Data.” Public Opinion Quarterly 74: 880–906. Doi: http://dx.doi.org/10.1093/poq/nfq0.Search in Google Scholar

Link, M.W. and A.H. Mokdad. 2006. “Can Web and Mail Survey Modes Improve Participation in an RDD-Based National Health Surveillance?” Journal of Official Statistics 22: 293–312.Search in Google Scholar

Lipps, O., N. Pekari, and C. Roberts. 2015. “Undercoverage and Nonresponse in a List-Sampled Telephone Election Study.” Survey Research Methods 9(2): 71–82. Doi: http://dx.doi.org/10.18148/srm/2015.v9i2.6139.Search in Google Scholar

Lynn, P. 2013. “Alternative Sequential Mixed-Mode Designs: Effects on Attrition Rates, Attrition Bias, and Costs.” Journal of Survey Statistics and Methodology 1: 183–205. Doi: http://dx.doi.org/10.1093/jssam/smt015.10.1093/jssam/smt015Search in Google Scholar

Massey, D.S. and R. Tourangeau. 2013. “Where Do We Go from Here? Nonresponse and Social Measurement.” The Annals of the American Academy of Political and Social Science 645(1): 222–236. Doi: http://dx.doi.org/10.1177/0002716212464191.10.1177/0002716212464191425747725484368Search in Google Scholar

Millar, M.M. and D.A. Dillman. 2011. “Improving Response to Web and Mixed-Mode Surveys.” Public Opinion Quarterly 75(2): 249–269. Doi: http://dx.doi.org/10.1093/poq/nfr003.10.1093/poq/nfr003Search in Google Scholar

Olson, K. 2006. “Survey Participation, Nonresponse Bias, Measurement Error Bias, and Total Bias.” Public Opinion Quarterly 70(5): 737–758. Doi: http://dx.doi.org/10.1093/poq/nfl038.10.1093/poq/nfl038Search in Google Scholar

Olson, K., J.D. Smyth, and H. Wood. 2012. “Does Providing Sample Members with Their Preferred Survey Mode Really Increase Participation Rates?” Public Opinion Quarterly 76(4): 611–635. Doi: http://dx.doi.org/10.1093/poq/nfs024.10.1093/poq/nfs024Search in Google Scholar

Peytchev, A., R.K. Baxter, and L.R. Carley-Baxter. 2009. “Not All Survey Effort is Equal. Reduction of Nonresponse Bias and Nonresponse Error.” Public Opinion Quarterly 73(4): 785–806. Doi: http://dx.doi.org/10.1093/poq/nfp037.10.1093/poq/nfp037Search in Google Scholar

Peytcheva, E. and R.M. Groves. 2009. “Using Variation in Response Rates of Demographic Subgroups as Evidence of Nonresponse Bias in Survey Estimates.” Journal of Official Statistics 25(2): 193–201.Search in Google Scholar

Rao, J.N.K. and A.J. Scott. 1987. “On simple adjustments to chi-square tests with sample survey data.” The Annals of Statistics 15(1): 385–397. Doi: http://dx.doi.org/10.1214/aos/1176348654.10.1214/aos/1176348654Search in Google Scholar

Roberts, C., D. Joye, M. Ernst Stähli, and R. Sanchez Tome. 2016. Mixing modes of data collection in Swiss social surveys: Methodological Report of the LIVES-FORS Mixed Mode Experiment. LIVES Working Paper Series, 2016/48. Doi: http://dx.doi.org/10.12682/lives.2296-1658.2016.48.Search in Google Scholar

Sakshaug, J.W., T. Yan, and R. Tourangeau. 2010. “Nonresponse Error, Measurement Error, and Mode of Data Collection: Tradeoffs in a Multi-mode Survey of Sensitive and Non-sensitive Items.” Public Opinion Quarterly 74(5): 907–933. Doi: http://dx.doi.org/10.1093/poq/nfq057.10.1093/poq/nfq057Search in Google Scholar

Schouten, B., J. van den Brakel, B. Buelens, J. van der Laan, and T. Klausch. 2013. “Disentangling Mode-Specific Selection and Measurement Bias in Social Surveys.” Social Science Research 42(6): 1555–1570. Doi: http://dx.doi.org/10.1016/j.ssresearch.2013.07.005.10.1016/j.ssresearch.2013.07.00524090851Search in Google Scholar

Siemiatycki, J. 1979. “A Comparison of Mail, Telephone, and Home Interview Strategies for Household Health Surveys.” American Journal of Public Health 69: 238–245.10.2105/AJPH.69.3.238Search in Google Scholar

Smith, T.W. 2011. “Refining the Total Survey Error Perspective.” International Journal of Public Opinion Research 23(4): 464–484. Doi: http://dx.doi.org/10.1093/ijpor/edq052.10.1093/ijpor/edq052Search in Google Scholar

Suzer-Gurtekin, Z., S. Heeringa, and R. Vaillant. 2012. “Investigating the Bias of Alternative Statistical Inference Methods in Sequential Mixed-mode Surveys.” Proceedings of the JSM, Section on Survey Research Methods 4711-2. Available at: https://www.niss.org/sites/default/files/VII%201%20Suzer-Gurtekin_itsew2013.pdf (accessed April 2016).Search in Google Scholar

Tourangeau, R. 2017. “Mixing modes: Tradeoffs among coverage, nonresponse, and measurement error.” In Total Survey Error in Practice: Improving Quality in the Era of Big Data, edited by P.P. Biemer, E.D. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L.E. Lyberg, C. Tucker, and B.T. West. 115–132. Hoboken, NJ: John Wiley and Sons, Inc.10.1002/9781119041702.ch6Search in Google Scholar

Tourangeau, R., R.M. Groves, and C.D. Redline. 2010. “Sensitive topics and reluctant respondents: Demonstrating a link between nonresponse bias and measurement error.” Public Opinion Quarterly 74(3): 413–432. Doi: http://dx.doi.org/10.1093/poq/nfq004.10.1093/poq/nfq004Search in Google Scholar

Vannieuwenhuyze, J.T.A., G. Loosveldt, and G. Molenberghs. 2010. “A Method for Evaluating Mode Effects in Mixed-mode Surveys.” Public Opinion Quarterly 74: 1027–1045. Doi: http://dx.doi.org/10.1093/poq/nfq059.10.1093/poq/nfq059Search in Google Scholar

Vannieuwenhuyze, J.T.A. 2014. “On the Relative Advantage of Mixed-Mode versus Single-Mode Surveys.” Survey Research Methods 8(1): 31–42. Doi: http://dx.doi.org/10.18148/srm/2014.v8i1.5500#sthash.xSmtK1fH.dpuf.Search in Google Scholar

Vannieuwenhuyze, J.T.A. and G. Loosveldt. 2012. “Evaluating Relative Mode Effects in Mixed-mode Surveys: Three Methods to Disentangle Selection and Measurement Effects.” Sociological Methods and Research 42: 82–104. Doi: http://dx.doi.org/10.1177/0049124112464868.10.1177/0049124112464868Search in Google Scholar

Wagner, J., J. Arrieta, H. Guyer, and M.B. Ofstedal. 2014. “Does Sequence Matter in Multimode Surveys: Results from an Experiment.” Field Methods 26(2): 141–155. Doi: http://dx.doi.org/10.1177/1525822X13491863.10.1177/1525822X13491863399248024764767Search in Google Scholar