1. bookVolume 37 (2021): Issue 4 (December 2021)
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
access type Open Access

Comparing the Response Burden between Paper and Web Modes in Establishment Surveys

Published Online: 26 Dec 2021
Page range: 907 - 930
Received: 01 Sep 2019
Accepted: 01 Mar 2021
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
Abstract

Previous research is inconclusive regarding the effects of paper and web surveys on response burdens. We conducted an establishment survey with random assignment to paper and web modes to examine this issue. We compare how the actual and perceived response burdens differ when respondents complete a survey in the paper mode, in the web mode and when they are allowed to choose between the two modes. Our results show that in the web mode, respondents have a lower estimated time to complete the questionnaire, while we do not find differences between paper and the web on the perceived response time and perceived burden. Even though the response burden in the web mode is lower, our study finds no evidence of an increased response burden when moving an establishment survey from paper to the web.

Keywords

AAPOR. 2016. Standard Definitions: Final Disposition of Case Codes and Outcome Rates for Surveys. Available at: http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf (accessed March 2019). Search in Google Scholar

Bavdaž, M., D. Giesen, S.K.Černe, T. Löfgren, and V. Raymond-Blaess. 2015. “Response burden in official business surveys: Measurement and reduction practices of national statistical institutes.” Journal of Official Statistics 31: 559–588. https://doi.org/10.1515/jos-2015-0035.10.1515/jos-2015-0035 Search in Google Scholar

Berglund, F., G. Haraldsen, and Ø. Kleven. 2013. “Causes and consequences of actual and perceived response burden based on Norwegian data.” In Comparative report on integration of case study results related to reduction of response burden and motivation of business, edited by D. Giesen, M. Bavdaž, and I. Bolko: 29–35. Search in Google Scholar

Bradburn, N.M. 1978. “Respondent burden.” In Proceedings of the Section on Survey Research Methods Section: American Statistical Association, August 14–17, 1978. 35–40. San Diego, USA. American Statistical Association: 35–40. Available at: http://www.asasrms.org/Proceedings/y1978f.html (accessed March 2021). Search in Google Scholar

Bremner, C. 2011. “An investigation into the use of mixed mode data collection methods for UK business surveys.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys, edited by D. Giesen and M. Bavdaž. March 22–23, (pp. 217–220). Heerlen, The Netherlands: Statistics Netherlands. Available at: http://www.cbs.nl/NR/rdonlyres/23FD3DF5-6696-4A04-B8EF-1FAACEAD995C/0/2011proceedingsblueets.pdf (accessed March 2019). Search in Google Scholar

Cameron, A.C., and P.K. Trivedi. 2005. Microeconometrics: Methods and Applications. New York: Cambridge University Press.10.1017/CBO9780511811241 Search in Google Scholar

Conrad, F.G., M.F. Schober, and T. Coiner. 2007. “Bringing Features of Human Dialogue to Web Surveys.” Applied Cognitive Psychology 21: 165–187. DOI: https://doi.org/10.1002/acp.1335.10.1002/acp.1335 Search in Google Scholar

Couper, M.P. 2008. Designing Effective Web Surveys. Cambridge: Cambridge University Press.10.1017/CBO9780511499371 Search in Google Scholar

Couper, M.P., and R.M. Groves. 1996. “Household-level determinants of survey nonresponse.” New Directions for Evaluation 70: 63–79. DOI: https://doi.org/10.1002/ev.1035.10.1002/ev.1035 Search in Google Scholar

Dale, T., J. Erikson, J. Fosen, G. Haraldsen, J. Jones, and O. Kleven. 2007. Handbook for monitoring and evaluating business survey response burdens. Luxemburg: Eurostat. Available at: https://ec.europa.eu/eurostat/documents/64157/4374310/12-HANDBOOK-FOR-MONITORING-AND-EVALUATING-BUSINESS-SURVEY-RESONSE-BURDEN.pdf/600e3c6d-8e8d-44f7-a8f5-0931c71d9920 (accessed October 2021). Search in Google Scholar

Destatis. 2008. Klassifikation der Wirtschaftszweige. Wiesbaden, Germany: Statistisches Bundesamt. Available at: https://www.destatis.de/DE/Methoden/Klassifikationen/Gueter-Wirtschaftsklassifikationen/klassifikation-wz-2008.html (accessed June 2019). Search in Google Scholar

Downey, K., D. Mccarthy, and W. Mccarthy. 2007. “Encouraging the Use of Alternative Modes of Electronic Data Collection: Results of Two Field Studies.” In Proceedings of the Third International Conference on Establishment Surveys, June 18–21, 2007. Montréal, Canada: 517–524. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000168.PDF (accessed September 2019). Search in Google Scholar

Erikson, J. 2007. “Effects of offering web questionnaires as an option in enterprise surveys.” In Proceedings of the Third International Conference on Establishment Surveys, June 18–21, 2007. Montréal, Canada: 1431–1435. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000168.PDF (accessed September 2019) Search in Google Scholar

European Commission. 2011. European Statistics Code of Practice for the National and Community Statistical Authorities. Adopted by the European Statistical System Committee, September 28, 2011. Available at: https://ec.europa.eu/eurostat/documents/3859598/5921861/KS-32-11-955-EN.PDF/5fa1ebc6-90bb-43fa-888f-dde032471e15 (accessed September 2019). Search in Google Scholar

Gelman, A. 2007. “Struggles with Survey Weighting and Regression Modeling.” Statistical Science 22(2) : 153–164. DOI: https://doi.org/10.1214/088342306000000691.10.1214/088342306000000691 Search in Google Scholar

Giesen, D. 2007. “Does mode matter? Comparing response burden and data quality of paper and an electronic business questionnaire.” In Proceedings of the 6th Conference on Questionnaire Evaluation Standards (QUEST), April 24–26, 2007. Ottawa, Canada: 150–161. Available at: https://wwwn.cdc.gov/QBank/QUEST/2007/QUEST%202007%20Proceedings-all%20papers.pdf (accessed September 2019). Search in Google Scholar

Giesen, D. 2011. “Burden reduction by communication.” In Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical Institutes, edited by D. Giesen: 33–42. Search in Google Scholar

Giesen, D. 2012. “Exploring Causes and Effects of Perceived Response Burden.” In Proceedings of the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montréal, Canada. Available at: https://ww2.amstat.org/meetings/ices/2012/papers/302171.pdf (accessed September 2019). Search in Google Scholar

Giesen, D. 2013a. “Causes and Consequences of Actual and Perceived Response Burden Based on Dutch Data.” In Comparative report on integration of case study results related to reduction of response burden and motivation of business, edited by D. Giesen, M. Bavdaž, and I. Bolko: 33–39. Search in Google Scholar

Giesen, D. 2013b. “Reducing Response Burden by Questionnaire Redesign.” In Comparative report on integration of case study results related to reduction of response burden and motivation of business, edited by D. Giesen, M. Bavdaž, and I. Bolko: 63–68. Search in Google Scholar

Giesen, D., M. Bavdaž, and G. Haraldsen. 2011. “Response burden measurement: Current diversity and proposal for moving towards standardisation.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys, edited by D. Giesen and M. Bavdaž. March 22–23: 125–134. Heerlen, The Netherlands: Statistics Netherlands. Available at: https://www.cbs.nl/-/media/imported/documents/2011/14/2011-4-4-4-2-giesen-et-al-presentation-blue-ets-2011.pdf (accessed March 2021). Search in Google Scholar

Giesen, D., and J. Burger. 2013. “Measuring and understanding response quality in the Structural Business Survey questionnaires.” In Proceedings of the European Establishment Statistics Workshop, September 9–11, 2013. Nuremberg, Germany. Available at: http://doku.iab.de/fdz/events/2013/Session5%20Giesen.pdf (accessed September 2019). Search in Google Scholar

Giesen, D., M. Morren, and G. Snijkers. 2009. “The effect of survey redesign on response burden: An evaluation of the redesign of the SBS questionnaires.” Paper presented at the European Survey Research Association Conference 2009, June 29–July 3, 2009. Warsaw, Poland. European Survey Research Association. Search in Google Scholar

Giesen, D., M. Vella, and C. Brady. 2018. “Response Burden Management for Establishment Surveys at Four National Statistical Institutes.” Journal of Official Statistics 34(2): 397–418. DOI: https://doi.org/10.2478/jos-2018-0018.10.2478/jos-2018-0018 Search in Google Scholar

Groves, R.M., R.B. Cialdini, and M.P. Couper. 1992. “Understanding the decision to participate in a survey.” Public Opinion Quarterly 56: 475–495. DOI: https://doi.org/10.1086/269338.10.1086/269338 Search in Google Scholar

Gregory, G., and M. Earp. 2007. “Evolution of Web at USDA’ National Agricultural Statistics Service.” In Proceedings of the Third International Conference on Establishment Surveys (ICES-III), June 18–21, 2007. Montréal, Canada: 1442–1445. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000192.PDF (accessed September 2019). Search in Google Scholar

Gravem, D. 2011. “Response burden trends and consequences.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys, edited by D. Giesen and M. Bavdaž. March 22–23: 221–236. Heerlen, The Netherlands: Statistics Netherlands. Available at: http://www.cbs.nl/NR/rdonlyres/23FD3DF5-6696-4A04-B8EF-1FAACEAD995C/0/2011proceedingsblueets.pdf (accessed March 2019). Search in Google Scholar

Haas, G.-C., S. Eckman, R. Bach, and F. Kreuter. 2016. “Is Moving Establishment Surveys from Mail to Web a Good or Bad Decision in Terms of Performance and Data Quality?” In Proceedings of the International Conference for Establishment Surveys 2016 (ICES-V). June 21–23, 2016. Geneva, Switzerland. Available at: https://ww2.amstat.org/meetings/ices/2016/proceedings/ICESV_TOC.pdf (accessed February 2020). Search in Google Scholar

Haraldsen, G., and J. Jones. 2007. “Paper and Web Questionnaires Seen from the Business Respondent’s Perspective.” In Proceedings of the Third International Conference on Establishment Surveys (ICES-III), June 18–21, 2007, Montreal, Canada: 1040–1047. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000259.PDF (accessed September 2019). Search in Google Scholar

Haraldsen, G., J. Jones, D. Giesen, and L.C. Zhang. 2013. “Understanding and coping with response burden.” In Designing and Conducting Business Surveys, edited by G. Snijkers, G. Haraldsen, J. Jones, and D. Willimack: 219–252. Hoboken, NJ: John Wiley and Sons.10.1002/9781118447895.ch06 Search in Google Scholar

Harrell, L., H. Yu, and R. Rosen. 2007. “Respondent acceptance of web and E-mail data reporting for an establishment survey.” In Proceedings of the Third International Conference on Establishment Surveys (ICES-III), June 18–21, 2007, Montreal, Canada: 1442–1445. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000230.PDF (accessed September 2019). Search in Google Scholar

Hedlin, D., T. Dale, G. Haraldsen, and J. Jones. 2005. Developing Methods for Assessing Perceived Response Burden. Luxembourg: Eurostat. Available at: https://ec.europa.eu/eurostat/documents/64157/4374310/10-DEVELOPING-METHODS-FOR-ASSESSING-PERCEIVED-RESPONSE-BURDEN.pdf/1900efc8-1a07-4482-b3c9-be88ee71df3b (accessed October 2021). Search in Google Scholar

Jones, J. 2012. “Response Burden: Introductory Overview Lecture.” In Proceedings of the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montréal, Canada. Available at: http://www.amstat.org/meetings/ices/2012/papers/302289.pdf (accessed September 2019). Search in Google Scholar

Jones, J., G. Snijkers, and G. Haraldsen. 2013. “Surveys and Business Surveys.” In Designing and Conducting Business Surveys, edited by G. Snijkers, G. Haraldsen, J. Jones, and D.K. Willimack: 1–33. Hoboken, NJ: John Wiley & Sons.10.1002/9781118447895.ch01 Search in Google Scholar

Kaplowitz, M.D., T.D. Hadlock, and R. Levine. 2004. “A Comparison of Web and Mail Survey Response Rates.” Public Opinion Quarterly 68: 94 – 101. DOI: https://doi.org/10.1093/poq/nfh006.10.1093/poq/nfh006 Search in Google Scholar

Krosnick, J.A. 1991. “Response strategies for coping with the cognitive demands of attitude measures in surveys.” Applied Cognitive Psychology 5: 213–236. DOI: https://doi.org/10.1002/acp.2350050305.10.1002/acp.2350050305 Search in Google Scholar

Kwak, N., and B. Radler. 2002. “A Comparison between Mail and Web Surveys: Response Pattern, Respondent Profile, and Data Quality.” Journal of Official Statistics 18: 257–273. DOI: https://doi.org/10.1177/1525822X08317085.10.1177/1525822X08317085 Search in Google Scholar

Lagerstøm, B. 2018. “Chatbots as digital interviewers.” Paper presented at the International Household Nonresponse Workshop, August 22–24, 2018. Budapest, Hungary Search in Google Scholar

Löfgren, T. 2011. “Burden reduction by instrument design.” In Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical Institutes, edited by D. Giesen: 43–50. Search in Google Scholar

Lyly-Yrjänäinen, M., and G. van Houten. 2011. “Reduce burden, increase motivation. Main findings of the quality assessment of the second European company survey.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys March 22–23, 2011, Heerlen, Netherlands, edited by D. Giesen and M. Bavdaž: 107–118. Available at: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiFvfTm2OrzAhU8BWMBHYOlDv8QFnoECAgQAQ&url=https%3A%2F%2Fwww.cbs.nl%2F-%2Fmedia%2Fimported%2Fdocuments%2F2011%2F25%2F2011-06-20-lyly-yrjanainen-van-houten-2011-reduce-burden-increase-motivation.pdf&usg=AOvVaw0n4dUPG4Ectn1K-baV7hE0 (accessed October 2021). Search in Google Scholar

Medway, R.L., and J. Fulton. 2012. “When More Gets You Less: A Meta-Analysis of the Effect of Concurrent Web Options on Mail Survey Response Rates.” Public Opinion Quarterly 76: 733–746. DOI: https://doi.org/10.1093/poq/nfs047.10.1093/poq/nfs047 Search in Google Scholar

Messer, B.L., and D.A. Dillman. 2010. Using Address Based Sampling to Survey the General Public by Mail vs. “Web plus Mail”. Technical Report 10–13. Social and Economic Sciences Research Center, Washington State University, Pullman. Available at: http://www.sesrc.wsu.edu/dillman/papers/2010/Messer%20Dillman%20WCSTechReport.pdf (accessed October 2021). Search in Google Scholar

Millar, M.M., O.A.C. Neill, and D.A. Dillman. 2009. “Are Mode Preferences Real?” Technical Report of the Social and Economic Sciences Research Center. Pullman, Washington: Washington State University Search in Google Scholar

Sear, J. 2011. “Response burden measurement and motivation at Statistics Canada.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys Heerlen, March 22–23, 2011, Heerlen, Netherlands. edited by D. Giesen and M. Bavdaž: 151–160. Available at: https://www.cbs.nl/-/media/imported/-documents/2011/25/2011-06-20-sear-2011-response-burden-measurement-and-motivation-at-statistics-canada.pdf (accessed October 2021). Search in Google Scholar

Snedecor, G.W., and W.G. Cochran. 1989. Statistical Methods, (8th edition). Ames, IA: Iowa State University Press. Search in Google Scholar

Snijkers, G., G. Haraldsen, A. Sundvoll, T. Vik, and H.P. Stax. 2011. “Utilizing web technology in business data collection: Some Norwegian, Dutch and Danish experiences.” In Proceeding of the European Conference on New Techniques and Technologies for Statistics (NTTS), Brussels, Belgium Search in Google Scholar

Snijkers, G., E. Onat, and R. Vis-Visschers. 2007. “The Annual Structural Business Survey: Developing and Testing an Electronic Form.” In Proceedings of the International Conference for Establishment Surveys 2007 (ICES-III) June 18–21, 2007, Montreal, Canada: 456–463. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000095.PDF (accessed October 2021). Search in Google Scholar

Verkruyssen, F., and S. Moens. 2011. “Communication as a tool to reduce perceived response burden: Tips and tricks.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys, March 22–23, 2011, Heerlen, Netherlands edited by D. Giesen and M. Bavdaž. 237 – 242. Available at: https://www.cbs.nl/-/media/imported/documents/2011/14/2011-4-4-8-1-verkruyssenmoens-presentation-blue-ets-2011.ashx?la=en-gb (accessed October 2021). Search in Google Scholar

Willimack, D., and E. Nichols. 2010. “A Hybrid Response Process Model for Business Surveys.” Journal of Official Statistics 26: 3–24. Available at: https://www.scb.se/-contentassets/ca21efb41fee47d293bbee5bf7be7fb3/a-hybrid-response-process-model-for-business-surveys.pdf (accessed September 2021). Search in Google Scholar

Van Loon, A.J.M., M. Tijhuis, H.S. Picavet, P.G. Surtees, and J. Ormel. 2003. “Survey Non-response in the Netherlands: Effects on Prevalence Estimates and Associations.” Ann Epidemiol 13: 105–110. DOI: https://doi.org/10.1016/S1047-2797(02)00257-0.10.1016/S1047-2797(02)00257-0 Search in Google Scholar

Yan, T., S. Fricker, and S. Tsai. 2020. “Response Burden: What Is It and What Predicts It?” In Advances in Questionnaire Design, Development, Evaluation and Testing, edited by P.C. Beatty, D. Collins, L. Kaye, J.-L. Padilla, G.B. Willis, and A. Wilmot: 193–212. Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo