1. bookVolume 37 (2021): Issue 4 (December 2021)
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
access type Open Access

Response Burden and Data Quality in Business Surveys

Published Online: 26 Dec 2021
Page range: 811 - 836
Received: 01 Aug 2019
Accepted: 01 Mar 2021
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
Abstract

Response burden has long been a concern for data producers. In this article, we investigate the relationship between some measures of actual and perceived burden and we provide empirical evidence of their association with data quality. We draw on two business surveys conducted by Banca d’Italia since 1970, which provide a very rich and unique source of information. We find evidence that the perceived burden is affected by actual burden but the latter is not the only driver. Our results also show a clear link between a respondent’s perceived effort and the probability of not answering some important questions (such as those relating to expectations of future investments and turnover) or of dropping out of the survey. On the contrary, we do not find significant effects on the quality of answers to quantitative questions such as business turnover and investments. Overall, these findings have implications for data producers that should target the perceived burden, besides the actual burden, to increase data quality.

Keywords

Bavdaž, M. 2010. “Sources of Measurement Errors in Business Surveys.” Journal of Official Statistics 26 (1): 24–42. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/sources-of-measurement-errors-in-business-surveys.pdf (accessed September 2021). Search in Google Scholar

Bavdaž, M., D. Giesen, S.K. Černe, T. Löfgren, and V. Raymond-Blaess. 2015. “Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical Institutes.” Journal of Official Statistics 31 (4): 559–588. DOI: https://doi.org/10.1515/jos-2015-0035.. Search in Google Scholar

Bergman, L.R., and R. Brage. 2008. “Survey Experience and Later Survey Attitudes, Intention and Behavior.” Journal of Official Statistics 24 (1): 99–113. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/survey-experiences-and-later-survey-attitudesintentions-and-behaviour.pdf (accessed September 2021). Search in Google Scholar

Bradburn, N.M. 1978. “Respondent burden.” In Proceedings of the Survey Research Methods Section of the American Statistical Association, 35–40. Available at: http://www.asasrms.org/Proceedings/papers/1978_007.pdf (accessed March 2021) Search in Google Scholar

Crawford, S.D., M.P. Couper, and M.J. Lamias. 2001. “Web surveys: Perceptions of burden.” Social Science Computer Review 19 (2): 146–162. DOI: https://doi.org/10.1177/089443930101900202.10.1177/089443930101900202 Search in Google Scholar

Dale, T., J. Erikson, J. Fosen, G. Haraldsen, J. Jones, and Ø. Kleven. 2007. Handbook for Monitoring and Evaluating Business Survey Response Burdens. European Commission. Available at: https://ec.europa.eu/eurostat/documents/64157/4374310/12-HANDBOOK-FORMONITORING-AND-EVALUATING-BUSINESS-SURVEY-RESONSE-BURDEN.pdf/600e3c6d-8e8d-44f7-a8f5-0931c71d9920 (accessed March 2021) Search in Google Scholar

D’Aurizio, L., and G. Papadia. 2019. “Using Administrative Data to Evaluate Sampling Bias in a Business Panel Survey.” Journal of Official Statistics 35 (1): 67–92. DOI: https://doi.org/10.2478/jos-2019-0004.10.2478/jos-2019-0004 Search in Google Scholar

Edwards, P., I. Roberts, M. Clarke, C. DiGuiseppi, S. Pratap, R. Wentz, and I. Kwan. 2002. “Increasing response rates to postal questionnaires: systematic review.” BMJ 324 (7347): 1183. DOI: https://doi.org/10.1136/bmj.324.7347.1183.10.1136/bmj.324.7347.1183 Search in Google Scholar

European Commission. Statistical Office of the European Union. 2014. Manuals and guidelines. ESS Handbook for Quality Reports. DOI: https://doi.org/10.2785/983454. Search in Google Scholar

Fisher, S., and L. Kydoniefs. 2001. “Using a theoretical model of response burden to identify sources of burden in surveys.” Paper presented at the 12th International Workshop on Household Survey Nonresponse, September 12–14, Oslo, Norway. Search in Google Scholar

Galesic, M. 2006. “Dropouts on the Web: Effects of Interest and Burden Experienced During an Online Survey.” Journal of Official Statistics 22: 313–328. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/dropouts-onthe-web-effects-of-interest-and-burden-experienced-during-an-online-survey.pdf (accessed September 2021). Search in Google Scholar

Galesic, M., and M. Bosnjak. 2009. “Effects of questionnaire length on participation and indicators of response quality in a web survey.” Public Opinion Quarterly 73 (2): 349–360. DOI: https://doi.org/10.1093/poq/nfp031.10.1093/poq/nfp031 Search in Google Scholar

Giesen, D. 2012. “Exploring causes and effects of perceived response burden.” In Proceedings of the Fourth International Conference on Establishment Surveys: American Statistical Association, June 11–14, Montréal, Canada. Available at: https://ww2.amstat.org/meetings/ices/2012/papers/302171.pdf (accessed March 2021) Search in Google Scholar

Groves, R.M., M.P. Couper, S. Presser, E. Singer, R. Tourangeau, G.P. Acosta, and L. Nelson. 2006. “Experiments in producing nonresponse bias.” Public Opinion Quarterly 70 (5): 720–736. DOI: https://doi.org/10.1093/poq/nfl036.10.1093/poq/nfl036 Search in Google Scholar

Haraldsen, G. 2004. “Identifying and Reducing Response Burden in Internet Business Surveys.” Journal of Official Statistics 20 (2): 393–410. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/identifying-and-reducing-response-burdens-in-internet-business-surveys.pdf (accessed September 2021). Search in Google Scholar

Haraldsen, G. 2013. “Quality issues in business surveys.” Designing and Conducting Business Surveys, 83–125. John Wiley & Sons. DOI: https://doi.org/10.1002/9781118447895.ch03.10.1002/9781118447895.ch03 Search in Google Scholar

Haraldsen, G., and J. Jones. 2007. “Web and paper questionnaires seen from the business respondent’s perspective.” In Proceedings of the Third International Conference for Establishments Surveys, JUNE 18–21, Montréal, Canada. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000259.PDF (accessed March 2021) Search in Google Scholar

Haraldsen, G., J. Jones, D. Giesen, and L.-C. Zhang. 2013. “Understanding and coping with response burden.” In Designing and Conducting Business Surveys, 219–252. John Wiley & Sons. DOI: https://doi.org/10.1002/9781118447895.ch06.10.1002/9781118447895.ch06 Search in Google Scholar

Heerwegh, D., and G. Loosveldt. 2002. “An evaluation of the effect of response formats on data quality in web surveys.” Social Science Computer Review 20 (4): 471–484. DOI: https://doi.org/10.1177/089443902237323.10.1177/089443902237323 Search in Google Scholar

Holbrook,A.L.,J.A.Krosnick,D.Moore,andR. Tourangeau. 2007. “Response order effects in dichotomous categorical questions presented orally: The impact of question and respondent attributes.” Public Opinion Quarterly 71 (3): 325–348. DOI: https://doi.org/10.1093/poq/nfm024.10.1093/poq/nfm024 Search in Google Scholar

Jones, J., J. Rushbrooke, G. Haraldsen, T. Dale, and D. Hedlin. 2005. “Conceptualising total business survey burden.” In Survey Methodology Bulletin 55. UK Office for National Statistics, 1–10. Available at: http://www.ons.gov.uk/ons/guide-method/method-quality/survey-methodology-bulletin/smb-55/index.html (accessed March 2021) Search in Google Scholar

Peytchev, A. 2011. “Breakoff and Unit Nonresponse Across Web Surveys.” Journal of Official Statistics 27 (1): 33. Available at: https://www.scb.se/contentassets/ca21efb41-fee47d293bbee5bf7be7fb3/breakoff-and-unit-nonresponse-across-web-surveys.pdf (accessed September 2021). Search in Google Scholar

Presser, S., J. Blair, and T. Triplett. 1992. “Survey sponsorship, response rates, and response effects.” Social Science Quarterly 73 (5): 699–702. Available at: http://www.jstor.org/stable/42863089. Search in Google Scholar

Snijkers, G., G. Haraldsen, J. Jones, and D.K. Willimack. 2013. Designing and conducting business surveys. New York: Wiley.10.1002/9781118447895 Search in Google Scholar

Tomaskovic-Devey, D., J. Leiter, and S. Thompson. 1994. “Organizational survey nonresponse.” Administrative Science Quarterly 39 (3): 439–457. DOI: https://doi.org/10.2307/2393298.10.2307/2393298 Search in Google Scholar

Tourangeau, R., T. Yan, and H. Sun. 2019. “Who Can You Count On? Understanding The Determinants of Reliability.” Journal of Survey Statistics and Methodology 8 (5): 903–931. DOI: https://doi.org/10.1093/jssam/smz034.10.1093/jssam/smz034 Search in Google Scholar

Willeboordse, A. 1997. “Minimizing response burden.” In Handbook on Design and Implementation of Business Surveys, edited by Willeboordse. 111–118. Available at: https://ec.europa.eu/eurostat/documents/3859598/5825949/CA-09-97-818-EN.PDF/f5ee3198-6fc0-4672-96a1-8fdb4a81ca93?version=1.0 (accessed March 2021). Search in Google Scholar

Yan, T., F.G. Conrad, R. Tourangeau, and M.P. Couper. 2010a. “Should I stay or should I go: The effects of progress feedback, promised task duration, and length of questionnaire on completing web surveys.” International Journal of Public Opinion Research 23 (2): 131–147. DOI: https://doi.org/10.1093/ijpor/edq046.10.1093/ijpor/edq046 Search in Google Scholar

Yan, T., R. Curtin, and M. Jans. 2010b. “Trends in Income Nonresponse Over Two Decades.” Journal of Official Statistics 26 (1): 145 – 164. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/trends-in-income-nonresponse-over-two-decades.pdf (accessed September 2021). Search in Google Scholar

Yan, T., S. Fricker, and S. Tsai. 2014. “The impact of response burden on data quality in a longitudinal survey.” In Proceedings of the International Total Survey Error Workshop, October 1–3, Washington, D.C. Available at: https://www.niss.org/events/2014-international-total-survey-error-workshop-itsew-2014 (accessed March 2021) Search in Google Scholar

Yan, T., S. Fricker, and S. Tsai. 2019. “Response burden: What is it and what predicts it?” In Advances in Questionnaire Design, Development, Evaluation and Testing, 193–212. John Wiley & Sons. DOI: https://doi.org/10.1002/9781119263685.ch8.10.1002/9781119263685.ch8 Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo