Survey Mode Effects on Objective and Subjective Questions: Evidence from the Labour Force Survey

Aggarwal, V.A., and D.H. Hsu. 2013. “Entrepreneurial exits and innovation.” Management Science, 60(4): 867–887. DOI: https://doi.org/10.1287/mnsc.2013.1801.10.1287/mnsc.2013.1801Search in Google Scholar

Athey, S., and G.W. Imbens. 2017. “The state of applied econometrics: Causality and policy evaluation.” Journal of Economic Perspectives, 31(2): 3–32. DOI: http://dx.doi.org/10.1257/jep.31.2.3.10.1257/jep.31.2.3Search in Google Scholar

Azur, M.J., E.A. Stuart, C. Frangakis, and P.J. Leaf. 2011. “Multiple imputation by chained equations: What is it and how does it work?” International Journal of Methods in Psychiatric Research, 20(1): 40–49. DOI: https://doi.org/10.1002/mpr.329.10.1002/mpr.329307424121499542Search in Google Scholar

Bell, J., Huber, J., and W.K. Viscusi. 2011. “Survey mode effects on valuation of environmental goods.” International Journal of Environmental Research and Public Health, 8(4): 1222–1243. DOI: https://doi.org/10.3390/ijerph8041222.10.3390/ijerph8041222311888621695037Search in Google Scholar

Bianchi, A., Biffgnandi, S., and P. Lynn. 2017. “Web-face-to-face mixed-mode design in a longitudinal survey: Effects on participation rates, sample composition, and costs.” Journal of Official Statistics, 33(2): 385–408. DOI: 10.1515/jos-2017-0019.10.1515/jos-2017-0019Search in Google Scholar

Blackwell, M., S. Iacus, G. King, and G. Porro. 2009. “CEM: Coarsened Exact Matching in Stata.” The Stata Journal, 9: 524–546. DOI: https://doi.org/10.1177/1536867X0900900402.10.1177/1536867X0900900402Search in Google Scholar

Blanke, K., and A. Luiten. 2014. Query on Data Collection for Social Surveys. ESSnet Project “Data Collection for Social Surveys using Multiple Modes”. Available at: http://ec.europa.eu/eurostat/cros/system/files/Query_report_DCSS.pdf_en (accessed October 2019).Search in Google Scholar

Bos, J.T., N.C.G.M. Donders, K.M. Bouwman-Brouwer, and J.W.J.V. der Gulden. 2009. “Work characteristics and determinants of job satisfaction in four age groups: university employees’ point of view.” International Archives of Occupational and Environmental Health, 82(10): 1249–1259. DOI: https://doi.org/10.1007/s00420-009-0451-4.10.1007/s00420-009-0451-4276751819621236Search in Google Scholar

Breiman, L. 2001. “Random Forests”. Machine Learning, 45(1): 5 – 32. DOI: https://doi.org/10.1023/A:1010933404324.10.1023/A:1010933404324Search in Google Scholar

Breiman, L., A. Cutler, A. Liaw, and M. Wiener. 2018. Package randomForest. Available at: https://cran.r-project.org/web/packages/randomForest/randomForest.pdf (accessed February 2021).Search in Google Scholar

Brookhart, M.A., S. Schneeweiss, K.J. Rothman, R.J. Glynn, J. Avorn, and T. Stärmer. 2006. “Variable selection for propensity score models.” American Journal of Epidemiology, 163(12): 1149–1156. DOI: https://doi.org/10.1093/aje/kwj149.10.1093/aje/kwj149151319216624967Search in Google Scholar

Buelens, B., and J.A. van den Brakel. 2015. “Measurement Error Calibration in Mixed-mode Sample Surveys.” Sociological Methods and Research, 44(3): 391–426. DOI: https://doi.org/10.1177/0049124114532444.10.1177/0049124114532444Search in Google Scholar

Couper, M.P. 2011. “The Future of Modes of Data Collection.” Public Opinion Quarterly (5): 889–908. DOI: https://doi.org/10.1093/poq/nfr046.10.1093/poq/nfr046Search in Google Scholar

De Leeuw, E.D. 2005. “To Mix or Not to Mix Data Collection Modes in Surveys.” Journal of Official Statistics, 21(2): 233–255. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/to-mix-or-not-to-mix-data-collection-modes-in-surveys.pdf. (accessed December 2020).Search in Google Scholar

De Leeuw, E.D. 2018. “Mixed-mode: Past, present, and future.” Survey Research Methods, 12(2): 75–89. DOI: https://doi.org/10.18148/srm/2018.vl2i2.7402.Search in Google Scholar

Dillman, D.A. 2017. “The promise and challenge of pushing respondents to the web in mixed-mode surveys.” Survey Methodology, 43(1): 3–30. Available at: http://www.statcan.gc.ca/pub/12-001-x/2017001/article/14836-eng.htm (accessed February 2021).Search in Google Scholar

Dillman, D.A., and L.M. Christian. 2005. “Survey Mode as a Source of Instability in Responses across Surveys.” Field Methods, 17(1): 30 – 52. DOI: https://doi.org/10.1177/1525822X04269550.10.1177/1525822X04269550Search in Google Scholar

Editus Luxembourg. 2018. Available at: https://www.editus.lu/Search in Google Scholar

Eurostat. 2018a. Eu labour force survey – methodology. Available at: https://ec.-europa.eu/eurostat/statistics-explained/index.php/EU_labour_force_survey_-_methodology (Accessed February 2021).Search in Google Scholar

Eurostat. 2018b. High-tech industry and knowledge-intensive services. Available at: https://ec.europa.eu/eurostat/cache/metadata/en/htec_esms.htm (Accessed February 2021).Search in Google Scholar

Fan, W., and Z. Yan. 2010. “Factors affecting response rates of the web survey: A systematic review.” Computers in Human Behavior (2): 132–139. DOI: https://doi.org/10.1016/j.chb.2009.10.015.10.1016/j.chb.2009.10.015Search in Google Scholar

Felderer, B., A. Kirchner, and F. Kreuter. 2019. “The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias.” Journal of Official Statistics, 35(1): 93–115. DOI: https://doi.org/10.2478/jos-2019-0005.10.2478/jos-2019-0005Search in Google Scholar

Fricker, S., M. Galesic, R. Tourangeau, and T. Yan. 2005. “An experimental comparison of web and telephone surveys.” Public Opinion Quarterly, 69(3): 370–392. Available at. https://academic.oup.com/poq/article-abstract/69/3/370/1941667?redirectedFrom=-fulltext (accessed February 2021).10.1093/poq/nfi027Search in Google Scholar

Garrouste, C.L., and M. Rodrigues. 2014. “Employability of young graduates in Europe.” Journal of Manpower, 35(4): 425–447. DOI: https://doi.org/10.1108/IJM-05-2013-0106.10.1108/IJM-05-2013-0106Search in Google Scholar

Grande, E.D., and A.W. Taylor. 2010. “Sampling and coverage issues of telephone surveys used for collecting health information in Australia: results from a face-to-face survey from 1999 to 2008.” BMC Medical Research Methodology, 77(10). DOI: https://doi.org/10.1186/1471-2288-10-77.10.1186/1471-2288-10-77294289420738884Search in Google Scholar

Greene, J., Speizer, H., and W. Wiitala. 2008. “Telephone and web: Mixed-mode challenge.” Health Services Research, 43(l Pl): 230–248. DOI: https://doi.org/10.1111/j.l475-6773.2007.00747.x.Search in Google Scholar

Groves, R.M. 2005. Survey errors and survey costs, (2nd ed.). Hoboken, New Jersey: John Wiley & Sons.Search in Google Scholar

Helppie-McFall, B., and J.W. Hsu. 2017. “A test of web and mail mode effects in a financially sensitive survey of older Americans.” Journal of Economic and Social Measurement, 42(2): 151–169. DOI: https://doi.org/10.3233/JEM-170444.10.3233/JEM-170444597624829861541Search in Google Scholar

Iacus, S.M., G. King, and G. Porro. 2011. “Multivariate Matching Methods That Are Monotonic Imbalance Bounding.” Journal of the American Statistical Association, 106(493): 345–361. DOI: https://doi.org/10.1198/jasa.2011.tm09599.10.1198/jasa.2011.tm09599Search in Google Scholar

Iacus, S.M., G. King, and G. Porro. 2012. “Causal Inference without Balance Checking: Coarsened Exact Matching.” Political Analysis, 20(1): 1–24. DOI: https://doi.org/10.1093/pan/mpr013.10.1093/pan/mpr013Search in Google Scholar

Iacus, S.M., G. King, and G. Porro. 2018. Package cem. Available at: https://cran.r-project.org/web/packages/cem/cem.pdf (accessed February 2021).Search in Google Scholar

ICLS Resolution, 19th. 2013. Resolution concerning statistics of work, employment and labour underutilization. Available at: https://www.ilo.org/wcmsp5/groups/public/—dgreports/—stat/documents/normativeinstrument/wcms_230304.pdf (accessed February 2021).Search in Google Scholar

Imbens, G.W. 2004. “Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A review.” The Review of Economics and Statistics, 86(1): 4–29. DOI: https://doi.org/10.1162/003465304323023651.10.1162/003465304323023651Search in Google Scholar

Körner, T., and A. Liersch. 2014. Case study on mode effects in the German Labour Force Survey. Deliverable for work package III of the ESSnet on Data Collection for Social Surveys Using Multiple Modes. Available at: http://www.cros-portal.eu/content/data-collection (accessed December 2014).Search in Google Scholar

Liaw, A., and M. Wiener. 2002. “Classification and Regression by randomForest.” R News, 2(3): 18–22. Available at: https://www.researchgate.net/profile/Andy_Liaw/publication/228451484_Classification_and_Regression_by_RandomForest/links/53fb24cc0cf20a45497047ab/Classification-and-Regression-by-RandomForest.pdf (accessed February 2021).Search in Google Scholar

Lipps, O., N. Pekari, and C. Roberts. 2013. Coverage and nonresponse errors in an individual register frame based Swiss telephone election study. FORS. Available at: https://serval.unil.ch/resource/serval:BIB_7865820E3A4F.P001/REF.pdf (accessed February 2021).Search in Google Scholar

Long, J.S., and J. Freese. 2006. Regression models for categorical dependent variables using Stata, (2nd edition). Stata press.Search in Google Scholar

Lugtig, P., G.J.L.M. Lensvelt-Mulders, R. Frerichs, and A. Greven. 2011. “Estimating Nonresponse Bias and Mode Effects in a Mixed-Mode Survey.” International Journal of Market Research, 53(5): 1–16. DOI: https://doi.org/10.2501/IJMR-53-5-669-686.10.2501/IJMR-53-5-669-686Search in Google Scholar

Luiten, A., and K. Blanke. 2015. “Conclusions of the ESSnet – DCSS on web and mixed mode data collection in official social surveys.” Conference of New Techniques and Technologies for Statistics (NTTS 9–13 March 2015. Brussels, Belgium. DOI: https://doi.org/10.2901/EUROSTAT.C2015.001.Search in Google Scholar

Manfreda, L.K., M. Bosnjak, J. Berzelak, I. Haas, and V. Vehovar. 2008. “Web Surveys versus other Survey Modes: a Meta-Analysis Comparing Response Rates.” International Journal of Market Research, 50(1): 79–104. DOI: https://doi.org/10.1177/147078530805000107.10.1177/147078530805000107Search in Google Scholar

Mathur, M.B., P. Ding, and T.J. VanderWeele. 2018. Package EValue. Available at: https://cran.r-project.org/web/packages/EValue/EValue.pdf (accessed February 2021).Search in Google Scholar

Meinfelder, F. 2014. “Multiple Imputation: An attempt to retell the evolutionary process.” AStA Wirtschafts- und Sozialstatistisches Archiv, 8(4): 249 – 267. DOI: https://doi.org/10.1007/S11943-014-0151-8.10.1007/s11943-014-0151-8Search in Google Scholar

Mercer, A.W., F. Kreuter, S. Keeter, and E.A. Stuart. 2017. “Theory and Practice in Nonprobability Surveys: Parallels between Causal Inference and Survey Inference.” Public Opinion Quarterly, 81: 250–279. DOI: https://doi.org/10.1093/poq/nfw060.10.1093/poq/nfw060Search in Google Scholar

Morgan, S.L., and C. Winship. 2007. Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge University Press.10.1017/CBO9780511804564Search in Google Scholar

Morris, T.P., I.R. White, and P. Royston. 2014. “Tuning multiple imputation by predictive mean matching and local residual draws.” BMC Medical Research Methodology, 14. DOI: https://doi.org/10.1186/1471-2288-14-75.10.1186/1471-2288-14-75405196424903709Search in Google Scholar

Pohjanpää, K. 2014. The report of web pilot study of LFS (WPIII). Statistics Finland. The ESSnet project on Data Collection for Social Surveys using Multiple Modes. 5 September, 2014, Wiesbaden, Germany. Available at: https://ec.europa.eu/eurostat/cros/content/workshop-%E2%80%9Cdata-collection-social-surveys-using-multiple-modes%E2%80%9D-dcss_en (accessed October 2019).Search in Google Scholar

Riillo, C.A.F. 2017. Beyond the question “Does it pay to be green?”: How much green? and when?” Journal of Cleaner Production, 141: 626–640. DOI: http://dx.doi.org/10.1016/j.jclepro.2016.09.039.10.1016/j.jclepro.2016.09.039Search in Google Scholar

Roberts, C., and C. Vandenplas. 2017. “Estimating Components of Mean Squared Error to Evaluate the Benefits of Mixing Data Collection Modes.” Journal of Official Statistics, 33(2): 303–334. DOI: https://doi.org/10.1515/jos-2017-0016.10.1515/jos-2017-0016Search in Google Scholar

Rosenbaum, P.R. 2005. “Sensitivity Analysis in Observational Studies.” In Encyclopedia of Statistics in Behavioral Science, edited by B.S. Everitt and D.C. Howell, 1451–1462. John Wiley & Sons, Ltd, Chichester.10.1002/0470013192.bsa606Search in Google Scholar

Rosenbaum, P.R., and D.B. Rubin. 1983. “The central role of the propensity score in observational studies for causal effects.” Biometrika, 70(1): 41 – 55. DOI: https://doi.org/10.1093/biomet/70.l.41.Search in Google Scholar

Sarracino, F., C.A.F. Riillo, and M. Mikucka. 2017. “Comparability of web and telephone surveys for the measurement of subjective well-being.” Survey Research Methods, 11(2): 141–169. DOI: https://doi.org/10.18148/srm/2017.vlli2.6740.Search in Google Scholar

Schonlau, M., K. Zapert, L.P. Simon, K. Sanstad, S. Marcus, J. Adams, M. Spranca, H. Kan, R. Turner, and S.H. Berry. 2003. “A Comparison Between Responses From a Propensity-Weighted Web Survey and an Identical RDD Survey.” Social Science Computer Review, 21(10): 1–11. DOI: https://doi.org/10.1177/0894439303256551.10.1177/0894439303256551Search in Google Scholar

Schouten, B., and J. van der Laan. 2014. ESSnet deliverable WPIII: Mode effect decompositions for the Dutch Labour Force Survey. Deliverable for work package III of the ESSnet on Data Collection for Social Surveys Using Multiple Modes. 5 September, 2014, Wiesbaden, Germany. Available at: https://ec.europa.eu/eurostat/cros/content/-workshop-%E2%80%9Cdata-collection-social-surveys-using-multipleQ15modes%E2%80%9D-dcss_en (accessed October 2019).Search in Google Scholar

Sentas, P., and L. Angelis. 2006. “Categorical missing data imputation for software cost estimation by multinomial logistic regression.” Journal of Systems and Software, 79(3): 404–414. DOI: https://doi.org/10.1016/j.jss.2005.02.026.10.1016/j.jss.2005.02.026Search in Google Scholar

Shrier, I., R.W. Platt, and R.J. Steele. 2007. Re: “Variable Selection for Propensity Score Models.” American Journal of Epidemiology, 166(2): 238–239. DOI: https://doi.org/10.1093/aje/kwj149.10.1093/aje/kwj149Search in Google Scholar

Sidney, J.A., C. Coberley, J.E. Pope, and A. Wells. 2015. “Extending coarsened exact matching to multiple cohorts: an application to longitudinal well-being program evaluation within an employer population.” Health Services and Outcomes Research Methodology, 15(2): 136–156. DOI: https://doi.org/10.1007/S10742-014-0136-7.10.1007/s10742-014-0136-7Search in Google Scholar

Sinclair, M., J. O’Toole, M. Malawaraarachchi, and K. Leder. 2012. “Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches.” BMC Medical Research Methodology, 12(1): 132. DOI: https://doi.org/10.1186/1471-2288-12-132.10.1186/1471-2288-12-132Search in Google Scholar

Sousa-Poza, A., and A.A. Sousa-Poza. 2000. “Well-being at work: a cross-national analysis of the levels and determinants of job satisfaction.” The Journal of Socio-Economics, 29(6): 517–538. DOI: https://doi.org/10.1016/S1053-5357(00)00085-8.10.1016/S1053-5357(00)00085-8Search in Google Scholar

Stuart, E.A. 2010. “Matching Methods for Causal Inference: A Review and a Look Forward.” Statistical Science, 25(1): 1–21. DOI: http://dx.doi.org/10.1214/09-STS313.10.1214/09-STS313294367020871802Search in Google Scholar

Ting, Y. 1997. “Determinants of Job Satisfaction of Federal Government Employees.” Public Personnel Management, 26(3): 313–334. DOI: https://doi.org/10.1177/009102609702600302.10.1177/009102609702600302Search in Google Scholar

Tourangeau, R., L.J. Rips, and K. Rasinski. 2000. The psychology of survey response. Cambridge, UK: Cambridge University Press.10.1017/CBO9780511819322Search in Google Scholar

Van Buuren, S. 2012. Flexible imputation of missing data. New York, NY: Taylor & Francis Ltd.10.1201/b11826Search in Google Scholar

Van Buuren, S., and K. Groothuis-Oudshoorn. 2011. “mice: Multivariate Imputation by Chained Equations in R.” Journal of Statistical Software, 45(3). DOI: http://dx.doi.org/10.18637/jss.v045.i03.10.18637/jss.v045.i03Search in Google Scholar

VanderWeele, T.J., and P. Ding. 2017. “Sensitivity Analysis in Observational Research: Introducing the E-value.” Annals of Internal Medicine, 167(4): 268–274. DOI: https://doi.org/10.7326/M16-2607.10.7326/M16-260728693043Search in Google Scholar

Vannieuwenhuyze, J.T.A., G. Loosveldt, and G. Molenberghs. 2014. “Evaluating Mode Effects in Mixed-Mode Survey Data Using Covariate Adjustment Models.” Journal of Official Statistics, 30(1): 1–21. DOI: https://doi.org/10.2478/jos-2014-0001.10.2478/jos-2014-0001Search in Google Scholar

Vila, L.E., and B. Garcia-Mora. 2005. “Education and the Determinants of Job Satisfaction.” Education Economics, 13(4): 409–425. DOI: https://doi.org/10.1080/09645290500251730.10.1080/09645290500251730Search in Google Scholar

Ye, C., Fulton, J., and R. Tourangeau. 2011. “More positive or More Extreme? A Meta-Analysis of Mode Differences in Response Choice.” Public Opinion Quarterly, 75(2). DOI: https://doi.org/10.1093/poq/nfr009.10.1093/poq/nfr009Search in Google Scholar