Testing a Planned Missing Design to Reduce Respondent Burden in Web and SMS Administrations of the CAHPS Clinician and Group Survey (CG-CAHPS)

Ahmad, F., A.K. Jhajj, D.E. Stewart, M. Burghardt, and A.S. Bierman. 2014. “Single Item Measures of Self-Rated Mental Health: A Scoping Review.” BMC Health Services Research 14: 398. DOI: https://doi.org/10.1186/1472-6963-14-398.417716525231576 Search in Google Scholar

AHRQ, Agency for Healthcare Research and Quality. n.d. “How to Report Results of the CAHPS Clinician & Group Survey.” Available at: https://www.ahrq.gov/sites/default/-files/wysiwyg/cahps/surveys-guidance/cg/cgkit/HowtoReportResultsofCGCAHPS080610FINAL.pdf (accessed September 2021). Search in Google Scholar

Alemi, F., and H.M. Jasper. 2014. “An Alternative to Satisfaction Surveys: Let the Patients Talk.” Quality Management in Health Care 23(1): 10–19. DOI: https://doi.org/10.1097/QMH.0000000000000014.24368718 Search in Google Scholar

Antoun, C., and A. Cernat. 2020. “Factors Affecting Completion Times: A Comparative Analysis of Smartphone and PC Web Surveys.” Social Science Computer Review 38(4): 477–489. DOI: https://doi.org/10.1177/0894439318823703. Search in Google Scholar

Ayers, R., S. Brothers, and W. Jansen. 2014. “Guidelines on Mobile Device Forensics.” National Institute of Standards and Technology, Special Publication 800-101r1. Washington, DC: US Department of Commerce. DOI: http://dx.doi.org/10.6028/NIST.Sp.800-101r1.10.6028/NIST.SP.800-101r1 Search in Google Scholar

Blumberg, S.J., and J.V. Luke. 2020. “Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December 2019.” National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/nhis.htm. (accessed September 2021).10.15620/cdc:108678 Search in Google Scholar

Bosnjak, M., W. Neibarth, M.P. Couper, W. Bandilla, and L. Kaczmirek. 2008. “Prenotification in Web-Based Access Panel Surveys.” Social Science Computer Review 26(2): 213–223. DOI: https://doi.org/10.1177/0894439307305895. Search in Google Scholar

Bradburn, N. 1978. “Respondent Burden.” In Proceedings of the Survey Research Methods Section of the American Statistical Association: 35–40. Alexandria, VA: American Statistical Association. Available at: http://www.asasrms.org/Proceedings/y1978f.html (accessed September 2021). Search in Google Scholar

Brenner, P.S., and J.D. DeLamater. 2013. “Paradata Correlates of Data Quality in an SMS Time Use Study–Evidence from a Validation Study.” Electronic International Journal of Time Use Research 10(1): 38–54. DOI: dx.doi.org/10.13085/eIJTUR.10.1.38-54.10.13085/eIJTUR.10.1.38-54 Search in Google Scholar

Conrad, FG., M.P. Couper, R. Tourangeau, and C. Zhang. 2017. “Reducing Speeding in Web Surveys by Providing Immediate Feedback.” Survey Research Methods 11(1): 45–61. DOI: https://doi.org/10.18148/srm/2017.v11i1.6304.686351531745400 Search in Google Scholar

Crawford, S.D., M.P. Couper, and M.J. Lamias. 2001. “Web Surveys: Perception of Burden.” Social Science Computer Review 19(2): 146–162. DOI: https://doi.org/10.1177/089443930101900202. Search in Google Scholar

De Leeuw, E.D. 2018. “Mixed-Mode: Past, Present, and Future.” Survey Research Methods 12(2): 75–89. DOI: doi: 10.18148/srm/2018.v12i2.7402. Open DOISearch in Google Scholar

DeSalvo, K.B., V.S. Fan, M.B. McDonell, and S.D. Fihn. 2004. “Predicting Mortality and Healthcare Utilization with a Single Question.” Health Services Research 40(4): 1234–1246. DOI: https://doi.org/10.1111/j.1475-6773.2005.00404.x.136119016033502 Search in Google Scholar

Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: Wiley. Search in Google Scholar

Dyer, N., J.S. Sorra, S.A. Smith, P. Cleary, and R. Hays. 2012. “Psychometric Properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group Adult Visit Survey.” Medical Care 50: S28. DOI: https://doi.org/10.1097/MLR.0b013e31826cbc0d.348067123064274 Search in Google Scholar

Elliott M.N., M.K. Beckett, W.G. Lehrman, P. Cleary, C.W. Cohea, L.A. Giordano, E.H. Goldstein, and C.L. Damberg. 2016. “Understanding The Role Played By Medicare’s Patient Experience Points System In Hospital Reimbursement.” Health Affairs 35(9): 1673–1680. DOI: https://doi.org/10.1377/hlthaff.2015.0691.27605650 Search in Google Scholar

Elliott, M.N., A.M. Zaslavsky, E. Goldstein, W. Lehrman, K. Hambarsoomians, M.K. Beckett, and L. Giordano. 2009. Effects of survey mode, patient mix, and nonresponse on CAHPSw hospital survey scores. Health Services Research, 44 (2p1): 501–518. DOI: https://doi.org/10.1111/j.1475-6773.2008.00914.x.267705119317857 Search in Google Scholar

Enders, C.K. 2010. Applied Missing Data Analysis. New York: Guilford. Search in Google Scholar

Galesic, M., and M. Bosnjak. 2009. “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey.” Public Opinion Quarterly 73(2): 349–360. DOI: https://doi.org/10.1093/poq/nfp031. Search in Google Scholar

Garrido, T., B. Raymond, and B. Wheatley. 2016. “Lessons From More than a Decade In patient Portals.” Health Affairs Blog. DOI: https://doi.org/10.1377/hblog20160407.054362. Search in Google Scholar

Graham, J.W., B.J. Taylor, A.E. Olchowski, and P.E. Cumsille. 2006. “Planned Missing Data Designs in Psychological Research.” Psychological Methods 11(4): 323–343. DOI: https://doi.org/10.1037/1082-989X.11.4.323.17154750 Search in Google Scholar

Hillygus, D.S., N. Jackson, and M. Young. 2014. “Professional respondents in non-probability online panels.” In Online panel research: A data quality perspective, edited by M. Callegaro, R. Baker, J. Bethlehem, A.S. Góritz, J.A. Krosnick and P.J. Lavrakas: 219–237. New York: Wiley.10.1002/9781118763520.ch10 Search in Google Scholar

Holt, J.M. 2019. “Patient Experience in Primary Care: A Systematic Review of CGCAHPS Surveys.” Journal of Patient Experience 6(2): 93–102. DOI: https://doi.org/10.1177/2374373518793143.655894431218253 Search in Google Scholar

Johnson, D.R., V. Roth, and R. Young. 2013. “Planned Missing Data Designs in Health Surveys.” In Proceedings of the Tenth Conference on Health Survey Research Methods, November 2013: 43–53. Hyattsville, MD: Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/hsrmc/hsrmc_10th_proceedings_2011.pdf (accessed February 2020). Search in Google Scholar

Keller, S., A.J. O’Malley, R.D. Hays, R.A. Matthew, A.M. Zaslavsky, K.A. Hepner, and P.D. Cleary. 2005. “Methods Used to Streamline the CAHPSw Hospital Survey.” Health Services Research, 40: 2057–2077. DOI: https://doi.org/10.1111/j.1475-6773.2005.00478.x.136124816316438 Search in Google Scholar

Kim, Y., J. Dykema, J. Stevenson, P. Black, and D.P. Moberg. 2019. “Straightlining: Overview of Measurement, Comparison of Indicators, and Effects in Mail–Web Mixed-Mode Surveys.” Social Science Computer Review 37(2): 214–233. DOI: https://doi.org/10.1177/0894439317752406. Search in Google Scholar

Lau, C.Q., H. Sanders, and A. Lombaard. 2019. “Questionnaire Design in Short Message Service (SMS) Surveys.” Field Methods 31(3): 214–229. DOI: https://doi.org/10.1177/1525822X19862506. Search in Google Scholar

Lee, S.S.S., X. Xin, W.P. Lee, E.J. Sim, B. Tan, M.P.G. Bien, A.S.T. Lau, and J. Thumboo. 2013. “The Feasibility of Using SMS as a Health Survey Tool: An Exploratory Study in Patients with Rheumatoid Arthritis.” International Journal of Medical Informatics 82(5): 427–434. DOI: https://doi.org/10.1016/j.ijmedinf.2012.12.003.23332386 Search in Google Scholar

Mavletova, A. and M.P. Couper. 2014. “Mobile Web Survey Design: Scrolling versus Paging, SMS versus E-mail Invitations.” Journal of Survey Statistics and Methodology 2: 498–518. DOI: https://doi.org/10.1093/jssam/smu015. Search in Google Scholar

Mavletova, A., and M.P. Couper. 2015. “A Meta-Analysis of Breakoff Rates in Mobile Web Surveys.” In Mobile Research Methods: Opportunities and Challenges of Mobile Research Methodologies, edited by D. Toninelli, R. Pinter, and P. de Pedraza: 81–98. London: Ubiquity Press. Search in Google Scholar

Mavletova, A., M.P. Couper, and D. Lebedev. 2018. “Grid and Item-by-Item Formats in PC and Mobile Web Surveys.” Social Science Computer Review 36(6):647-668. DOI: https://doi.org/10.1177/0894439317735307. Search in Google Scholar

Morgan, S.L., and E.S. Taylor Poppe. 2015. “A Design and a Model For Investigating the Heterogeneity of Context Effects in Public Opinion Surveys.” Sociological Methodology 45(1): 184–222. DOI: https://doi.org/10.1177/0081175015570095. Search in Google Scholar

Ng, J.H., E. Henry, T. Oberlander, P. Shi, S.H. Scholle. 2016. “Shortening a Patient Experiences Survey for Medical Homes.” Healthcare 4:1. DOI: https://doi.org/10.3390/healthcare4010001.493453527417589 Search in Google Scholar

Paddison, C.A.M., G.A. Abel, M.O. Roland, M.N. Elliott, G. Lyratzopoulos, and J.L. Campbell. 2013. “Drivers of Overall Satisfaction with Primary Care: Evidence from the English General Practice Patient Survey.” Health Expectations 18: 1081-1092. DOI: https://doi.org/10.1111/hex.12081.506081223721257 Search in Google Scholar

Pew Research Center. 2019. Mobile Technology and Home Broadband. Available at: https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2019/06/PI_2019.06.13_Mobile-Technology-and-Home-Broadband_FINAL2.pdf. (accessed September 2021). Search in Google Scholar

Peytchev, A. 2009. “Survey Breakoff.” Public Opinion Quarterly 73(1): 74–97. DOI: https://doi.org/10.1093/poq/nfp014. Search in Google Scholar

Peytchev, A. and E. Peytcheva. 2017. “Reduction of Measurement Error due to Survey Length: Evaluation of the Split Questionnaire Design Approach.” Survey Research Methods 11(4): 361–368. DOI: https://doi.org/10.18148/srm/2017.v11i4.7145. Search in Google Scholar

Revilla, M. and C. Ochoa. 2017. “Ideal and Maximum Length for a Web Survey.” International Journal of Market Research 59(5): 557–565. DOI: https://doi.org/10.2501/IJMR-2017-039. Search in Google Scholar

Rhemtulla, M., V. Savalei, and T.D. Little. 2016. “On the Asymptotic Relative Efficiency of Planned Missingness Designs.” Psychometrika 81(1): 60–89. DOI: https://doi.org/10.1007/s11336-014-9422-0.25217133 Search in Google Scholar

Schober, M.F., F.G. Conrad, C. Antoun, P. Ehlen, S. Fail, A.L. Hupp, M. Johnston, L. Vickers, H.Y. Yan, and C. Zhang. 2015. “Precision and Disclosure in Text and Voice Interviews on Smartphones.” PLoSONE 10. DOI: https://doi.org/10.1371/journal.pone.0128337.446518426060991 Search in Google Scholar

Schonlau, M. and M. Couper. 2017. “Options for Conducting Web Surveys.” Statistical Science 32(2): 279–292. DOI: https://doi.org/10.1214/16-STS597. Search in Google Scholar

Sharp, L.M., and J. Frankel. 1983. “Respondent Burden: A Test of Some Common Assumptions.” Public Opinion Quarterly 47: 36–53. DOI: https://doi.org/10.1086/268765. Search in Google Scholar

Spranca, M., D.E. Kanouse, M. Elliott, P.F. Short, D.O. Farley, and R.D. Hays. 2000. “Do Consumer Reports of Health Plan Quality Affect Health Plan Selection?” Health Services Review 35(5): 933–947. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1089177. (accessed September 2021). Search in Google Scholar

Steinbrecher, M, J. Roßmann, and J.E. Blumenstiel. 2015. “Why Do Respondents Break Off Web Surveys and Does it Matter? Results from Four Follow-up Surveys.” International Journal of Public Opinion Research 27(2): 289–302. DOI: https://doi.org/10.1093/ijpor/edu025. Search in Google Scholar

Stucky, B.D., R.D. Hays, M.O. Edelen, J. Gurvey, and J.A. Brown. 2016. “Possibilities for Shortening the CAHPS Clinician and Group Survey.” Medical Care 54(1): 32–37. DOI: https://doi.org/10.1097/MLR.0000000000000452.26536332 Search in Google Scholar

Swain, M.S. 2015. “The Effects of a Planned Missingness Design on Examinee Motivation and Psychometric Quality.” PhD Dissertation, James Madison University. Available at: https://www.proquest.com/docview/1685390254 (accessed September 2021). Search in Google Scholar

Toepoel, V., and P. Lugtig. 2018. “Modularization in an Era of Mobile Web: Investigating the Effects of Cutting a Survey Into Smaller Pieces on Data Quality.” Social Science Computer Review: 1–15. DOI: https://doi.org/10.1177/0894439318784882. Search in Google Scholar

Toepoel, V., M. Das, and A. van Soest. 2008. “Effects of Design in Web Surveys: Comparing Trained and Fresh Respondents.” Public Opinion Quarterly 72(5): 985–1007. DOI: https://doi.org/10.1093/poq/nfn060. Search in Google Scholar

Villar, A., M. Callegaro, and Y. Yang. 2013. “Where am I? A meta-analysis of experiments on the effects of progress indicators for web surveys.” Social Science Computer Review 31(6): 744–762. DOI: https://doi.org/10.1177/0894439313497468. Search in Google Scholar

West, B.T., D. Ghimire, and W.G. Axinn. 2015. “Evaluating a Modular Design Approach to Collecting Survey Data Using Text Messages.” Survey Research Methods 9:111–123. DOI: https://doi.org/10.18148/srm/2015.v9i2.6135. Search in Google Scholar

Yan, T., S. Fricker, and S. Tsai. 2020. “Response Burden: What Is It and What Predicts It?” In Advances in Questionnaire Design, Development, Evaluation, and Testing, edited by P.C. Beatty, D. Collins, L. Kaye, J.L. Padilla, G. B. Willis, and A. Wilmot: 193–212. New York: Wiley.10.1002/9781119263685.ch8 Search in Google Scholar

Zhang, C., and F.G. Conrad. 2018. “Intervening to Reduce Satisficing Behaviors in Web Surveys: Evidence From Two Experiments on How it Works.” Social Science Computer Review 36(1): 57–81. DOI: https://doi.org/10.1177/0894439316683923 Search in Google Scholar