Psychometric Validation of an Instrument for Measuring Patient Experiences with Outpatient Healthcare

1. Goodwin N. Towards people-centered integrated care: From passive recognition to active co-production. Int J Integr Care. 2016;16(2):15. doi: 10.5334/ijic.2492. GoodwinN Towards people-centered integrated care: From passive recognition to active co-production Int J Integr Care 2016 16 2 15 10.5334/ijic.2492 Open DOISearch in Google Scholar

2. Constand MK, MacDermid JC, Dal Bello-Haas V, Law M. Scoping review of patient-centered care approaches in healthcare. BMC Health Serv Res. 2014;14:271. doi: 10.1186/1472-6963-14-271. ConstandMK MacDermidJC Dal Bello-HaasV LawM Scoping review of patient-centered care approaches in healthcare BMC Health Serv Res 2014 14 271 10.1186/1472-6963-14-271 Open DOISearch in Google Scholar

3. Carinci F, Van Gool K, Mainz J, Veillard J, Pichora EC, Januel JM, et al. Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators. Int J Qual Health Care. 2015;27(2):137–146. doi: 10.1093/intqhc/mzv004. CarinciF Van GoolK MainzJ VeillardJ PichoraEC JanuelJM Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators Int J Qual Health Care 2015 27 2 137 146 10.1093/intqhc/mzv004 Open DOISearch in Google Scholar

4. De Rosis S, Cerasuolo D, Nuti S. Using patient reported measures to drive change in healthcare: The experience of the digital, continuous and systematic PREMs observatory in Italy. BMC Health Serv Res. 2020;20:315. doi: 10.1186/s12913-020-05099-4. De RosisS CerasuoloD NutiS Using patient reported measures to drive change in healthcare: The experience of the digital, continuous and systematic PREMs observatory in Italy BMC Health Serv Res 2020 20 315 10.1186/s12913-020-05099-4 Open DOISearch in Google Scholar

5. Bull C, Byrnes J, Hettiarachchi R, Downes M. A systematic review of the validity and reliability of patient reported experience measures. Health Serv Res. 2019;54(5):1023–1035. doi: 10.1111/1475-6773.13187. BullC ByrnesJ HettiarachchiR DownesM A systematic review of the validity and reliability of patient reported experience measures Health Serv Res 2019 54 5 1023 1035 10.1111/1475-6773.13187 Open DOISearch in Google Scholar

6. Organization for Economic Co-operation and Development. Recommendations to OECD ministers of health from the high-level reflection group on the future of health statistics [Internet]. 2017 [cited 2022 Mar 15]. Available from: https://www.oecd.org/health/Recommendations-from-high-level-reflection-group-on-the-future-of-health-statistics.pdf Organization for Economic Co-operation and Development Recommendations to OECD ministers of health from the high-level reflection group on the future of health statistics [Internet] 2017 [cited 2022 Mar 15]. Available from: https://www.oecd.org/health/Recommendations-from-high-level-reflection-group-on-the-future-of-health-statistics.pdf Search in Google Scholar

7. Fernandes S, Fond G, Zendjidjian X, Michel P, Baumstarck K, Lancon C, et al. The Patient reported experience measure for improving quality of care in mental health (PREMIUM) project in France: Study protocol for the development and implementation strategy. Patient Prefer Adherence. 2019;13:165–177. doi: 10.2147/PPA.S172100. FernandesS FondG ZendjidjianX MichelP BaumstarckK LanconC The Patient reported experience measure for improving quality of care in mental health (PREMIUM) project in France: Study protocol for the development and implementation strategy Patient Prefer Adherence 2019 13 165 177 10.2147/PPA.S172100 Open DOISearch in Google Scholar

8. Fujisawa R, Klazinga N. Measuring patient experiences (PREMs): Progress made by the OECD and its member countries between 2006 and 2016. OECD Health Working Papers, No. 102. OECD Publishing: Paris; 2017. doi: 10.1787/893a07d2-en. FujisawaR KlazingaN Measuring patient experiences (PREMs): Progress made by the OECD and its member countries between 2006 and 2016 OECD Health Working Papers, No. 102. OECD Publishing Paris 2017 10.1787/893a07d2-en Open DOISearch in Google Scholar

9. Sjetne, I, Bjertnaes O, Olsen RV, Iversen HH. The Generic Short Patient Experiences Questionnaire (GS-PEQ): Identification of core items from a survey in Norway. BMC Health Serv Res. 2011(1):88. doi: 10.1186/1472-6963-11-88. SjetneI BjertnaesO OlsenRV IversenHH The Generic Short Patient Experiences Questionnaire (GS-PEQ): Identification of core items from a survey in Norway BMC Health Serv Res 2011 1 88 10.1186/1472-6963-11-88 Open DOISearch in Google Scholar

10. OPEQ, Garratt AM, Bjaertnes OA, Krogstad U, Gulbrandsen P. The OutPatient Experiences Questionnaire (OPEQ): Data quality, reliability, and validity in patients attending 52 Norwegian hospitals. Qual Saf Health Care. 2005;14:433–437. doi: 10.1136/qshc.2005.014423. OPEQ GarrattAM BjaertnesOA KrogstadU GulbrandsenP The OutPatient Experiences Questionnaire (OPEQ): Data quality, reliability, and validity in patients attending 52 Norwegian hospitals Qual Saf Health Care 2005 14 433 437 10.1136/qshc.2005.014423 Open DOISearch in Google Scholar

11. CAHPS, The American Consumer Assessment of Healthcare Providers and Systems (CAHPS) [Internet]. 2012 [cited 2023 April 3]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3748811/pdf/nihms391989.pdf CAHPS The American Consumer Assessment of Healthcare Providers and Systems (CAHPS) [Internet] 2012 [cited 2023 April 3]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3748811/pdf/nihms391989.pdf Search in Google Scholar

12. Delnoij DM, Rademakers JJ, Groenewegen PP. The Dutch Consumer Quality Index: an example of stakeholder involvement in indicator development. BMC Health Serv Res. 2010;10:88. doi: 10.1186/1472-6963-10-88. DelnoijDM RademakersJJ GroenewegenPP The Dutch Consumer Quality Index: an example of stakeholder involvement in indicator development BMC Health Serv Res 2010 10 88 10.1186/1472-6963-10-88 Open DOISearch in Google Scholar

13. OECD. Health at a glance [Internet]. 2021 [cited 2023 April 4]. Available from: https://www.oecd-ilibrary.org/social-issues-migration-health/health-at-a-glance-2021_ae3016b9-en OECD Health at a glance [Internet] 2021 [cited 2023 April 4]. Available from: https://www.oecd-ilibrary.org/social-issues-migration-health/health-at-a-glance-2021_ae3016b9-en Search in Google Scholar

14. Ministrstvo za zdravje. Nacionalna anketa [Internet]. 2007 [cited 2023 April 3]. Available from: https://www.pb-begunje.si/gradiva/Rezultati2007135143657636.pdf Ministrstvo za zdravje Nacionalna anketa [Internet] 2007 [cited 2023 April 3]. Available from: https://www.pb-begunje.si/gradiva/Rezultati2007135143657636.pdf Search in Google Scholar

15. Murko E, Kralj M, Poldrugovac M, Ropret N, Šetinc M, Zaletel M. Izkušnje pacientov z zunajbolnišnično specialistično zdravstveno obravnavo: raziskava v Sloveniji med odraslimi pacienti. Javno zdravje. 2021;1:1–9. doi: 10.26318/JZ-2021-1. MurkoE KraljM PoldrugovacM RopretN ŠetincM ZaletelM Izkušnje pacientov z zunajbolnišnično specialistično zdravstveno obravnavo: raziskava v Sloveniji med odraslimi pacienti Javno zdravje 2021 1 1 9 10.26318/JZ-2021-1 Open DOISearch in Google Scholar

16. Pettersen K, Veenstra M, Guldvog B; Kolstad A. The Patient Experiences Questionnaire: Development, validity and reliability. Int J Quality Health Care. 2004;16(6):453–463. doi: 10.1093/intqhc/mzh074. PettersenK VeenstraM GuldvogB KolstadA The Patient Experiences Questionnaire: Development, validity and reliability Int J Quality Health Care 2004 16 6 453 463 10.1093/intqhc/mzh074 Open DOISearch in Google Scholar

17. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1–55. doi: 10.1080/10705519909540118. HuL BentlerPM Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Struct Equ Modeling 1999 6 1 1 55 10.1080/10705519909540118 Open DOISearch in Google Scholar

18. Bond TG, Fox CM. Applying the Rasch model: Fundamental measurement in the human sciences. 2nd ed. Mahwah: Lawrence Erlbaum Associates Publishers; 2007. BondTG FoxCM Applying the Rasch model: Fundamental measurement in the human sciences 2nd ed. Mahwah Lawrence Erlbaum Associates Publishers 2007 Search in Google Scholar

19. R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2021. R Core Team R: A language and environment for statistical computing Vienna R Foundation for Statistical Computing 2021 Search in Google Scholar

20. Rosseel Y. lavaan: An R package for structural equation modeling. J Stat Softw. 2012;48(2):1–36. doi: 10.18637/jss.v048.i02. RosseelY lavaan: An R package for structural equation modeling J Stat Softw 2012 48 2 1 36 10.18637/jss.v048.i02 Open DOISearch in Google Scholar

21. Kelley K. MBESS: The MBESS R Package. R package version 4.9.0 [Internet]. 2022 [cited 2023 April 3]. Available from: https://CRAN.R-project.org/package=MBESS KelleyK MBESS: The MBESS R Package. R package version 4.9.0 [Internet] 2022 [cited 2023 April 3]. Available from: https://CRAN.R-project.org/package=MBESS Search in Google Scholar

22. Mair P, Hatzinger, R, Maier MJ. eRm: Extended rasch modeling 1.0–2 [Internet]. 2021 [cited 2023 April 3]. Available from: https://cran.r-project.org/package=eRm.2021 MairP HatzingerR MaierMJ eRm: Extended rasch modeling 1.0–2 [Internet] 2021 [cited 2023 April 3]. Available from: https://cran.r-project.org/package=eRm.2021 Search in Google Scholar

23. Mair P, De Leeuw J. aspect: A general framework for multivariate analysis with optimal scaling. R package version 1.0–5. [Internet]. 2018 [cited 2023 April 3]. Available from: https://cran.r-project.org/package=aspect MairP De LeeuwJ aspect: A general framework for multivariate analysis with optimal scaling. R package version 1.0–5. [Internet] 2018 [cited 2023 April 3]. Available from: https://cran.r-project.org/package=aspect Search in Google Scholar

24. Revelle W. psych: Procedures for personality and psychological research, version 2.1.9. [Internet]. 2021 [cited 2023 April 3]. Evanston: Northwestern University; 2021. Available from: https://CRAN.R-project.org/package=psych RevelleW psych: Procedures for personality and psychological research, version 2.1.9. [Internet] 2021 [cited 2023 April 3]. Evanston: Northwestern University; 2021. Available from: https://CRAN.R-project.org/package=psych Search in Google Scholar

25. Stekhoven DJ. missForest: Nonparametric missing value imputation using random forest. R package version 1.5. 2022. StekhovenDJ missForest: Nonparametric missing value imputation using random forest. R package version 1.5 2022 Search in Google Scholar

26. McCoach DB, Gable RK, Madura JP. Instrument development in the affective domain: School and corporate applications. 3rd ed. New York: Springer; 2013. McCoachDB GableRK MaduraJP Instrument development in the affective domain: School and corporate applications 3rd ed. New York Springer 2013 Search in Google Scholar

27. Raykov T, Marcoulides GA. Introduction to psychometric theory. New York: Routledge, Taylor & Francis; 2011. RaykovT MarcoulidesGA Introduction to psychometric theory New York Routledge, Taylor & Francis 2011 Search in Google Scholar

28. Bruster S. The Picker Patient Experience Questionnaire: Development and validation using data from in-patient surveys in five countries. Int J Qual Health Care. 2002;14(5):353–358. doi: 10.1093/intqhc/14.5.353. BrusterS The Picker Patient Experience Questionnaire: Development and validation using data from in-patient surveys in five countries Int J Qual Health Care 2002 14 5 353 358 10.1093/intqhc/14.5.353 Open DOISearch in Google Scholar

29. de Boer D, Delnoij D, Rademakers J. The discriminative power of patient experience surveys. BMC Health Serv Res. 2011;11:332. doi: 10.1186/1472-6963-11-332. de BoerD DelnoijD RademakersJ The discriminative power of patient experience surveys BMC Health Serv Res 2011 11 332 10.1186/1472-6963-11-332 Open DOISearch in Google Scholar

30. Hays RD, Skootsky SA. Patient experience with in-person and telehealth visits before and during the COVID-19 pandemic at a large integrated health system in the United States. J Gen Intern Med. 2022;37(4):847–852. doi: 10.1007/s11606-021-07196-4. HaysRD SkootskySA Patient experience with in-person and telehealth visits before and during the COVID-19 pandemic at a large integrated health system in the United States J Gen Intern Med 2022 37 4 847 852 10.1007/s11606-021-07196-4 Open DOISearch in Google Scholar

31. Damman OC, Hendriks M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP. Consumers’ interpretation and use of comparative information on the quality of health care: the effect of presentation approaches. Health Expect. 2012;15(2):197–211. doi: 10.1111/j.1369-7625.2011.00671. DammanOC HendriksM RademakersJ SpreeuwenbergP DelnoijDM GroenewegenPP Consumers’ interpretation and use of comparative information on the quality of health care: the effect of presentation approaches Health Expect 2012 15 2 197 211 10.1111/j.1369-7625.2011.00671 Open DOISearch in Google Scholar

32. OECD. Measuring what matters: The patient reported indicator surveys: Patient reported indicators for assessing health system performance [Internet]. 2019 [cited 2023 April 17]. Available from: https://www.oecd.org/health/health-systems/Measuring-what-matters-the-Patient-Reported-Indicator-Surveys.pdf OECD Measuring what matters: The patient reported indicator surveys: Patient reported indicators for assessing health system performance [Internet] 2019 [cited 2023 April 17]. Available from: https://www.oecd.org/health/health-systems/Measuring-what-matters-the-Patient-Reported-Indicator-Surveys.pdf Search in Google Scholar

33. Van der Veer SN, Jager KJ, Visserman E, Beekman RJ, Boeschoten EW, de Keizer NF, et al. Development and validation of the Consumer Quality index instrument to measure the experience and priority of chronic dialysis patients. Nephrol Dial Transplant. 2012;27(8):3284–3291. doi: 10.1093/ndt/gfs023. Van der VeerSN JagerKJ VissermanE BeekmanRJ BoeschotenEW de KeizerNF Development and validation of the Consumer Quality index instrument to measure the experience and priority of chronic dialysis patients Nephrol Dial Transplant 2012 27 8 3284 3291 10.1093/ndt/gfs023 Open DOISearch in Google Scholar

34. NHS, Care Quality Commission. Adult inpatient survey: quality and methodology report [Internet]. 2021 [cited 2023 April 17]. Available from: https://www.cqc.org.uk/sites/default/files/2022-09/20220920_aip22_QualityMethodology.odt NHS, Care Quality Commission Adult inpatient survey: quality and methodology report [Internet] 2021 [cited 2023 April 17]. Available from: https://www.cqc.org.uk/sites/default/files/2022-09/20220920_aip22_QualityMethodology.odt Search in Google Scholar

35. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805–1806. doi: 10.1001/jama.2012.3532. JohnsonTP WislarJS Response rates and nonresponse errors in surveys JAMA 2012 307 17 1805 1806 10.1001/jama.2012.3532 Open DOISearch in Google Scholar

36. Faraz A, Burt J, Roland M. Measuring patient experience: Concepts and methods. Patient. 2014;7:235–241. doi: 10.1007/s40271-014-0060-5. FarazA BurtJ RolandM Measuring patient experience: Concepts and methods Patient 2014 7 235 241 10.1007/s40271-014-0060-5 Open DOISearch in Google Scholar

37. Garratt AM, Bjørngård JH, Dahle KA, et al. The Psychiatric Out-Patient Experiences Questionnaire (POPEQ): Development, reliability and validity. Nordic J Psychiatry (in press). GarrattAM BjørngårdJH DahleKA The Psychiatric Out-Patient Experiences Questionnaire (POPEQ): Development, reliability and validity Nordic J Psychiatry (in press). Search in Google Scholar

38. Millsap RE. Statistical approaches to measurement invariance. New York: Routledge; 2011. MillsapRE Statistical approaches to measurement invariance New York Routledge 2011 Search in Google Scholar