Truthfulness in patient-reported outcomes: factors affecting patients’ responses and impact on data quality

References

• Rothrock NE, Kaiser KA, Cella D. Developing a valid patient-reported outcome measure. Clin Pharmacol Ther. 2011;90(5):737–742. doi:10.1038/clpt.2011.19521975345
   PubMed Web of Science ®Google Scholar
• Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): conceptual framework and item selection. Med Care. 1992;30(6):473–483.1593914
   PubMed Web of Science ®Google Scholar
• Jones N, Jones SL, Miller NA. The medicare health outcomes survey program: overview, context, and near-term prospects. Health Qual Life Outcomes. 2004;2:33. doi:10.1186/1477-7525-2-3315248895
   PubMedGoogle Scholar
• Bass M, Morris S, Neapolitan R. Utilizing multidimensional computer adaptive testing to mitigate burden with patient reported outcomes. AMIA Annu Symp Proc. 2015;2015:320–328.26958163
   PubMedGoogle Scholar
• Quinten C, Maringwa J, Gotay CC, et al. Patient self-reports of symptoms and clinician ratings as predictors of overall cancer survival. J Natl Cancer Inst. 2011;103(24):1851–1858. doi:10.1093/jnci/djr48522157640
   PubMedGoogle Scholar
• Johnston BC, Patrick DL, Busse JW, Schünemann HJ, Agarwal A, Guyatt GH. Patient-reported outcomes in meta-analyses–part 1: assessing risk of bias and combining outcomes. Health Qual Life Outcomes. 2013;11(1):109. doi:10.1186/1477-7525-11-8323815754
   PubMed Web of Science ®Google Scholar
• Locklear T, DeBar LL, Willig J, et al. Case studies from the clinic: initiating and implementing patient-reported outcome measures. EGEMS (Wash DC). 2017;5(1):7.
   Google Scholar
• Cella DF, Hahn EA, Jensen SE, et al. Patient-Reported Outcomes in Performance Measurement. Research Triangle Park, NC: RTI Press/RTI International; 2015.
   Google Scholar
• Vodicka E, Kim K, Devine EB, Gnanasakthy A, Scoggins JF, Patrick DL. Inclusion of patient-reported outcome measures in registered clinical trials: evidence from ClinicalTrials.gov (2007–2013). Contemp Clin Trials. 2015;43:1–9. doi:10.1016/j.cct.2015.04.00425896116
   PubMed Web of Science ®Google Scholar
• 112th Congress (2011–2012). S.3187 food and drug administration safety and innovation act; 2012 Available from: https://www.congress.gov/bill/112th-congress/senate-bill/3187. Accessed 1210, 2018.
   Google Scholar
• CDER Patient-Focused Drug Development [Website on the Internet]. Silver Spring: United States Food and Drug Administration; 2018 Available from: https://www.fda.gov/drugs/developmentapprovalprocess/ucm579400.htm. Accessed 417, 2019.
   Google Scholar
• United States Food and Drug Administration. Guidance for industry patient-reported outcome measures: use in medical product development to support labeling claims; 2009 Available from: https://www.fda.gov/downloads/drugs/guidances/ucm193282.pdf. Accessed 1210, 2018.
   Google Scholar
• DeMuro C, Clark M, Mordin M, Fehnel S, Copley-Merriman C, Gnanasakthy A. Reasons for rejection of patient-reported outcome label claims: a compilation based on a review of patient-reported outcome use among new molecular entities and biologic license applications, 2006–2010. Value Health. 2012;15(3):443–448. doi:10.1016/j.jval.2012.01.01022583454
   PubMed Web of Science ®Google Scholar
• Fallowfield L, Payne H, Jenkins V. Patient-reported outcomes in metastatic castration-resistant prostate cancer. Nat Rev Clin Oncol. 2016;13(10):643–650. doi:10.1038/nrclinonc.2016.10027349193
   PubMed Web of Science ®Google Scholar
• European Medicines Agency. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products; 2005 Available from: https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-regulatory-guidance-use-healthrelated-quality-life-hrql-measures-evaluation_en.pdf. Accessed 417, 2019.
   Google Scholar
• Dowling NM, Bolt DM, Deng S, Li C. Measurement and control of bias in patient reported outcomes using multidimensional item response theory. BMC Med Res Methodol. 2016;16(1):63. doi:10.1186/s12874-016-0161-z27229310
   PubMed Web of Science ®Google Scholar
• Van Vaerenbergh Y, Thomas TD. Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int J Public Opin Res. 2013;25(2):195–217. doi:10.1093/ijpor/eds021
   Web of Science ®Google Scholar
• Brossart DF, Clay DL, Wilson VL. Methodological and statistical considerations for threats to internal validity in pediatric outcome data: response shift in self-report outcomes. J Pediatr Psychol. 2002;27(1):97–107.11726684
   PubMed Web of Science ®Google Scholar
• Sandler KA, Mitchell SA, Basch E, et al. Content validity of anatomic site-specific patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE) item sets for assessment of acute symptomatic toxicities in radiation oncology. Int J Radiat Oncol Biol Phys. 2018;102(1):44–52. doi:10.1016/j.ijrobp.2018.04.04830102201
   PubMed Web of Science ®Google Scholar
• Basch E, Abernethy AP, Reeve BB. Assuring the patient centeredness of patient-reported outcomes: content validity in medical product development and comparative effectiveness research. Value Health. 2011;14(8):965–966. doi:10.1016/j.jval.2011.10.00222152164
   PubMed Web of Science ®Google Scholar
• Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1–eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967–977. doi:10.1016/j.jval.2011.06.01422152165
   PubMed Web of Science ®Google Scholar
• Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2–assessing respondent understanding. Value Health. 2011;14(8):978–988. doi:10.1016/j.jval.2011.06.01322152166
   PubMed Web of Science ®Google Scholar
• Wiklund I, Dimenäs E, Wahl M. Factors of importance when evaluating quality of life in clinical trials. Control Clin Trials. 1990;11(3):169–179.2163812
   PubMedGoogle Scholar
• Owolabi MO. Which is more valid for stroke patients: generic or stroke-specific quality of life measures? Neuroepidemiology. 2010;34(1):8–12. doi:10.1159/00025546019893323
   PubMed Web of Science ®Google Scholar
• Schifferdecker KE, Yount SE, Kaiser K, et al. A method to create a standardized generic and condition-specific patient-reported outcome measure for patient care and healthcare improvement. Qual Life Res. 2018;27(2):367–378. doi:10.1007/s11136-017-1675-528795261
   PubMed Web of Science ®Google Scholar
• Gries K, Berry P, Harrington M, et al. Literature review to assemble the evidence for response scales used in patient-reported outcome measures. J Patient Rep Outcomes. 2018;2(1):41. doi:10.1186/s41687-018-0056-330238086
   PubMedGoogle Scholar
• Mutebi A, Slack M, Warholak TL, Hudgens S, Coons SJ. Interpretation of verbal descriptors for response options commonly used in verbal rating scales in patient-reported outcome instruments. Qual Life Res. 2016;25(12):3181–3189. doi:10.1007/s11136-016-1333-327294436
   PubMed Web of Science ®Google Scholar
• Babakus E, Mangold WG. Adapting the SERVQUAL scale to hospital services: an empirical investigation. Health Serv Res. 1992;26(6):767–786.1737708
   PubMed Web of Science ®Google Scholar
• Finstad K. Response interpolation and scale sensitivity: evidence against 5-point scales. J Usability Stud. 2010;5(3):104–110.
   Google Scholar
• Stull DE, Leidy NK, Parasuraman B, Chassany O. Optimal recall periods for patient-reported outcomes: challenges and potential solutions. Curr Med Res Opin. 2009;25(4):929–942. doi:10.1185/0300799090277476519257798
   PubMed Web of Science ®Google Scholar
• McPhail S, Haines T. Response shift, recall bias and their effect on measuring change in health-related quality of life amongst older hospital patients. Health Qual Life Outcomes. 2010;8(1):65. doi:10.1186/1477-7525-8-6520618978
   PubMed Web of Science ®Google Scholar
• Reilly MC, Bracco A, Ricci J-F, Santoro J, Stevens T. The validity and accuracy of the Work Productivity And Activity Impairment questionnaire–irritable bowel syndrome version (WPAI:IBS). Aliment Pharmacol Ther. 2004;20(4):459–467. doi:10.1111/j.1365-2036.2004.02091.x15298641
   PubMed Web of Science ®Google Scholar
• Gagnier JJ, Lai J, Robbins C Bias arising from the use of patient-reported outcome measures [abstract]. Proceedings from the 8th International Congress on Peer Review and Scientific Publication; 9 10–12; 2017; Chicago, IL.
   Google Scholar
• Hawkins M, Elsworth GR, Osborne RH. Application of validity theory and methodology to patient-reported outcome measures (PROMs): building an argument for validity. Qual Life Res. 2018;27(7):1695–1710. doi:10.1007/s11136-018-1815-629464456
   PubMed Web of Science ®Google Scholar
• Cappelleri JC, Jason Lundy J, Hays RD. Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clin Ther. 2014;36(5):648–662. doi:10.1016/j.clinthera.2014.04.00624811753
   PubMed Web of Science ®Google Scholar
• Hobart J, Cano S, Baron R, et al. Achieving valid patient-reported outcomes measurement: a lesson from fatigue in multiple sclerosis. Mult Scler. 2013;19(13):1773–1783. doi:10.1177/135245851348337823574800
   PubMed Web of Science ®Google Scholar
• Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD. What Is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health. 2007;10:S94–S105. doi:10.1111/j.1524-4733.2007.00272.x17995479
   PubMed Web of Science ®Google Scholar
• Brazier JE, Harper R, Munro J, Walters SJ, Snaith ML. Generic and condition-specific outcome measures for people with osteoarthritis of the knee. Rheumatology (Oxford). 1999;38(9):870–877.10515649
   PubMed Web of Science ®Google Scholar
• Sayers A, Wylde V, Lenguerrand E, et al. A unified multi-level model approach to assessing patient responsiveness including; return to normal, minimally important differences and minimal clinically important improvement for patient reported outcome measures. BMJ Open. 2017;7(7):e014041. doi:10.1136/bmjopen-2016-014041
   PubMed Web of Science ®Google Scholar
• Grøvle L, Haugen AJ, Hasvik E, Natvig B, Brox JI, Grotle M. Patients’ ratings of global perceived change during 2 years were strongly influenced by the current health status. J Clin Epidemiol. 2014;67(5):508–515. doi:10.1016/j.jclinepi.2013.12.00124598378
   PubMed Web of Science ®Google Scholar
• Coon CD, Cook KF. Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores. Qual Life Res. 2018;27(1):33–40. doi:10.1007/s11136-017-1616-328620874
   PubMed Web of Science ®Google Scholar
• United States Food and Drug Administration. Discussion document for patient-focused drug development workshop on guidance 3; 2018 Available from: https://www.fda.gov/downloads/Drugs/NewsEvents/UCM620708.pdf. Accessed 417, 2019.
   Google Scholar
• Oczkowski C, O’Donnell M. Reliability of proxy respondents for patients with stroke: a systematic review. J Stroke Cerebrovasc Dis. 2010;19(5):410–416. doi:10.1016/j.jstrokecerebrovasdis.2009.08.00220554222
   PubMed Web of Science ®Google Scholar
• Sonder JM, Holman R, Knol DL, Bosma LVAE, Polman CH, Uitdehaag BMJ. Analyzing differences between patient and proxy on patient reported outcomes in multiple sclerosis. J Neurol Sci. 2013;334(1–2):143–147. doi:10.1016/j.jns.2013.08.01424018128
   PubMed Web of Science ®Google Scholar
• Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281–291. doi:10.1093/pubmed/fdi03115870099
   PubMed Web of Science ®Google Scholar
• Tourangeau R, Smith TW. Asking sensitive questions: the impact of data collection mode, question format, and question context. Public Opin Q. 1996;60(2):275–304. doi:10.1086/297751
   Web of Science ®Google Scholar
• Puhan MA, Ahuja A, Van Natta ML, Ackatz LE, Meinert C. The studies of ocular complications of AIDS research group. Interviewer versus self-administered health-related quality of life questionnaires - does it matter? Health Qual Life Outcomes. 2011;9(1):30. doi:10.1186/1477-7525-9-4021554737
   PubMed Web of Science ®Google Scholar
• Coons SJ, Eremenco S, Lundy JJ, O’Donohoe P, O’Gorman H, Malizia W. Capturing patient-reported outcome (PRO) data electronically: the past, present, and promise of ePRO measurement in clinical trials. Patient. 2015;8(4):301–309. doi:10.1007/s40271-014-0090-z25300613
   PubMedGoogle Scholar
• Morris S, Bass M, Lee M, Neapolitan RE. Advancing the efficiency and efficacy of patient reported outcomes with multivariate computer adaptive testing. J Am Med Inform Assoc. 2017;24(5):897–902. doi:10.1093/jamia/ocx00328444397
   PubMed Web of Science ®Google Scholar
• Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behav Res Methods. 2015;47(4):1237–1259. doi:10.3758/s13428-014-0533-425410404
   PubMed Web of Science ®Google Scholar
• Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11(2):322–333. doi:10.1111/j.1524-4733.2007.00231.x18380645
   PubMed Web of Science ®Google Scholar
• Muehlhausen W, Doll H, Quadri N, et al. Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual Life Outcomes. 2015;13:167. doi:10.1186/s12955-015-0362-x26446159
   PubMed Web of Science ®Google Scholar
• Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health. 2009;12(4):419–429. doi:10.1111/j.1524-4733.2008.00470.x19900250
   PubMed Web of Science ®Google Scholar
• Ivey SL, Shortell SM, Rodriguez HP, Wang YE. Patient engagement in ACO practices and patient-reported outcomes among adults with co-occurring chronic disease and mental health conditions. Med Care. 2018;56(7):551–556. doi:10.1097/MLR.000000000000092729762273
   PubMed Web of Science ®Google Scholar
• Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44(2 Pt 1):501–518. doi:10.1111/j.1475-6773.2008.00914.x19317857
   PubMed Web of Science ®Google Scholar
• Teresi JA, Ramirez M, Lai J-S, Silver S. Occurrences and sources of differential item functioning (DIF) in patient-reported outcome measures: description of DIF methods, and review of measures of depression, quality of life and general health. Psychol Sci Q. 2008;50(4):538.20165561
   PubMedGoogle Scholar
• Fukuhara S, Bito S, Green J, Hsiao A, Kurokawa K. Translation, adaptation, and validation of the SF-36 health survey for use in Japan. J Clin Epidemiol. 1998;51(11):1037–1044.9817121
   PubMed Web of Science ®Google Scholar
• McKenna SP, Doward LC. The translation and cultural adaptation of patient-reported outcome measures. Value Health. 2005;8(2):89–91. doi:10.1111/j.1524-4733.2005.08203.x15804316
   PubMed Web of Science ®Google Scholar
• Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8(2):94–104. doi:10.1111/j.1524-4733.2005.04054.x15804318
   PubMed Web of Science ®Google Scholar
• Regnault A, Herdman M. Using quantitative methods within the Universalist model framework to explore the cross-cultural equivalence of patient-reported outcome instruments. Qual Life Res. 2015;24(1):115–124. doi:10.1007/s11136-014-0722-824894383
   PubMed Web of Science ®Google Scholar
• Alonso J, Bartlett SJ, Rose M, et al. The case for an international patient-reported outcomes measurement information system (PROMIS®) initiative. Health Qual Life Outcomes. 2013;11:210. doi:10.1186/1477-7525-11-8324359143
   PubMed Web of Science ®Google Scholar
• Chen C, Lee S-Y, Stevenson HW. Response style and cross-cultural comparisons of rating scales among East Asian and North American students. Psychol Sci. 1995;6(3):170–175. doi:10.1111/j.1467-9280.1995.tb00327.x
   Web of Science ®Google Scholar
• Wetzel E, Böhnke JR, Carstensen CH, Ziegler M, Ostendorf F. Do individual response styles matter?: assessing differential item functioning for men and women in the NEO-PI-R. J Individ Differ. 2013;34(2):69–81. doi:10.1027/1614-0001/a000102
   Web of Science ®Google Scholar
• Adams J, Chapman J, Bradley S, Ryan SJ. Literacy levels required to complete routinely used patient-reported outcome measures in rheumatology. Rheumatology (Oxford). 2013;52(3):460–464. doi:10.1093/rheumatology/kes29623118412
   PubMed Web of Science ®Google Scholar
• Ghiassi R, Murphy K, Cummin AR, Partridge MR. Developing a pictorial epworth sleepiness scale. Thorax. 2011;66(2):97–100. doi:10.1136/thx.2010.13687920978025
   PubMed Web of Science ®Google Scholar
• Katz PP, Dall’Era M, Trupin L, et al. The impact of limited health literacy on patient-reported outcomes (PROs) in systemic lupus erythematosus. Arthritis Rheumatol. 2017;69(suppl10). Available from: https://acrabstracts.org/abstract/the-impact-of-limited-health-literacy-on-patient-reported-outcomes-pros-in-systemic-lupus-erythematosus-sle/. Accessed December 19, 2018.
   Google Scholar
• Perez JL, Mosher ZA, Watson SL, et al. Readability of orthopaedic patient-reported outcome measures: is there a fundamental failure to communicate? Clin Orthop Relat Res. 2017;475(8):1936–1947. doi:10.1007/s11999-017-5339-028374349
   PubMed Web of Science ®Google Scholar
• Courtney-Long EA, Carroll DD, Zhang QC, et al. Prevalence of disability and disability type among adults–united States, 2013. MMWR Morb Mortal Wkly Rep. 2015;64(29):777–783.26225475
   PubMed Web of Science ®Google Scholar
• Rothrock NE, Hays RD, Spritzer K, Yount SE, Riley W, Cella D. Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the patient-reported outcomes measurement information system (PROMIS). J Clin Epidemiol. 2010;63(11):1195–1204. doi:10.1016/j.jclinepi.2010.04.01220688471
   PubMed Web of Science ®Google Scholar
• Eich E, Reeves JL, Jaeger B, Graff-Radford SB. Memory for pain: relation between past and present pain intensity. Pain. 1985;23(4):375–380.4088698
   PubMed Web of Science ®Google Scholar
• Litwin MS, McGuigan KA. Accuracy of recall in health-related quality-of-life assessment among men treated for prostate cancer. J Clin Oncol. 1999;17(9):2882–2888. doi:10.1200/JCO.1999.17.9.288210561366
   PubMed Web of Science ®Google Scholar
• Kvam AK, Wisløff F, Fayers PM. Minimal important differences and response shift in health-related quality of life; a longitudinal study in patients with multiple myeloma. Health Qual Life Outcomes. 2010;8(1):79. doi:10.1186/1477-7525-8-7920678240
   PubMed Web of Science ®Google Scholar
• Atherton PJ, Sloan JA. Rising importance of patient-reported outcomes. Lancet Oncol. 2006;7(11):883–884. doi:10.1016/S1470-2045(06)70914-717081911
   PubMed Web of Science ®Google Scholar