Truthfulness in patient-reported outcomes: factors affecting patients’ responses and impact on data quality

Figures & data

Figure 1 The PRO instrument–development process.

Note: Adapted with permission from Rothrock NE, Kaiser KA, Cella D. Developing a valid patient-reported outcome measure. Clin Pharmacol Ther. 2011;90(5):737–742. © 2011 American Society for Clinical Pharmacology and Therapeutics.¹ Please refer to reference 1 for the original figure.
Abbreviation: PRO, patient-reported outcome.

Figure 2 US Food and Drug Administration (FDA) guidance for industry on PRO evaluation.

Note: Data from FDA.¹²
Abbreviation: PRO, patient-reported outcome.

Figure 3 Factors impacting patient responses to PRO measures.

Notes: Factors may occur during PRO-instrument development, during administration and data collection, or secondary to patient-level response behaviors.
Abbreviation: PRO, patient-reported outcome.

Figure 4 Examples of response scales used in PRO measures.

Abbreviations: PRO, patient-reported outcome; VRS, verbal rating scale; NRS, numeric rating scale; VAS, visual analogue scale.

Table 1 Types of response styles and impact on the data collected

	Definition	Example^a	Impact on data collected
Acquiescence response style	Tendency to agree or disagree with items to indicate positive connotation	○○○○●●●	Assuming higher response categories indicate positivity, inflates observed means and increases magnitude of multivariate relationships
Disacquiescence response style	Tendency to agree or disagree with items to indicate negative connotation	●●●○○○○	Assuming lower response categories indicate positivity, deflates observed means and increases magnitude of multivariate relationships
Midpoint response style	Tendency to use the middle response category of a scale	○○○●○○○	Brings observed means closer to midpoint, deflates variance, increases magnitude of multivariate relationships
Extreme response style	Tendency to use the highest and lowest response categories of a scale	●○○○○○●	Inflates observed mean variance, decreases magnitude of multivariate relationships
Mild response style	Tendency to avoid the highest and lowest response categories of a scale	○●●●●●○	Brings observed means closer to midpoint, deflates variance, increases magnitude of multivariate relationships
Net acquiescence response style	Tendency to show greater acquiescence than disacquiescence		Inflates variance, deflates observed means if negative
Response range	Tendency to use a narrow or wide range of response categories around the mean		When large, inflates variance, decreases magnitude of multivariate relationships
Noncontingent responding	Tendency to respond to items carelessly, randomly, or nonpurposefully		No a priori hypotheses about the effect can be specified

Notes: ^aExamples based on 7-point Likert scale. Adapted from Van Vaerenbergh Y, Thomas TD. Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int J Public Opin Res. 2013;25(2):195–217, by permission of Oxford University.¹⁷

Table 2 Factors impacting responses during PRO administration and data collection

	Impact on data collected
Source of report
Self	If populations with difficulties with self-administration excluded, may lead to incomplete or unrepresentative data
Proxy	Potential for disagreement between patient and proxy responses Direction and magnitude of disagreement may depend on targeted construct and proxy-related factors
Mode of administration
Self-administration	Increased potential for missing data May not allow for complex survey design
Interviewer-based	Potential for interviewer bias May increase social desirability bias and acquiescent response bias May limit disclosure of sensitive topics
Method of administration
Paper and pencil	Increased potential for data-entry errors may lead to inaccuracies May not allow for complex survey design May be less comfortable for disclosure of sensitive topics
Electronic	Potential for inaccuracies in patients with discomfort with technology Accessibility issues may increase potential for missing data
Setting of administration
Clinic	Interruptions secondary to clinic workflow may increase potential for missing data Privacy concerns may limit disclosure
Home	May exacerbate accessibility issues, increasing potential for missing data Lack of direct interaction may decrease response rates

Note: This table has been adapted from Table 3 in Cella DF, Hahn EA, Jensen SE, et al. Patient-Reported Outcomes in Performance Measurement. Research Triangle Park, NC: RTI Press/RTI International; 2015. Copyright RTI International; licensed under Creative Commons BY-NC-ND.⁸

Abbreviation: PRO, patient-reported outcome.

Rothrock NE, Kaiser KA, Cella D. Developing a valid patient-reported outcome measure. Clin Pharmacol Ther. 2011;90(5):737–742. doi:10.1038/clpt.2011.19521975345

 PubMed Web of Science ®Google Scholar

United States Food and Drug Administration. Guidance for industry patient-reported outcome measures: use in medical product development to support labeling claims; 2009 Available from: https://www.fda.gov/downloads/drugs/guidances/ucm193282.pdf. Accessed 1210, 2018.

 Google Scholar

Van Vaerenbergh Y, Thomas TD. Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int J Public Opin Res. 2013;25(2):195–217. doi:10.1093/ijpor/eds021

 Web of Science ®Google Scholar

Cella DF, Hahn EA, Jensen SE, et al. Patient-Reported Outcomes in Performance Measurement. Research Triangle Park, NC: RTI Press/RTI International; 2015.

 Google Scholar