Measuring the Patient Experience of Mental Health Care: A Systematic and Critical Review of Patient-Reported Experience Measures

Figures & data

Table 1 Quality Criteria

Property	Definition	Quality Criteria
Instrument development
Pre-study hypothesis and intended population	Specification of the hypothesis pre-study and if the intended population have been studied	✓✓- Clear statement of aims and target population, as well as intended population being studied in adequate depth ✓- Only one of the above or generic sample studied X- Neither reported
Actual content area (face validity)	Extent to which the content meets the pre-study aims and population	✓✓- Content appears relevant to the intended population ✓- Some relevant content areas missing X- Content area irrelevant to the intended population
Item identification	Items selected are relevant to the target population	✓✓- Evidence of consultation with patients, stakeholders and experts (through focus groups/one-to-one interview) and review of literature ✓- Some evidence of consultation X- Patients not involved in item identification
Item selection	Determining of final items to include in the instrument	✓✓- Rasch or factor analysis employed, missing items and floor/ceiling effects taken into consideration. Statistical justification for removal of items ✓- Some evidence of above analysis X- Nil reported
Unidimensionality	Demonstration that all items fit within an underlying construct	✓✓- Rasch analysis or factor loading for each construct. Factor loadings >0.4 for all items ✓- Cronbach’s alpha used to determine correlation with other items in instrument. Value >0.7 and <0.9 X- Nil reported
Response scale	Scale used to complete the measure	✓✓- Response scale noted and adequate justification given ✓- Response scale with no justification for selection X- Nil reported
Instrument performance
Convergent validity	Assessment of the degree of correlation with a related measure	✓✓- Tested against appropriate measure, Pearson’s correlation coefficient between 0.3 and 0.9 ✓- Inappropriate measure, but coefficient between 0.3 and 0.9 X- Nil reported or tested and correlates <0.3 or >0.9
Discriminant validity	Degree to which an instrument diverges from another instrument that it should not be similar to	✓✓- Tested against appropriate measure, Pearson’s correlation coefficient <0.3 ✓- Inappropriate measure, but coefficient <0.3 X- Nil reported or tested and correlates >0.3
Predictive validity	Ability for a measure to predict a future event	✓✓- Tested against appropriate measure and coefficient >0.3 ✓- Inappropriate measure but coefficient >0.3 X- Nil reported or tested and correlates <0.3
Test-retest reliability	Statistical technique used to estimate components of measurement error by testing comparability between two applications of the same test at different time points	✓✓- Pearson’s r value or ICC >0.8 ✓- Measured but Pearson’s r value or ICC <0.8 X- Nil reported
Responsiveness	Extent to which an instrument can detect clinically important differences over time	✓✓- Discussion of responsiveness and change over time. Score changes > MID over time ✓- Some discussion but no measure of MID X- Nil reported

Notes: ✓✓-positive rating, ✓-acceptable rating, X-negative rating.

Abbreviations: ICC, intraclass coefficient; MID, minimally important difference.

Figure 1 PRISMA flow chart.

Note: Adapted from Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 62(10):1006–1012.³⁵

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006–1012. doi:10.1016/j.jclinepi.2009.06.005

 PubMed Web of Science ®Google Scholar