Evidence Base Update for Brief, Free, and Accessible Youth Mental Health Measures

Figures & data

Table 1 Rubric for Evaluating Norms, Validity, and Utility

Criterion	Adequate	Good	Excellent
Norms	M and SD for total score (and subscores if relevant) from a large, relevant clinical sample	M and SD for total score (and subscores if relevant) from multiple large, relevant samples, at least one clinical and one nonclinical	Same as “good,” but must be from representative sample (i.e., random sampling, or matching to census data)
Internal Consistency (Cronbach’s Alpha, Split Half, etc.)	Most evidence shows alpha values of 0.70–0.79	Most reported alphas 0.80–0.89	Most reported alphas ≥ 0.90
Interrater Reliability	Most evidence shows kappas of 0.60–0.74, or intraclass correlations of 0.70–0.79	Most reported kappas of 0.75–0.84, ICCs of 0.80–0.89	Most kappas ≥ 0.85, or ICCs ≥ 0.90
Test–Retest Reliability (Stability)	Most evidence shows test–retest correlations ≥ 0.70 over several days or weeks	Most evidence shows test–retest correlations ≥ 0.70 over several months	Most evidence shows test–retest correlations ≥ 0.70 over 1 year or longer
Repeatability	Bland–Altman (Bland & Altman, 1986) plots show small bias, and/or weak trends; coefficient of repeatability is tolerable compared to clinical benchmarks (Vaz, Falkmer, Passmore, Parsons, & Andreou, 2013)	Bland–Altman plots and corresponding regressions show no significant bias, and no significant trends; coefficient of repeatability is tolerable	Bland–Altman plots and corresponding regressions show no significant bias, and no significant trends; established for multiple studies; coefficient of repeatability is small enough that it is not clinically concerning
Content Validity	Test developers clearly defined domain and ensured representation of entire set of facets	Same as “adequate,” plus all elements (items, instructions) evaluated by judges (experts or pilot participants)	Same as “good,” plus multiple groups of judges and quantitative ratings
Construct Validity (e.g., Predictive, Concurrent, Convergent, and Discriminant Validity)	Some independently replicated evidence of construct validity	Bulk of independently replicated evidence shows multiple aspects of construct validity	Same as “good,” plus evidence of incremental validity with respect to other clinical data
Discriminative Validity	Statistically significant discrimination in multiple samples; AUCs < 0.6 under clinically realistic conditions (i.e., not comparing treatment seeking and healthy youth)	AUCs of 0.60 to < 0.75 under clinically realistic conditions	AUCs of 0.75 to 0.90 under clinically realistic conditions
Prescriptive Validity	Statistically significant accuracy at identifying a diagnosis with a well-specified matching intervention, or statistically significant moderator of treatment	Same as “adequate,” with good kappa for diagnosis, or significant treatment moderation in more than one sample	Same as “good,” with good kappa for diagnosis in more than one sample, or moderate effect size for treatment moderation
Validity Generalization	Some evidence supports use with either more than one specific demographic group or in more than one setting	Bulk of evidence supports use with either more than one specific demographic group or in multiple settings	Bulk of evidence supports use with either more than one specific demographic group AND in multiple settings
Treatment Sensitivity	Some evidence of sensitivity to change over course of treatment	Independent replications show evidence of sensitivity to change over course of treatment	Same as “good,” plus sensitive to change across different types of treatments
Clinical Utility	After practical considerations (e.g., costs, respondent burden, ease of administration and scoring, availability of relevant benchmark scores, patient acceptability), assessment data are likely to be clinically actionable	Same as “adequate,” plus published evidence that using the assessment data confers clinical benefit (e.g., better outcome, lower attrition, greater satisfaction), in areas important to stakeholders	Same as “good,” plus independent replication

Rubric initially developed by Hunsley and Mash (2008) and extended by Youngstrom et al. (2017). Evaluation criteria were adapted in this study due to the large number of measure that met inclusion criteria for review. Table reproduced with permission from De Los Reyes and Langer (2018).

Figure 1 PRISMA Flow Diagram.

Figure 2 Measures at a Glance: Brief, Free, Accessible Youth Mental Health Measures.

Bland, J. M., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307–310. doi:10.1016/S0140-6736(86)90837-8

 Web of Science ®Google Scholar

Vaz, S., Falkmer, T., Passmore, A. E., Parsons, R., & Andreou, P. (2013). The case for using the repeatability coefficient when calculating test–Retest reliability. PLoS ONE, 8(9), 1–7. doi:10.1371/journal.pone.0073990

 Web of Science ®Google Scholar

Hunsley, J., & Mash, E. J. (2008). Developing criteria for evidence-based assessment: An introduction to assessments that work. In J. Hunsley & E. J. Mash (Eds.), A guide to assessments that work (pp. 3–14). New York, NY: Oxford University Press.

 Google Scholar

Youngstrom, E. A., Van Meter, A., Frazier, T. W., Hunsley, J., Prinstein, M. J., Ong, M. L., & Youngstrom, J. K. (2017). Evidence‐based assessment as an integrative model for applying psychological science to guide the voyage of treatment. Clinical Psychology: Science and Practice, 24(4), 331–363. doi:10.1111/cpsp.12207

 Web of Science ®Google Scholar

De Los Reyes, A., & Langer, D. A. (2018). Assessment and the journal of clinical child and adolescent psychology’s evidence base updates series: Evaluating the tools for gathering evidence. Journal of Clinical Child & Adolescent Psychology, 47(3), 357–365. doi:10.1080/15374416.2018.1458314

 PubMed Web of Science ®Google Scholar