What is the Status of Multi-Informant Treatment Fidelity Research?

References

• Aarons, G., Hurlburt, M., & Horwitz, S. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7
   PubMed Web of Science ®Google Scholar
• Barber, J. P., Mercer, D., Krakauer, I., & Calvo, N. (1996). Development of an adherence/competence rating scale for individual drug counseling. Drug and Alcohol Dependence, 43(3), 125–132. https://doi.org/10.1016/S03768716(96)013051
   PubMed Web of Science ®Google Scholar
• Beale, S., Liness, S., & Hirsch, C. R. (2020). Trainee self-assessment of cognitive behaviour therapy competence during and after training. The Cognitive Behaviour Therapist, 13, 13. https://doi.org/10.1017/S1754470X19000357
   Web of Science ®Google Scholar
• Becker-Haimes, E. M., Klein, M. R., McLeod, B. D., Schoenwald, S. K., Dorsey, S., Hogue, A., Fugo, P. B., Phan, M. L., Hoffacker, C., & Beidas, R. S. (2021). The TPOCS-self-reported therapist intervention fidelity for youth (TPOCS-SeRTIFY): A case study of pragmatic measure development. Implementation Research and Practice, 2, 2633489521992553. https://doi.org/10.1177/2633489521992553
   Google Scholar
• Becker-Haimes, E. M., Marcus, S. C., Klein, M. R., Schoenwald, S. K., Fugo, P. B., McLeod, B. D., Dorsey, S., Williams, N. J., Mandell, D. S., & Beidas, R. S. (2022). A randomized trial to identify accurate measurement methods for adherence to cognitive-behavioral therapy. Behavior Therapy, 53 (6), 1191–1204. Advance online publication. https://doi.org/10.1016/j.beth.2022.06.001
   PubMed Web of Science ®Google Scholar
• Beidas, R. S., Cross, W., & Dorsey, S. (2014). Show me, don’t tell me: Behavioral rehearsal as a training and analogue fidelity tool. Cognitive and Behavioral Practice, 21(1), 1–11. https://doi.org/10.1016/j.cbpra.2013.04.002
   PubMed Web of Science ®Google Scholar
• Beidas, R. S., Maclean, J. C., Fishman, J., Dorsey, S., Schoenwald, S. K., Mandell, D. S., Shea, J A., McLeod, B D., French, M T., Hogue, A., & Adams, D R. (2016). A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: Project FACTS study protocol. BMC Psychiatry, 16(1), 1–10. https://doi.org/10.1186/s12888-016-1034-z
   PubMedGoogle Scholar
• Blackburn, I. M., James, I. A., Milne, D. L., Reichelt, F. K., Standart, S., Garland, A., & Reichelt, F. K. (2001). The Revised Cognitive Therapy Scale (CTSR): Psychometric properties. Behavioural and Cognitive Psychotherapy, 29(4), 431–447. https://doi.org/10.1017/S1352465801004040
   Web of Science ®Google Scholar
• Blackburn, I. M., Milne, D. L., & James, I. A. (1997). How are therapeutic competencies evaluated? [Paper presentation]. Annual Conference of the British Psychological Society (Division of Clinical Psychology). Edinburgh: Herriot–Watt University.
   Google Scholar
• Breitenstein, S. M., Fogg, L., Garvey, C., Hill, C., Resnick, B., & Gross, D. (2010). Measuring implementation fidelity in a community-based parenting intervention. Nursing Research, 59(3), 158. https://doi.org/10.1097/NNR.0b013e3181dbb2e2
   PubMed Web of Science ®Google Scholar
• Brookman-Frazee, L., Stadnick, N. A., Lind, T., Roesch, S., Terrones, L., Barnett, M. L., Regan, J., Kennedy, C. A., F Garland, A., & Lau, A. S. (2020). Therapist-observer concordance in ratings of EBP strategy delivery: Challenges and targeted directions in pursuing pragmatic measurement in children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 48(1), 1–16. https://doi.org/10.1007/s10488-020-01054-x
   Web of Science ®Google Scholar
• Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self-evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36(5), 581–587. https://doi.org/10.1017/S1352465808004438
   Web of Science ®Google Scholar
• Caron, E. B., & Dozier, M. (2021). Self-coding of fidelity as a potential active ingredient of consultation to improve clinicians’ fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 49(2), 237–254. https://doi.org/10.1007/s10488-021-01160-4
   PubMed Web of Science ®Google Scholar
• Caron, E. B., Muggeo, M. A., Souer, H. R., Pella, J. E., & Ginsburg, G. S. (2019). Concordance between clinician, supervisor and observer ratings of therapeutic competence in CBT and treatment as usual: Does clinician competence or supervisor session observation improve agreement? Behavioural and Cognitive Psychotherapy, 48(3), 350–363. https://doi.org/10.1017/S1352465819000699
   PubMed Web of Science ®Google Scholar
• Carroll, K., Nich, C., & Rounsaville, B. (1998). Utility of therapist session checklists to monitor delivery of coping skills treatment for cocaine abusers. Psychotherapy Research, 8(3), 307–320. https://doi.org/10.1093/ptr/8.3.307
   Web of Science ®Google Scholar
• Chaffin, M., Hecht, D., Aarons, G., Fettes, D., Hurlburt, M., & Ledesma, K. (2016). EBT fidelity trajectories across training cohorts using the interagency Collaborative Team Strategy. Administration and Policy in Mental Health and Mental Health Services Research, 43(2), 144–156. https://doi.org/10.1007/s10488-015-0627-z
   PubMed Web of Science ®Google Scholar
• Chapman, J. E., McCart, M. R., Letourneau, E. J., & Sheidow, A. J. (2013). Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol. Journal of Consulting and Clinical Psychology, 81(4), 674. https://doi.org/10.1037/a0033021
   PubMed Web of Science ®Google Scholar
• Chapman, J. E., Sheidow, A. J., Henggeler, S. W., Halliday-Boykins, C. A., & Cunningham, P. B. (2008). Developing a measure of therapist adherence to contingency management: An application of the many-facet rasch model. Journal of Child & Adolescent Substance Abuse, 17(3), 47–68. https://doi.org/10.1080/15470650802071655
   PubMed Web of Science ®Google Scholar
• Chaturvedi, S. R. B. H., & Shweta, R. C. (2015). Evaluation of inter-rater agreement and inter-rater reliability for observational data: An overview of concepts and methods. Journal of the Indian Academy of Applied Psychology, 41(3), 20–27.
   Google Scholar
• Chevron, E. S., & Rounsaville, B. J. (1983). Evaluating the clinical skills of psychotherapists: A comparison of techniques. Archives of General Psychiatry, 40(10), 1129–1132. https://doi.org/10.1001/archpsyc.1983.01790090091014
   PubMed Web of Science ®Google Scholar
• Chu, B. C., & Kendall, P. C. (2004). Positive of child involvement and treatment outcome within a manual-based cognitive-behavioral treatment for children with anxiety. Journal of Consulting and Clinical Psychology, 72(5), 821–829. https://doi.org/10.1037/0022-006X.72.5.821
   PubMed Web of Science ®Google Scholar
• Cicchetti, D. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.
   Google Scholar
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
   Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Earlbaum Associates, Inc.
   Google Scholar
• Comer, J. S., & Kendall, P. C. (2004). A symptom-level examination of parent–child agreement in the diagnosis of anxious youths. Journal of the American Academy of Child & Adolescent Psychiatry, 43(7), 878–886. https://doi.org/10.1097/01.chi.0000125092.35109.c5
   PubMed Web of Science ®Google Scholar
• Couturier, J., Kimber, M., Barwick, M., McVey, G., Findlay, S., Webb, C., Niccols, A., & Lock, J. (2021). Assessing fidelity to family-based treatment: An exploratory examination of expert, therapist, parent, and peer ratings. Journal of Eating Disorders, 9(1), 1–9. https://doi.org/10.1186/s40337-020-00366-5
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A. (2011). Introduction to the special section. More than measurement error: discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. Journal of Clinical Child & Adolescent Psychology, 40(1), 1–9. https://doi.org/10.1080/15374416.2011.533405
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A., Cook, C. R., Gresham, F. M., Makol, B. A., & Wang, M. (2019). Informant discrepancies in assessments of psychosocial functioning in school-based services and research: Review and directions for future research. Journal of School Psychology, 74, 74–89. https://doi.org/10.1016/j.jsp.2019.05.005
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A., Henry, D. B., Tolan, P. H., & Wakschlag, L. S. (2009). Linking informant discrepancies to observed variations in young children’s disruptive behavior. Journal of Abnormal Child Psychology, 37(5), 637–652. https://doi.org/10.1007/s10802-009-9307-3
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A., & Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131(4), 483–509. https://doi.org/10.1037/0033-2909.131.4.483
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A., Thomas, S. A., Goodman, K. L., & Kundey, S. M. A. (2013). Principles underlying the use of multiple informants’ reports. Annual Review of Clinical Psychology, 9(1), 123–149. https://doi.org/10.1146/annurev-clinpsy-050212-185617
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A., Wang, M., Lerner, M. D., Makol, B. A., Fitzpatrick, W., & Weisz, J. R. (2022). The operations triad model and youth mental health assessments: Catalyzing an paradigm shift in measurement validation. Journal of Clinical Child & Adolescent Psychology, 1–36. Advance online publication. https://doi.org/10.1080/15374416.2022.2111684
   Google Scholar
• Dennhag, I., Gibbons, M. B. C., Barber, J. P., Gallop, R., & Crits-Christoph, P. (2012). Do supervisors and independent judges agree on evaluations of therapist adherence and competence in the treatment of cocaine dependence? Psychotherapy Research, 22(6), 720–730. https://doi.org/10.1080/10503307.2012.716528
   PubMed Web of Science ®Google Scholar
• Dickson, K. S., & Suhrheinrich, J. (2021). Concordance between community supervisor and provider ratings of fidelity: Examination of multi-level predictors and outcomes. Journal of Child and Family Studies, 30(2), 542–555. https://doi.org/10.1007/s10826-020-01877-0
   PubMed Web of Science ®Google Scholar
• Dorsey, S., Lyon, A. R., Pullmann, M. D., Jungbluth, N., Berliner, L., & Beidas, R. (2017). Behavioral rehearsal for analogue fidelity: Feasibility in a state-funded children’s mental health initiative. Administration and Policy in Mental Health and Mental Health Services Research, 44(3), 395–404. https://doi.org/10.1007/s10488-016-0727-4
   PubMed Web of Science ®Google Scholar
• Fjermestad, K. W., McLeod, B. D., Tully, C. B., & Liber, J. M. (2016). Therapist characteristics and interventions: enhancing alliance and involvement with youth. In S. Maltzman (Ed.), Oxford handbook of treatment processes and outcomes in counseling psychology (pp. 97–116). Oxford University Press.
   Google Scholar
• Frank, H. E., Becker‐haimes, E. M., & Kendall, P. C. (2020). Therapist training in evidence‐based interventions for mental health: A systematic review of training approaches and outcomes. Clinical Psychology: Science and Practice, 27(3), e12330. https://doi.org/10.1111/cpsp.12330
   PubMed Web of Science ®Google Scholar
• Gisev, N., Bell, J. S., & Chen, T. F. (2013). Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy, 9(3), 330–338. https://doi.org/10.1016/j.sapharm.2012.04.004
   PubMed Web of Science ®Google Scholar
• Goodwin, L. D. (2001). Interrater agreement and reliability. Measurement in Physical Education and Exercise Science, 5(1), 13–34. https://doi.org/10.1207/S15327841MPEE0501_2
   Google Scholar
• Gresham, F. M., Dart, E. H., & Collins, T. A. (2017). Generalizability of multiple measures of treatment integrity: Comparisons among direct observation, permanent products, and self-report. School Psychology Review, 46(1), 108–121. https://doi.org/10.1080/02796015.2017.12087606
   Web of Science ®Google Scholar
• Gumport, N. B., Stephanie, H. Y., Mullin, A. C., Mirzadegan, I. A., & Harvey, A. G. (2019). The validation of a provider-reported fidelity measure for the transdiagnostic sleep and circadian intervention in a community mental health setting. Behavior Therapy, 51(5), 800–813. https://doi.org/10.1016/j.beth.2019.11.006
   PubMed Web of Science ®Google Scholar
• Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23. https://doi.org/10.20982/tqmp.08.1.p023
   PubMedGoogle Scholar
• Hartmann, D. P. (1977). Considerations in the choice of interobserver reliability estimates. Journal of Applied Behavior Analysis, 10(1), 103–116. https://doi.org/10.1901/jaba.1977.10-103
   PubMed Web of Science ®Google Scholar
• Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letourneau, E. J., & Edwards, D. L. (2002). Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs. Journal of Clinical Child & Adolescent Psychology, 31(2), 155–167. https://doi.org/10.1207/S15374424JCCP3102_02
   PubMed Web of Science ®Google Scholar
• Herschell, A. D., Quetsch, L. B., & Kolko, D. J. (2020). Measuring adherence to key teaching techniques in an evidence-based treatment: A comparison of caregiver, therapist, and behavior observation ratings. Journal of Emotional and Behavioral Disorders, 28(2), 92–103. https://doi.org/10.1177/1063426618821901
   Web of Science ®Google Scholar
• Hill, C. E. (1991). Almost everything you ever wanted to know about how to do process research on counseling and psychotherapy but didn’t know who to ask. In C. E. Hill & L. J. Schneider (Eds.), Research in counseling (pp. 85–118). Lawrence Erlbaum Associates.
   Google Scholar
• Hogue, A. (2022). Behavioral intervention fidelity in routine practice: Pragmatism Moves to Head of the Class. School Mental Health, 14(1), 1–7. https://doi.org/10.1007/s12310-021-09488-w
   Web of Science ®Google Scholar
• Hogue, A., Bobek, M., Dauber, S., Henderson, C. E., McLeod, B. D., & Southam-Gerow, M. A. (2019). Core elements of family therapy for adolescent behavior problems: Empirical distillation of three manualized treatments. Journal of Clinical Child & Adolescent Psychology, 48(1), 29–41. https://doi.org/10.1080/15374416.2018.1555762
   PubMed Web of Science ®Google Scholar
• Hogue, A., Bobek, M., Porter, N., Dauber, S., Southam-Gerow, M. A., McLeod, B. D., & Henderson, C. E. (2021). Core elements of family therapy for adolescent behavioral health problems: Validity generalization in community settings. Journal of Clinical Child & Adolescent Psychology, 1–13. https://doi.org/10.1080/15374416.2021.1969939
   Google Scholar
• Hogue, A., Bobek, M., Porter, N., MacLean, A., Bruynesteyn, L., Jensen Doss, A., Dauber, S., & Henderson, C. E. (2022). Therapist self-report of fidelity core elements of family therapy for adolescent behavior problems: Psychometrics of a pragmatic quality indicator tool. Administration and Policy in Mental Health and Mental Health Services Research, 49(2), 298–311. https://doi.org/10.1007/s10488-021-01164-0
   PubMed Web of Science ®Google Scholar
• Hogue, A., Dauber, S., Henderson, C. E., & Liddle, H. A. (2014). Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy in Mental Health and Mental Health Services Research, 41(5), 697–705. https://doi.org/10.1007/s10488-013-0520-6
   PubMed Web of Science ®Google Scholar
• Hogue, A., Dauber, S., Lichvar, E., Bobek, M., & Henderson, C. E. (2015). Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: Correspondence between therapists and observers. Administration and Policy in Mental Health and Mental Health Services Research, 42(2), 229–243. https://doi.org/10.1007/s10488-014-0548-2
   PubMed Web of Science ®Google Scholar
• Hogue, A., Henderson, C. E., Dauber, S., Barajas, P. C., Fried, A., & Liddle, H. A. (2008). Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76(4), 544–555. https://doi.org/10.1037/0022-006X.76.4.544
   PubMed Web of Science ®Google Scholar
• Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy: Theory, Research, Practice, & Training, 33(2), 332–345. https://doi.org/10.1037/0033-3204.33.2.332
   Web of Science ®Google Scholar
• Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 230–244. https://doi.org/10.1007/s10488-009-0251-x
   PubMed Web of Science ®Google Scholar
• Jensen Doss, A., Cusack, K. J., & De Arellano, M. A. (2008). Workshop-based training in trauma-focused CBT: An in-depth analysis of impact on provider practices. Community Mental Health Journal, 44(4), 227–244. https://doi.org/10.1007/s10597-007-9121-8
   PubMed Web of Science ®Google Scholar
• Jensen Doss, A., Douglas, S., Phillips, O., Gencdur, D. A., Zalman, A., & Gomez, N. E. (2020). Measurement-based care as a practice improvement tool: Clinical and organizational application in youth mental health. Evidence-Based Practice in Child and Adolescent Mental Health, 5(3), 233–250. https://doi.org/10.1080/23794925.2020.1784062
   PubMedGoogle Scholar
• Kerns, S. E., Perrine, C. M., Sedlar, G., Peterson, R., & Monroe DeVita, M. (2021). Keeping the faith while keeping it real: Practical, empirical approaches to evaluating treatment fidelity. Global Implementation Research and Applications, 1(2), 1–12. https://doi.org/10.1007/s43477-021-00012-5
   Google Scholar
• Koddebusch, C., & Herrmann, C. (2019). Multi-informant assessment of therapeutic competence: Development and initial validation of a set of measurements. International Journal of Psychology and Psychological Therapy, 19(1), 15–28.
   Web of Science ®Google Scholar
• Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163.
   PubMed Web of Science ®Google Scholar
• Kraemer, H. C., Measelle, J. R., Ablow, J. C., Essex, M. J., Boyce, W. T., & Kupfer, D. J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. The American Journal of Psychiatry, 160(9), 1566–1577. https://doi.org/10.1176/appi.ajp.160.9.1566
   PubMed Web of Science ®Google Scholar
• Laird, R. D. (2020). Analytical challenges of testing hypotheses of agreement and discrepancy: Comment on Campione-Barr, Lindell, and Giron (2020). Developmental Psychology, 56(5), 970–977.https://doi.org/10.1037/dev0000763
   PubMed Web of Science ®Google Scholar
• LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852. https://doi.org/10.1177/1094428106296642
   Web of Science ®Google Scholar
• Loades, M. E., & Myles, P. J. (2016). Does a therapist’s reflective ability predict the accuracy of their self-evaluation of competence in cognitive behavioural therapy? The Cognitive Behaviour Therapist, 9, 9. https://doi.org/10.1017/S1754470X16000027
   Google Scholar
• Mahatody, T., Sagar, M., & Kolski, C. (2010). State of the art on the cognitive walkthrough method, its variants and evolutions. International Journal of Human-Computer Interaction, 26(8), 741–85.
   Web of Science ®Google Scholar
• Martino, S., Ball, S. A., Nich, C., Frankforter, T. F., & Carroll, K. M. (2008). Community program therapist adherence and competence in motivational enhancement therapy. Drug and Alcohol Dependence, 96(1–2), 37–48. https://doi.org/10.1016/j.drugalcdep.2008.01.020
   PubMed Web of Science ®Google Scholar
• Martino, S., Ball, S., Nich, C., Frankforter, T. L., & Carroll, K. M. (2009). Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychotherapy Research, 19(2), 181–193. https://doi.org/10.1080/10503300802688460
   PubMed Web of Science ®Google Scholar
• Masia Warner, C., Brice, C., Esseling, P. G., Stewart, C. E., Mufson, L., & Herzig, K. (2013). Consultants’ perceptions of school counselors’ ability to implement an empirically-based intervention for adolescent social anxiety disorder. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 541–554. https://doi.org/10.1007/s10488-013-0498-0
   PubMed Web of Science ®Google Scholar
• Mathieson, F. M., Barnfield, T., & Beaumont, G. (2009). Are we as good as we think we are? Self-assessment versus other forms of assessment of competence in psychotherapy. The Cognitive Behaviour Therapist, 2(1), 43–50. https://doi.org/10.1017/S1754470X08000081
   Google Scholar
• McGrew, J. H., Stull, L. G., Rollins, A. L., Salyers, M. P., & Hicks, L. J. (2011). A comparison of phone-based and on-site assessment of fidelity for assertive community treatment in Indiana. Psychiatric Services, 62(6), 670–674. https://doi.org/10.1176/ps.62.6.pss6206_0670
   PubMed Web of Science ®Google Scholar
• McGrew, J. H., White, L. M., Stull, L. G., & Wright-Berryman, J. (2013). A comparison of self-reported and phone-administered methods of ACT fidelity assessment: A pilot study in Indiana. Psychiatric Services, 64(3), 272–276. https://doi.org/10.1176/appi.ps.001252012
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Cox, J. R., Jensen‐doss, A., Herschell, A., Ehrenreich‐may, J., & Wood, J. J. (2018). Proposing a mechanistic model of clinician training and consultation. Clinical Psychology: Science and Practice, 25(3), e12260. https://doi.org/10.1111/cpsp.12260
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Islam, N. Y., & Wheat, E. (2013). Designing, conducting, and evaluating therapy process research. In J. Comer & P. Kendall (Eds.), The Oxford handbook of research strategies for clinical psychology (pp. 142–164). Oxford University Press.
   Google Scholar
• McLeod, B. D., Martinez, R., Southam-Gerow, M. A., Weisz, J. R., & Chorpita, B. F. (2022). Can a single measure estimate protocol adherence for two psychosocial treatments for youth anxiety delivered in community mental health settings? Behavior Therapy, 53(1), 119–136. https://doi.org/10.1016/j.beth.2021.06.008
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Smith, M. M., Southam‐gerow, M. A., Weisz, J. R., & Kendall, P. C. (2015). Measuring treatment differentiation for implementation research: The Therapy Process Observational Coding System for Child Psychotherapy Revised Strategies Scale. Psychological Assessment, 27(1), 314–325. https://doi.org/10.1037/pas0000037
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Southam‐gerow, M. A., Tully, C. B., Rodríguez, A., & Smith, M. M. (2013). Making a case for treatment fidelity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20(1), 14–32. https://doi.org/10.1111/cpsp.12020
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Southam-Gerow, M. A., & Weisz, J. R. (2009). Conceptual and methodological issues in treatment integrity measurement. School Psychology Review, 38(4), 541–546.
   Web of Science ®Google Scholar
• McLeod, B. D., Sutherland, K. S., Broda, M., Granger, K. L., Cecilione, J., Cook, C. R., Conroy, M. A., Snyder, P. A., & Southam-Gerow, M. A. (2022). Examining the correspondence between teacher-and observer-report treatment integrity measures. School Mental Health, 14(1), 20–34. https://doi.org/10.1007/s12310-021-09437-7
   Web of Science ®Google Scholar
• McManus, F., Rakovshik, S., Kennerley, H., Fennell, M., & Westbrook, D. (2012). An investigation of the accuracy of therapists’ self‐assessment of cognitive‐behaviour therapy skills. British Journal of Clinical Psychology, 51(3), 292–306. https://doi.org/10.1111/j.2044-8260.2011.02028.x
   PubMedGoogle Scholar
• Moyers, T. B., Martin, T., Manuel, J. K., Miller, W. R., & Ernst, D. (2010). Revised global scales: Motivational interviewing treatment integrity 3.1. 1 (MITI 3.1. 1) [ Unpublished manuscript]. University of New Mexico.
   Google Scholar
• Mullin, D. J., Saver, B., Savageau, J. A., Forsberg, L., & Forsberg, L. (2016). Evaluation of online and in-person motivational interviewing training for healthcare providers. Families, Systems, & Health, 34(4), 357. https://doi.org/10.1037/fsh0000214
   PubMed Web of Science ®Google Scholar
• Park, A. L., Chorpita, B. F., Regan, J., Weisz, J. R., & The Research Network on Youth Mental Health. (2015). Integrity of evidence-based practice: Are providers modifying practice content or practice sequencing? Administration and Policy in Mental Health and Mental Health Services Research, 42(2), 186–196. https://doi.org/10.1007/s10488-014-0559-z
   PubMed Web of Science ®Google Scholar
• Peavy, K. M., Guydish, J., Manuel, J. K., Campbell, B. K., Lisha, N., Le, T., Delucchi, K., & Garrett, S. (2014). Treatment adherence and competency ratings among therapists, supervisors, study-related raters and external raters in a clinical trial of a 12-step facilitation for stimulant users. Journal of Substance Abuse Treatment, 47(3), 222–228. https://doi.org/10.1016/j.jsat.2014.05.008
   PubMed Web of Science ®Google Scholar
• Perepletchikova, F. (2011). On the topic of treatment fidelity. Clinical Psychology: Science and Practice, 18(2), 148–153. https://doi.org/10.1111/j.1468-2850.2011.01246.x
   PubMed Web of Science ®Google Scholar
• Perepletchikova, F., & Kazdin, A. E. (2004). Assessment of parenting practices related to conduct problems: Development and validation of the management of children. Journal of Child and Family Studies, 13(4), 385–403. https://doi.org/10.1023/B:JCFS.0000044723.45902.70
   Google Scholar
• Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12(4), 365–383. https://doi.org/10.1093/clipsy.bpi045
   Web of Science ®Google Scholar
• Regan, J., Park, A., & Chorpita, B. (2019). Choices in treatment integrity: Considering the protocol and consultant recommendations in child and adolescent therapy. Journal of Clinical Child & Adolescent Psychology, 48(sup1), S79–S89.
   PubMed Web of Science ®Google Scholar
• Rollins, A. L., McGrew, J. H., Kukla, M., McGuire, A. B., Flanagan, M. E., Hunt, M. G., Leslie, D. L., Collins, L. A., Wright-Berryman, J. L., Hicks, L. J., & Salyers, M. P. (2016). Comparison of assertive community treatment fidelity assessment methods: Reliability and validity. Administration and Policy in Mental Health and Mental Health Services Research, 43(2), 157–167. https://doi.org/10.1007/s10488-015-0641-1
   PubMed Web of Science ®Google Scholar
• Rozek, D. C., Serrano, J. L., Marriott, B. R., Scott, K. S., Hickman, L. B., Brothers, B. M., Lewis, C. C., & Simons, A. D. (2018). Cognitive behavioural therapy competency: Pilot data from a comparison of multiple perspectives. Behavioural and Cognitive Psychotherapy, 46(2), 244–250. https://doi.org/10.1017/S1352465817000662
   PubMed Web of Science ®Google Scholar
• Sanetti, L. M., Charbonneau, S., Knight, A., Cochrane, W. S., Kulcyk, M. C. M., & Kraus, K. E. (2020). Treatment fidelity reporting in intervention outcome studies in the school psychology literature from 2009 to 2016. Psychology in the Schools, 57(6), 901–922. https://doi.org/10.1002/pits.22364
   Web of Science ®Google Scholar
• Schoenwald, S. K. (2011). It’s a bird, it’s a plane, it’s … fidelity measurement in the real world. Clinical Psychology: Science and Practice, 18(2), 142–147. https://doi.org/10.1111/j.1468-2850.2011.01245.x
   PubMed Web of Science ®Google Scholar
• Sheshko, D. M., Lee, C. M., & Gagné, M. H. (2020). Multimethod adherence measurement in an evidence-based parenting program. Practice Innovations, 5(4), 290. https://doi.org/10.1037/pri0000110
   Google Scholar
• Stirman, S. W., Gutner, C. A., Gamarra, J., Suvak, M. K., Vogt, D., Johnson, C., & STRONG STAR Consortium. (2021). A novel approach to the assessment of fidelity to a cognitive behavioral therapy for PTSD using clinical Worksheets: A proof of concept with cognitive processing therapy. Behavior Therapy, 52(3), 656–672. https://doi.org/10.1016/j.beth.2020.08.005
   PubMed Web of Science ®Google Scholar
• Sutherland, K. S., & McLeod, B. D. (2015a). Scoring manual for the treatment integrity measure for early childhood settings: the adherence and competence scale [ Unpublished scoring manual prepared]. Virginia Commonwealth University.
   Google Scholar
• Sutherland, K. S., & McLeod, B. D. (2015b). Scoring manual for the treatment integrity measure for early childhood settings: the school Mental Health teacher report scale [ Unpublished scoring manual prepared]. Virginia Commonwealth University.
   Google Scholar
• Sutherland, K. S., & McLeod, B. D. (2022). Advancing the science of integrity measurement in school mental health research. School Mental Health, 14(1), 1–6. https://doi.org/10.1007/s12310-021-09468-0
   Web of Science ®Google Scholar
• Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., Okamura, K., & Research Network on Youth Mental Health. (2013). Tracking evidence-based practice with youth: Validity of the MATCH and standard manual consultation records. Journal of Clinical Child & Adolescent Psychology, 42(1), 44–55. https://doi.org/10.1080/15374416.2012.700505
   PubMed Web of Science ®Google Scholar
• Ware, N. C., Dickey, B., Tugenberg, T., & McHorney, C. A. (2003). CONNECT: A measure of continuity of care in mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 5(4), 209–221. https://doi.org/10.1023/A:1026276918081
   Google Scholar