Pilot Trial of Online Measurement Training and Feedback in Family Therapy for Adolescent Behavior Problems

References

• Baldwin, S. A., Christian, S., Berkeljon, A., & Shadish, W. R. (2012). The effects of family therapies for adolescent delinquency and substance abuse: A meta-analysis. Journal of Marital and Family Therapy, 38(1), 281–304. https://doi.org/10.1111/j.1752-0606.2011.00248.x
   PubMed Web of Science ®Google Scholar
• Barth, R. P., Kolivoski, K. M., Lindsey, M. A., Lee, B. R., & Collins, K. S. (2014). Translating the common elements approach: Social work’s experiences in education, practice, and research. Journal of Clinical Child & Adolescent Psychology, 43(2), 301–311. https://doi.org/10.1080/15374416.2013.848771
   PubMed Web of Science ®Google Scholar
• Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
   Web of Science ®Google Scholar
• Becker, E. M., & Jensen-Doss, A. (2014). Therapist attitudes towards computer-based trainings. Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 845–854. https://doi.org/10.1007/s10488-013-0522-4
   PubMed Web of Science ®Google Scholar
• Beidas, R. S., Cross, W., & Dorsey, S. (2014). Show me, don’t tell me: Behavioral rehearsal as a training and analogue fidelity tool. Cognitive and Behavioral Practice, 21(1), 1–11. https://doi.org/10.1016/j.cbpra.2013.04.002
   PubMed Web of Science ®Google Scholar
• Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence‐based practice: A critical review of studies from a systems‐contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30. https://doi.org/10.1111/j.1468-2850.2009.01187.x
   PubMed Web of Science ®Google Scholar
• Ben-Shachar, M. S. (2021). Indices of effect size and standardized parameters. R package version 0.4.5. https://cran.rproject.org/web/packages/effectsize/effectsize.pdf
   Google Scholar
• Bleise, P. (2016). Multilevel functions. R package version 2.6. https://cran.r-project.org/web/packages/multilevel/multilevel.pdf
   Google Scholar
• Brookman-Frazee, L., Stadnick, N. A., Lind, T., Roesch, S., Terrones, L., Barnett, M. L., … Lau, A. S. (2021). Therapist-observer concordance in ratings of EBP strategy delivery: Challenges and targeted directions in pursuing pragmatic measurement in children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 48(1), 155–170. https://doi.org/10.1007/s10488-020-01054-x
   PubMed Web of Science ®Google Scholar
• Caron, E. B., & Dozier, M. (2019). Effects of fidelity-focused consultation on clinicians’ implementation: An exploratory multiple baseline design. Administration and Policy in Mental Health and Mental Health Services Research, 46(4), 445–457. https://doi.org/10.1007/s10488-019-00924-3
   PubMed Web of Science ®Google Scholar
• Caron, E. B., & Dozier, M. (2022). Self-coding of fidelity as a potential active ingredient of consultation to improve clinicians’ fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 49(2), 237–254. https://doi.org/10.1007/s10488-021-01160-4
   PubMed Web of Science ®Google Scholar
• Chambers, D. A., Feero, W. G., & Khoury, M. J. (2016). Convergence of implementation science, precision medicine, and the learning health care system: A new model for biomedical research. JAMA, 315(18), 1941–1942. https://doi.org/10.1001/jama.2016.3867
   PubMed Web of Science ®Google Scholar
• Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77(3), 566–579. https://doi.org/10.1037/a0014565
   PubMed Web of Science ®Google Scholar
• Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
   Google Scholar
• Dopp, A. R., Borduin, C. M., White, I., Mark, H., & Kuppens, S. (2017). Family-based treatments for serious juvenile offenders: A multilevel meta-analysis. Journal of Consulting and Clinical Psychology, 85(4), 335. https://doi.org/10.1037/ccp0000183
   PubMed Web of Science ®Google Scholar
• Ehrenreich-May, J., Dimeff, L. A., Woodcock, E. A., Queen, A. H., Kelly, T., Contreras, I. S., … Kennedy, S. M. (2016). Enhancing online training in an evidence-based treatment for adolescent panic disorder: A randomized controlled trial. Evidence-Based Practice in Child and Adolescent Mental Health, 1(4), 241–258. https://doi.org/10.1080/23794925.2016.1245116
   Google Scholar
• Fleiss, J. L. (1981). Balanced incomplete block designs for inter-rater reliability studies. Applied Psychological Measurement, 5(1), 105–112. https://doi.org/10.1177/014662168100500115
   Web of Science ®Google Scholar
• Gamer, M., Lemon, J., Fellows, I., & Singh, P. (2012). Various coefficients of interrater reliability and agreement. R package version 0.84.1. https://cran.r-project.org/web/packages/irr/irr.pdf
   Google Scholar
• Garland, A. F., Hurlburt, M. S., Brookman-Frazee, L., Taylor, R. M., & Accurso, E. C. (2010). Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 208–220. https://doi.org/10.1007/s10488-009-0237-8
   PubMed Web of Science ®Google Scholar
• Gonder, J., Metlay, W., & Shapiro, T. (2018). Testing assumptions: Can performance rating feedback result in objective performance improvements? Journal of Management and Innovation, 4(2). https://doi.org/10.18059/jmi.v4i2.100
   Google Scholar
• Hogue, A., Bobek, M., Dauber, S., Henderson, C. E., McLeod, B. D., & Southam-Gerow, M. A. (2019). Core elements of family therapy for adolescent behavior problems: Empirical distillation of three manualized treatments. Journal of Clinical Child and Adolescent Psychology, 48(1), 29–41. https://doi.org/10.1080/15374416.2018.1555762
   PubMed Web of Science ®Google Scholar
• Hogue, A., Bobek, M., MacLean, A., Porter, N., Jensen-Doss, A., & Henderson, C. E. (2019). Measurement training and feedback system for implementation of evidence-based treatment for adolescent externalizing problems: Protocol for a randomized trial of pragmatic clinician training. Trials, 20(1), 1–12. https://doi.org/10.1186/s13063-019-3783-8
   PubMed Web of Science ®Google Scholar
• Hogue, A., Bobek, M., Porter, N., Dauber, S., Southam-Gerow, M. A., McLeod, B. D., & Henderson, C. E. (2021). Core elements of family therapy for adolescent behavioral health problems: Validity generalization in community settings. Journal of Clinical Child and Adolescent Psychology, 1–13. https://doi.org/10.1080/15374416.2021.1969939
   Web of Science ®Google Scholar
• Hogue, A., Bobek, M., Porter, N., MacLean, A., Bruynesteyn, L., Jensen-Doss, A., & Henderson, C. E. (2022). Therapist self-report of fidelity to core elements of family therapy for adolescent behavior problems: Psychometrics of a pragmatic quality indicator tool. Administration and Policy in Mental Health and Mental Health Services Research, 49(2), 298–311. https://doi.org/10.1007/s10488-021-01164-0
   PubMed Web of Science ®Google Scholar
• Hogue, A., Dauber, S., Bobek, M., Jensen-Doss, A., & Henderson, C. E. (2019). Measurement training and feedback system for implementation of family-based services for adolescent substance use: Protocol for a cluster randomized trial of two implementation strategies. Implementation Science, 14(1), 1–12. https://doi.org/10.1186/s13012-019-0874-6
   PubMedGoogle Scholar
• Hogue, A., Dauber, S., Henderson, C. E., & Liddle, H. A. (2014). Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy in Mental Health and Mental Health Services Research, 41(5), 697–705. https://doi.org/10.1007/s10488-013-0520-6
   PubMed Web of Science ®Google Scholar
• Hogue, A., Dauber, S., Lichvar, E., Bobek, M., & Henderson, C. E. (2015). Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: Correspondence between therapists and observers. Administration and Policy in Mental Health and Mental Health Services Research, 42(2), 229–243. https://doi.org/10.1007/s10488-014-0548-2
   PubMed Web of Science ®Google Scholar
• Hogue, A., Henderson, C. E., Becker, S. J., & Knight, D. K. (2018). Evidence base on outpatient behavioral treatments for adolescent substance use, 2014–2017: Outcomes, treatment delivery, and promising horizons. Journal of Clinical Child and Adolescent Psychology, 47(4), 499–526. https://doi.org/10.1080/15374416.2018.1466307
   PubMed Web of Science ®Google Scholar
• Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy: Theory, Research, Practice, & Training, 33(2), 332–345. https://doi.org/10.1037/0033-3204.33.2.332
   Web of Science ®Google Scholar
• Hogue, A., Ozechowski, T. J., Robbins, M. S., & Waldron, H. B. (2013). Making fidelity an intramural game: Localizing quality assurance procedures to promote sustainability of evidence‐based practices in usual care. Clinical Psychology: Science and Practice, 20(1), 60–77.https://doi.org/10.1111/cpsp.12023
   Web of Science ®Google Scholar
• Hogue, A., Porter, N., Bobek, M., MacLean, A., Bruynesteyn, L., Jensen-Doss, A., Dauber, S., & Henderson, C. E. (2022). Online training of community therapists in observational coding of family therapy techniques: Reliability and accuracy. Administration and Policy in Mental Health and Mental Health Services Research, 49(1), 139–151. https://doi.org/10.1007/s10488-021-01152-4
   PubMed Web of Science ®Google Scholar
• Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37(3), 230–244. https://doi.org/10.1007/s10488-009-0251-x
   PubMed Web of Science ®Google Scholar
• Isenhart, C., Dieperink, E., Thuras, P., Fuller, B., Stull, L., Koets, N., & Lenox, R. (2014). Training and maintaining motivational interviewing skills in a clinical trial. Journal of Substance Use, 19(1–2), 164–170. https://doi.org/10.3109/14659891.2013.765514
   Web of Science ®Google Scholar
• James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69(1), 85–98. https://doi.org/10.1037/0021-9010.69.1.85
   Web of Science ®Google Scholar
• Jensen-Doss, A., Smith, A. M., Walsh, L. M., Ringle, V. M., Casline, E., Patel, Z., … Webster, R. (2020). Preaching to the choir? Predictors of engagement in a community-based learning collaborative. Administration and Policy in Mental Health and Mental Health Services Research, 47(2), 279–290. https://doi.org/10.1007/s10488-019-00985-4
   Web of Science ®Google Scholar
• Kline, P. (1979). Psychometrics and psychology. Academic Press.
   Google Scholar
• Knight, D. K., Belenko, S., Wiley, T., Robertson, A. A., Arrigona, N., Dennis, M., … and Leukefeld, C. (2015). Juvenile Justice—Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS): a cluster randomized trial targeting system-wide improvement in substance use services. Implementation Science, 11(1), 1–18.
   Google Scholar
• Kobak, K. A., Wolitzky-Taylor, K., Craske, M. G., & Rose, R. D. (2017). Therapist training on cognitive behavior therapy for anxiety disorders using internet-based technologies. Cognitive Therapy and Research, 41(2), 252–265. https://doi.org/10.1007/s10608-016-9819-4
   PubMed Web of Science ®Google Scholar
• Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
   PubMed Web of Science ®Google Scholar
• Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1–26. https://doi.org/10.18637/jss.v082.i13
   Web of Science ®Google Scholar
• LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852. https://doi.org/10.1177/1094428106296642
   Web of Science ®Google Scholar
• Lock, J. (2015). An update on evidence-based psychosocial treatments for eating disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 44(5), 707–721. https://doi.org/10.1080/15374416.2014.971458
   PubMed Web of Science ®Google Scholar
• Lyon, A., & Koerner, K. (2016). User‐centered design for psychosocial intervention development and implementation. Clinical Psychology: Science and Practice, 23(2), 180–200. https://doi.org/10.1111/cpsp.12154
   PubMed Web of Science ®Google Scholar
• Martino, S., Ball, S., Nich, C., Frankforter, T. L., & Carroll, K. M. (2009). Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychotherapy Research, 19(2), 181–193. https://doi.org/10.1080/10503300802688460
   PubMed Web of Science ®Google Scholar
• Mastroleo, N. R., Humm, L., Williams, C. M., Kiluk, B. D., Hoadley, A., & Magill, M. (2020). Initial testing of a computer-based simulation training module to support clinicians’ acquisition of CBT skills for substance use disorder treatment. Journal of Substance Abuse Treatment, 114, 108014. https://doi.org/10.1016/j.jsat.2020.108014
   PubMed Web of Science ®Google Scholar
• McCart, M. R., & Sheidow, A. J. (2016). Evidence-based psychosocial treatments for adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology, 45(5), 529–563. https://doi.org/10.1080/15374416.2016.1146990
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Cox, J. R., Jensen‐Doss, A., Herschell, A., Ehrenreich‐May, J., & Wood, J. J. (2018). Proposing a mechanistic model of clinician training and consultation. Clinical Psychology: Science and Practice, 25(3), e12260. https://doi.org/10.1111/cpsp.12260
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Southam‐Gerow, M. A., Tully, C. B., Rodriguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clinical Psychology: Science and Practice, 20(1), 14–32. https://doi.org/10.1111/cpsp.12020
   PubMed Web of Science ®Google Scholar
• Minuchin, S., & Fishman, H. C. (1981). Family therapy techniques. Harvard University Press.
   Google Scholar
• R Core Team (2021). R: A language and environment for statistical computing. R foundation for statistical computing. http://www.R-project.org/
   Google Scholar
• RStudio Team (2020). RStudio: Integrated development for R. http://www.rstudio.com/
   Google Scholar
• Schafer, J. L., & Graham, J. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. https://doi.org/10.1037/1082-989X.7.2.147
   PubMed Web of Science ®Google Scholar
• Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. https://doi.org/10.1007/s10488-010-0321-0
   PubMed Web of Science ®Google Scholar
• Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298. https://doi.org/10.1037/a0019247
   PubMed Web of Science ®Google Scholar
• Shrout, P., & Fleiss, J. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
   PubMed Web of Science ®Google Scholar
• Sibley, M. H., Ortiz, M., Rios-Davis, A., Zulauf-mccurdy, C. A., Graziano, P. A., & Bickman, L. (2022). Stakeholder-generated implementation strategies to promote evidence-based ADHD treatment in community mental health. Administration and Policy in Mental Health and Mental Health Services Research, 49, 44–58. https://doi.org/10.1007/s10488-021-01143-5
   PubMed Web of Science ®Google Scholar
• Stewart, R. E., Williams, N., Byeon, Y. V., Buttenheim, A., Sridharan, S., Zentgraf, K., … Beidas, R. S. (2019). The clinician crowdsourcing challenge: Using participatory design to seed implementation strategies. Implementation Science, 14(1), 1–8. https://doi.org/10.1186/s13012-019-0914-2
   PubMedGoogle Scholar
• Stirman, S. W. (2020). Commentary: Challenges and opportunities in the assessment of fidelity and related constructs. Administration and Policy in Mental Health and Mental Health Services Research, 47(6), 932–934. https://doi.org/10.1007/s10488-020-01069-4
   PubMed Web of Science ®Google Scholar
• Tanner-Smith, E. E., Wilson, S. J., & Lipsey, M. W. (2013). The comparative effectiveness of outpatient treatment for adolescent substance abuse: A meta-analysis. Journal of Substance Abuse Treatment, 44(2), 145–158. https://doi.org/10.1016/j.jsat.2012.05.006
   PubMed Web of Science ®Google Scholar
• Valenstein-Mah, H., Greer, N., McKenzie, L., Hansen, L., Strom, T. Q., Wiltsey Stirman, S., … Kehle-Forbes, S. M. (2020). Effectiveness of training methods for delivery of evidence-based psychotherapies: A systematic review. Implementation Science, 15(1), 1–17. https://doi.org/10.1186/s13012-020-00998-w
   PubMed Web of Science ®Google Scholar
• Wain, R. M., Kutner, B. A., Smith, J. L., Carpenter, K. M., Hu, M. C., Amrhein, P. C., & Nunes, E. V. (2015). Self-report after randomly assigned supervision does not predict ability to practice motivational interviewing. Journal of Substance Abuse Treatment, 57, 96–101. https://doi.org/10.1016/j.jsat.2015.04.006
   PubMed Web of Science ®Google Scholar
• Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78(2), 200–211. https://doi.org/10.1037/a0018912
   PubMed Web of Science ®Google Scholar
• Weersing, R. V., Jeffreys, M., Do, M., Schwartz, K., & Bolano, C. (2017). Evidence base update of psychosocial treatments for child and adolescent depression. Journal of Clinical Child and Adolescent Psychology, 46(1), 11–43. https://doi.org/10.1080/15374416.2016.1220310
   PubMed Web of Science ®Google Scholar
• Weisz, J. R., Bearman, S., Santucci, L., & Jensen-Doss, A. (2017). Initial test of a principle-guided approach to transdiagnostic psychotherapy with children and adolescents. Journal of Clinical Child and Adolescent Psychology, 46(1), 44–58. https://doi.org/10.1080/15374416.2016.1163708
   PubMed Web of Science ®Google Scholar