External and Internal Validity Considerations in Youth Effectiveness Trials: Lessons Learned from the COMET Study

References

• Amaya-Jackson, L., Hagele, D., Sideris, J., Potter, D., Briggs, E. C., Keen, L., Murphy, R. A., Dorsey, S., Patchett, V., Ake, G. S., & Socolar, R. (2018). Pilot to policy: Statewide dissemination and implementation of evidence-based treatment for traumatized youth. BMC Health Services Research, 18(1), 589. https://doi.org/10.1186/s12913-018-3395-0
   PubMedGoogle Scholar
• Angold, A., Costello, E. J., Messer, S. C., & Pickles, A. (1995). Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. International Journal of Methods in Psychiatric Research, 5, 237–249.
   Web of Science ®Google Scholar
• Bartlett, J. D., Barto, B., Griffin, J. L., Fraser, J. G., Hodgdon, H., & Bodian, R. (2016). Trauma-informed care in the Massachusetts child trauma project. Child Maltreatment, 21(2), 101–112. https://doi.org/10.1177/1077559515615700
   PubMed Web of Science ®Google Scholar
• Becker, S. J. (2015). Direct-to-consumer marketing: A complementary approach to traditional dissemination and implementation efforts for mental health and substance abuse interventions. Clinical Psychology, 22(1), 85–100. https://doi.org/10.1111/cpsp.12086
   Google Scholar
• Beidas, R. S., Marcus, S., Wolk, C. B., Powell, B., Aarons, G. A., Evans, A. C., Hurford, M. O., Hadley, T., Adams, D. R., Walsh, L. M., Babbar, S., Barg, F., & Mandell, D. S. (2016). A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research, 43(5), 640–649. https://doi.org/10.1007/s10488-015-0673-6
   PubMed Web of Science ®Google Scholar
• Beidas, R. S., Saldana, L., & Shelton, R. C. (2023). Testing psychosocial interventions in the contexts they are meant to be delivered. Journal of Consulting and Clinical Psychology, 91(4), 189–191. https://doi.org/10.1037/ccp0000797
   PubMed Web of Science ®Google Scholar
• Ben-Dror, R. (1994). Employee turnover in community mental health organization: A developmental stages study. Community Mental Health Journal, 30(3), 243–257. https://doi.org/10.1007/BF02188885
   PubMed Web of Science ®Google Scholar
• Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., & Neer, S. M. (1997). The screen for child anxiety related emotional disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child & Adolescent Psychiatry, 36(4), 545–553. https://doi.org/10.1097/00004583-199704000-00018
   PubMed Web of Science ®Google Scholar
• Burlingame, G. M., Cox, J. C., Wells, M. G., Lambert, M. J., Latkowski, M., & Ferre, R. (2005). The administration and scoring manual of the youth outcome questionnaire. American Professional Credentialing Services.
   Google Scholar
• Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs. Medical Care, 50(3), 217–226. https://doi.org/10.1097/MLR.0b013e3182408812
   PubMed Web of Science ®Google Scholar
• Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50
   PubMed Web of Science ®Google Scholar
• Ehrenreich-May, J., Kennedy, S. M., Sherman, J. A., Bilek, E. L., Buzzella, B. A., Bennett, S. M., & Barlow, D. H. (2017a). Unified protocols for transdiagnostic treatment of emotional disorders in children and adolescents: Therapist guide. Oxford University Press.
   Google Scholar
• Ehrenreich-May, J., Rosenfield, D., Queen, A. H., Kennedy, S. M., Remmes, C. S., & Barlow, D. H. (2017b). An initial waitlist-controlled trial of the unified protocol for the treatment of emotional disorders in adolescents. Journal of Anxiety Disorders, 46, 46–55. https://doi.org/10.1016/j.janxdis.2016.10.006
   PubMed Web of Science ®Google Scholar
• Fredericks, S., Sidani, S., Fox, M., & Miranda, J. (2019). Strategies for balancing internal and external validity in evaluations of interventions. Nurse Researcher, 27(4), 19–23. https://doi.org/10.7748/nr.2019.e1646
   PubMed Web of Science ®Google Scholar
• Glasgow, R. E., Lichtenstein, E., & Marcus, A. C. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health, 93(8), 1261–1267. https://doi.org/10.2105/AJPH.93.8.1261
   PubMed Web of Science ®Google Scholar
• Goodman, R., Meltzer, H., & Bailey, V. (1998). The strengths and difficulties questionnaire: A pilot study on the validity of the self-report version. European Child & Adolescent Psychiatry, 7(3), 125–130. https://doi.org/10.1007/s007870050057
   PubMed Web of Science ®Google Scholar
• Guy, W. (1976). Clinical global impressions. Assessment Manual for Psychopharmacology, 218–222.
   Google Scholar
• Hariton, E., & Locascio, J. J. (2018). Randomised controlled trials—the gold standard for effectiveness research. BJOG: An International Journal of Obstetrics and Gynaecology, 125(13), 1716. https://doi.org/10.1111/1471-0528.15199
   PubMed Web of Science ®Google Scholar
• Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. https://doi.org/10.1016/j.cpr.2010.02.005
   PubMed Web of Science ®Google Scholar
• Jensen-Doss, A., Ehrenreich-May, J., Nanda, M. M., Maxwell, C. A., LoCurto, J., Shaw, A. M., Souer, H., Rosenfield, D., & Ginsburg, G. S. (2018). Community study of outcome monitoring for emotional disorders in teens (COMET): A comparative effectiveness trial of a transdiagnostic treatment and a measurement feedback system. Contemporary Clinical Trials, 74, 18–24. https://doi.org/10.1016/j.cct.2018.09.011
   PubMed Web of Science ®Google Scholar
• Jensen-Doss, A., Smith, A. M., Walsh, L. M., Mora Ringle, V., Casline, E., Patel, Z., Shaw, A. M., Maxwell, C., Hanson, R., & Webster, R. (2020). Preaching to the choir? Predictors of engagement in a community-based learning collaborative. Administration and Policy in Mental Health and Mental Health Services Research, 47(2), 279–290. https://doi.org/10.1007/s10488-019-00985-4
   Web of Science ®Google Scholar
• Kendall, P. C., Compton, S. N., Walkup, J. T., Birmaher, B., Albano, A. M., Sherrill, J., Ginsburg, G., Rynn, M., McCracken, J., Gosch, E., Keeton, C., Bergman, L., Sakolsky, D., Suveg, C., Iyengar, S., March, J., & Piacentini, J. (2010). Clinical characteristics of anxiety disordered youth. Journal of Anxiety Disorders, 24(3), 360–365. https://doi.org/10.1016/j.janxdis.2010.01.009
   PubMed Web of Science ®Google Scholar
• Kennedy-Martin, T., Curtis, S., Faries, D., Robinson, S., & Johnston, J. (2015). A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials, 16(1), 495. https://doi.org/10.1186/s13063-015-1023-4
   PubMed Web of Science ®Google Scholar
• Lane-Fall, M. B., Curran, G. M., & Beidas, R. S. (2019). Scoping implementation science for the beginner: Locating yourself on the “subway line” of translational research. BMC Medical Research Methodology, 19(1), 133. https://doi.org/10.1186/s12874-019-0783-z
   PubMedGoogle Scholar
• Lyon, A. R., Liu, F. F., Connors, E. H., King, K. M., Coifman, J. I., Cook, H., McRee, E., Ludwig, K., Law, A., Dorsey, S., & McCauley, E. (2022). How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement- based care. Implementation Science Communications, 3(1), 79. https://doi.org/10.1186/s43058-022-00325-y
   PubMedGoogle Scholar
• Marchette, L. K., & Weisz, J. R. (2017). Practitioner review: Empirical evolution of youth psychotherapy toward transdiagnostic approaches. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 58(9), 970–984. https://doi.org/10.1111/jcpp.12747
   PubMed Web of Science ®Google Scholar
• Messer, S. C., Angold, A., Costello, E. J., Loeber, R., Van Kammen, W., & Stouthamer-Loeber, M. (1995). Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents: Factor composition and structure across development. International Journal of Methods in Psychiatric Research, 5, 251–262.
   Web of Science ®Google Scholar
• Miklowitz, D. J., & Clarkin, J. F. (1999). Balancing internal and external validity. Prevention & Treatment, 2(1). Article 4c. https://doi.org/10.1037/1522-3736.2.1.24c
   Google Scholar
• National Institute of Mental Health. (2014). Clinical trials to test the effectiveness of treatment, preventive, and services interventions (Collaborative R01; RFA-MH-15-325). https://grants.nih.gov/grants/guide/rfa-files/RFA-MH-15-320.html
   Google Scholar
• National Institute of Mental Health. (n.d.). Support for clinical trials at NIMH. Retrieved October 14, 2022, from https://www.nimh.nih.gov/funding/opportunities-announcements/clinical-trials-foas#ctfoa
   Google Scholar
• Patel, Z. S., Philips, D., Casline, E., Aarons, G. A., Maxwell, C. A., Ginsburg, G. S., Ehrenreich-May, J., & Jensen-Doss, A. (2022). Who participates in effectiveness research? A comparison of effectiveness trial clinicians to national survey samples. Administration and Policy in Mental Health and Mental Health Services Research, 49(5), 899–908. https://doi.org/10.1007/s10488-022-01202-5
   PubMed Web of Science ®Google Scholar
• Porzsolt, F., Rocha, N. G., Toledo-Arruda, A. C., Thomaz, T. G., Moraes, C., Bessa-Guerra, T. R., Leão, M., Migowski, A., Araujo da Silva, A. R., & Weiss, C. (2015). Efficacy and effectiveness trials have different goals, use different tools, and generate different messages. Pragmatic and Observational Research, 6, 47–54. https://doi.org/10.2147/POR.S89946
   PubMedGoogle Scholar
• Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the expert recommendations for implementing change (ERIC) project. Implementation Science, 10(1), 21–21. https://doi.org/10.1186/s13012-015-0209-1
   PubMed Web of Science ®Google Scholar
• Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., & Aluwahlia, S. (1983). A children’s global assessment scale (CGAS). Archives of General Psychiatry, 40(11), 1228–1231. https://doi.org/10.1001/archpsyc.1983.01790100074010
   PubMed Web of Science ®Google Scholar
• Silverman, W. K., Albano, A. M., & Kerns, C. M. (2023). Anxiety disorder interview schedule for the DSM-5, child and parent versions. Oxford University Press.
   Google Scholar
• Singal, A. G., Higgins, P. D. R., & Waljee, A. K. (2014). A primer on effectiveness and efficacy trials. Clinical and Translational Gastroenterology, 5(1), e45. https://doi.org/10.1038/ctg.2013.13
   PubMed Web of Science ®Google Scholar
• Treatment for Adolescents with Depression Study (TADS) Team. (2005). The treatment for adolescents with depression study (TADS): Demographic and clinical characteristics. The Journal of the American Academy of Child & Adolescent Psychiatry, 44(1), 28–40. https://doi.org/10.1097/01.chi.0000145807.09027.82
   PubMed Web of Science ®Google Scholar
• U.S. Census Bureau. (2021a). 2021 poverty guidelines. ASPE. https://aspe.hhs.gov/2021-poverty-guidelines
   Google Scholar
• U.S. Census Bureau. (2021b). Real median family income in the United States. FRED, Federal Reserve Bank of St. Louis; FRED, Federal Reserve Bank of St. Louis. https://fred.stlouisfed.org/series/MEFAINUSA672N
   Google Scholar
• U.S. Census Bureau. (2022a). U.S. Census Bureau QuickFacts: Broward county. https://www.census.gov/quickfacts/browardcountyflorida
   Google Scholar
• U.S. Census Bureau. (2022b). U.S. census Bureau QuickFacts: Connecticut. https://www.census.gov/quickfacts/CT
   Google Scholar
• U.S. Census Bureau. (2022c). U.S. Census Bureau QuickFacts: Miami-Dade county. https://www.census.gov/quickfacts/fact/table/miamidadecountyflorida/POP060210
   Google Scholar
• Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care a multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761. https://doi.org/10.1001/jamapsychiatry.2013.1176
   PubMed Web of Science ®Google Scholar
• Weisz, J. R., Weiss, B., & Donenberg, G. R. (1992). The lab versus the clinic: Effects of child and adolescent psychotherapy. Annual Progress in Child Psychiatry & Child Development, 47(12), 1578–1585. https://doi.org/10.1037/0003-066X.47.12.1578
   Google Scholar
• Williams, N. J., & Beidas, R. S. (2019). Annual Research Review: The state of implementation science in child psychology and psychiatry: A review and suggestions to advance the field. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 60(4), 430–450. https://doi.org/10.1111/jcpp.12960
   PubMed Web of Science ®Google Scholar