Measurement-based Care as a Practice Improvement Tool: Clinical and Organizational Applications in Youth Mental Health

References

• Aarons, G. A., Sommerfeld, D. H., & Willging, C. E. (2011). The soft underbelly of system change: The role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychological Services, 8(4), 269. https://doi.org/10.1037/a0026196
   PubMed Web of Science ®Google Scholar
• Ackerman, S. J., Benjamin, L. S., Beutler, L. E., Gelso, C. J., Goldfried, M. R., Hill, C., Lambert, M. J., Norcross, J. C., Orlinsky, D. E., & Rainer, J. (2001). Empirically supported therapy relationships: Conclusions and recommendations fo the Division 29 Task Force. Psychotherapy: Theory, Research, Practice, Training, 38(4), 495–497. https://doi.org/10.1037/0033-3204.38.4.495
   Web of Science ®Google Scholar
• Agency for Healthcare Research and Quality. Quality Improvement in Primary Care. Content last reviewed April 2020. Rockville, MD: Agency for Healthcare Research and Quality. https://www.ahrq.gov/research/findings/factsheets/quality/qipc/index.html
   Google Scholar
• Akinci, F., & Patel, P. M. (2014). Quality improvement in healthcare delivery utilizing the patient-centered medical home model. Hospital Topics, 92(4), 96–104. https://doi.org/10.1080/00185868.2014.968493
   PubMedGoogle Scholar
• American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285. https://doi.org/10.1037/0003-066X.61.4.271
   PubMed Web of Science ®Google Scholar
• American Psychological Association Task Force on Evidence-Based Practice for Children and Adolescents. (2008). Disseminating evidence-based practice for children and adolescents: A systems approach to enhancing care. American Psychological Association.
   Google Scholar
• Athay, M. M., Riemer, M., & Bickman, L. (2012). The Symptoms and Functioning Severity Scale (SFSS): Psychometric evaluation and discrepancies among youth, caregiver, and clinician ratings over time. Administration and Policy in Mental Health and Mental Health Services Research, 39(1–2), 13–29. https://doi.org/10.1007/s10488-012-0403-2
   PubMed Web of Science ®Google Scholar
• Bao, Y., Casalino, L. P., & Pincus, H. A. (2013). Behavioral health and health care reform models: Patient-centered medical home, health home, and accountable care organization. The Journal of Behavioral Health Services & Research, 40(1), 121–132. https://doi.org/10.1007/s11414-012-9306-y
   PubMed Web of Science ®Google Scholar
• Bergman, H., Kornør, H., Nikolakopoulou, A., Hanssen‐Bauer, K., Soares‐Weiser, K., Tollefsen, T. K., & Bjørndal, A. (2018). Client feedback in psychological therapy for children and adolescents with mental health problems. Cochrane Database of Systematic Reviews, 8, Art. No.: CD011729. https://doi.org/10.1002/14651858.CD011729.pub2
   PubMedGoogle Scholar
• Bickman, L., Athay, M., Riemer, M., Lambert, E. W., Douglas Kelley, S., Breda, C., Tempesti, T., Dew-Reeves, S. E., Brannan, A. M., & Vides de Andrade, A. R. (2010). Manual of the peabody treatment progress battery (2nd ed.). Vanderbilt University.
   Google Scholar
• Bickman, L., de Andrade, A. R. V., Athay, M. M., Chen, J. I., De Nadai, A. S., Jordan-Arthur, B. L., & Karver, M. S. (2012). The relationship between change in therapeutic alliance ratings and improvement in youth symptom severity: Whose ratings matter the most? Administration and Policy in Mental Health and Mental Health Services Research, 39(1–2), 78–89. https://doi.org/10.1007/s10488-011-0398-0
   PubMed Web of Science ®Google Scholar
• Bickman, L., Douglas Kelley, S., Breda, C., De Andrade, A. R. V., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423–1429. https://doi.org/10.1176/appi.ps.002052011
   PubMed Web of Science ®Google Scholar
• Bickman, L., Douglas, S., De Andrade, A. R. V., Tomlinson, M., Gleacher, A., Olin, S., & Hoagwood, K. (2016). Implementing a measurement feedback system: A tale of two sites. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 410–425. https://doi.org/10.1007/s10488-015-0647-8
   PubMed Web of Science ®Google Scholar
• Bickman, L., Lyon, A. R., & Wolpert, M. (2016). Achieving precision mental health through effective assessment, monitoring, and feedback processes. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 271–276. https://doi.org/10.1007/s10488-016-0718-5
   PubMed Web of Science ®Google Scholar
• Boysen, P. G. (2013). Just culture: A foundation for balanced accountability and patient safety. Ochsner Journal, 13(3), 400–406. https://doi.org/10.1002/14651858.CD011729.pub2
   PubMedGoogle Scholar
• Brattland, H., Koksvik, J. M., Burkeland, O., Klöckner, C. A., Lara-Cabrera, M. L., Miller, S. D., Wampold, B., Ryum, T., & Iversen, V. C. (2019). Does the working alliance mediate the effect of routine outcome monitoring (ROM) and alliance feedback on psychotherapy outcomes? A secondary analysis from a randomized clinical trial. Journal of Counseling Psychology, 66(2), 234–246. https://doi.org/10.1037/cou0000320
   PubMed Web of Science ®Google Scholar
• Breda, C. S., & Riemer, M. (2012). Motivation for Youth’s Treatment Scale (MYTS): A new tool for measuring motivation among youths and their caregivers. Administration and Policy in Mental Health and Mental Health Services Research, 39(1–2), 118–132. https://doi.org/10.1007/s10488-012-0408-x
   PubMed Web of Science ®Google Scholar
• Brown, B., Gude, W. T., Blakeman, T., van der Veer, S. N., Ivers, N., Francis, J. J., Lorencatto, F., Presseau, J., Peek, N., & Daker-White, G. (2019). Clinical Performance Feedback Intervention Theory (CP-FIT): A new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implementation Science, 14(1), 40. https://doi.org/10.1186/s13012-019-0883-5
   PubMedGoogle Scholar
• Burlingame, G. M., Cox, J. C., Wells, M. G., Lambert, M. J., Latkowski, M., & Ferre, R. (2005). The administration and scoring manual of the Youth Outcome Questionnaire. American Professional Credentialing Services.
   Google Scholar
• Cannon, J. A. N., Warren, J. S., Nelson, P. L., & Burlingame, G. M. (2010). Change trajectories for the Youth Outcome Questionnaire self-report: Identifying youth at risk for treatment failure. Journal of Clinical Child and Adolescent Psychology, 39(3), 289–301. https://doi.org/10.1080/15374411003691727
   PubMed Web of Science ®Google Scholar
• Carlier, Ingrid V. E.., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, Nic J. A.., & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluation in Clinical Practice, 18, 104–110. https://doi.org/10.1111/jep.2012.18.issue-1 1
   PubMed Web of Science ®Google Scholar
• Centers for Medicare & Medicaid Services. (2016). CMS quality measure development plan: Supporting the transition to the Merit-based incentive payment system (MIPS) and alternative payment models (APMs). Health Services Advisory Group, Inc.
   Google Scholar
• Claiborn, C. D., & Goodyear, R. K. (2005). Feedback in psychotherapy. Journal of Clinical Psychology, 61(2), 209–217. https://doi.org/10.1002/jclp.20112
   PubMed Web of Science ®Google Scholar
• Coalition for the Advancement and Application of Psychological Science. (2018). evidence-based practice decision-making for mental and behavioral health care. Anxiety and Depression Association of America
   Google Scholar
• Cohen, J. A., Mannarino, A. P., & Deblinger, E. (2012). Trauma-focused CBT for children and adolescents: Treatment applications. Guilford Press.
   Google Scholar
• De Jong, K. (2017). Effectiveness of progress monitoring: A meta-analysis. The 48th International Annual Meeting for the Society for Psychotherapy Research, Toronto, CA.
   Google Scholar
• De Jong, K., Timman, R., Hakkaart-Van Roijen, L., Vermeulen, P., Kooiman, K., Passchier, J., & Van Busschbach, J. (2014). The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research, 24(6), 629–639. https://doi.org/10.1080/10503307.2013.871079
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D. A., Burgers, D. E., & Rabinowitz, J. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900. https://doi.org/10.1037/a0038498
   PubMed Web of Science ®Google Scholar
• Delgadillo, J., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Gilbody, S., Ali, S., Aguirre, E., Appleton, M., Nevin, J., O’Hayon, H., Patel, U., Sainty, A., Spencer, P., & McMillan, D. (2018). Feedback-informed treatment versus usual psychological treatment for depression and anxiety: A multisite, open-label, cluster randomised controlled trial. The Lancet Psychiatry, 5(7), 564–572. https://doi.org/10.1016/S2215-0366(18)30162-7
   PubMed Web of Science ®Google Scholar
• Delgadillo, J., Overend, K., Lucock, M., Groom, M., Kirby, N., McMillan, D., Gilbody, S., Lutz, W., Rubel, J. A., & de Jong, K. (2017). Improving the efficiency of psychological treatment using outcome feedback technology. Behaviour Research and Therapy, 99, 89–97. https://doi.org/10.1016/j.brat.2017.09.011
   PubMed Web of Science ®Google Scholar
• Dishop, C. R., Green, A. E., Torres, E., & Aarons, G. A. (2019). Predicting turnover: The moderating effect of functional climates on emotional exhaustion and work attitudes. Community Mental Health Journal, 55(5), 733–741. https://doi.org/10.1007/s10597-019-00407-7
   PubMed Web of Science ®Google Scholar
• Douglas Kelley, S., Bickman, L., & Norwood, E. (2010). Evidence-based treatments and common factors in youth psychotherapy. In B. L. Duncan, S. D. Miller, B. E. Wampold, & M. A. Hubble (Eds.), The heart and soul of change: Delivering what works in therapy (pp. 325–355). American Psychological Association.
   Google Scholar
• Douglas Kelley, S., & Bickman, L. (2009). Beyond outcomes monitoring: Measurement feedback systems in child and adolescent clinical practice. Current Opinion in Psychiatry, 22(4), 363–368. https://doi.org/10.1097/YCO.0b013e32832c9162
   PubMed Web of Science ®Google Scholar
• Douglas, S., Button, S., & Casey, S. E. (2016). Implementing for sustainability: Promoting use of a measurement feedback system for innovation and quality improvement. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 286–291. https://doi.org/10.1007/s10488-014-0607-8
   PubMed Web of Science ®Google Scholar
• Douglas, S., Jonghyuk, B., de Andrade, A. R. V., Tomlinson, M. M., Hargraves, R. P., & Bickman, L. (2015). Feedback mechanisms of change: How problem alerts reported by youth clients and their caregivers impact clinician-reported session content. Psychotherapy Research, 25(6), 678–693. https://doi.org/10.1080/10503307.2015.1059966
   PubMed Web of Science ®Google Scholar
• Dyer, K., Hooke, G. R., & Page, A. C. (2016). Effects of providing domain specific progress monitoring and feedback to therapists and patients on outcome. Psychotherapy Research, 26(3), 297–306. https://doi.org/10.1080/10503307.2014.983207
   PubMed Web of Science ®Google Scholar
• Ehrenreich-May, J., Kennedy, S. M., Sherman, J. A., Bilek, E. L., Buzzella, B. A., Bennett, S. M., & Barlow, D. H. (2017). Unified protocols for transdiagnostic treatment of emotional disorders in children and adolescents: Therapist guide. Oxford University Press.
   Google Scholar
• Ehrenreich-May, J., Southam-Gerow, M. A., Hourigan, S. E., Wright, L. R., Pincus, D. B., & Weisz, J. R. (2011). Characteristics of anxious and depressed youth seen in two different clinical contexts. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 398–411. https://doi.org/10.1007/s10488-010-0328-6
   PubMed Web of Science ®Google Scholar
• Eyberg, S. M., & Funderburg, B. (2011). Parent–child interaction therapy protocol. PCIT International.
   Google Scholar
• Finn, S. E., & Tonsager, M. E. (1997). Information-gathering and therapeutic models of assessment: Complementary paradigms. Psychological Assessment, 9(4), 374–385. https://doi.org/10.1037/1040-3590.9.4.374
   Web of Science ®Google Scholar
• Fortney, J. C., Unützer, J., Wrenn, G., Pyne, J. M., Smith, G. R., Schoenbaum, M., & Harbin, H. T. (2017). A tipping point for measurement-based care. Psychiatric Services, 68(2), 179–188. https://doi.org/10.1176/appi.ps.201500439
   PubMed Web of Science ®Google Scholar
• Fox, J. (2007). The uncertain relationship between transparency and accountability. Development in Practice, 17(4–5), 663–671. https://doi.org/10.1080/09614520701469955
   Google Scholar
• Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163. https://doi.org/10.1002/jclp.20108
   PubMed Web of Science ®Google Scholar
• Harmon, C., Hawkins, E. J., Lambert, M. J., Slade, K., & Whipple, J. L. (2005). Improving outcomes ior poorly desponding clients: The use of clinical support tools and feedback to clients. Journal of Clinical Psychology, 61(2), 175–185. https://doi.org/10.1002/jclp.20109
   PubMed Web of Science ®Google Scholar
• Harmon, S. C., Lambert, M. J., Smart, D. M., Hawkins, E., Nielsen, S. L., Slade, K., & Lutz, W. (2007) Enhancing outcome for potential treatment failures: Therapist–client feedback and clinical support tools, Psychotherapy Research, 17(4), 379-392. https://doi.org/10.1080/10503300600702331
   Web of Science ®Google Scholar
• Hooke, G. R., & Page, A. C. (2017, June). Tailored progress monitoring questions that respond to patient responses dynamically: How does this inform clinicians, how does this assist patient treatment? International Meeting of the Society for Psychotherapy Research, Toronto, CA.
   Google Scholar
• Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037/0022-006X.59.1.12
   PubMed Web of Science ®Google Scholar
• Janse, P. D., Jong, K. D., Dijk, M. K. V., Hutschemaekers, G. J. M., & Verbraak, M. J. P. M. (2017). Improving the efficiency of cognitive-behavioural therapy by using formal client feedback. Psychotherapy Research, 27(5), 525–538. https://doi.org/10.1080/10503307.2016.1152408
   PubMed Web of Science ®Google Scholar
• Jensen-Doss, A., Ehrenreich-May, J., Nanda, M. M., Maxwell, C. A., LoCurto, J., Shaw, A. M., Souer, H., Rosenfield, D., & Ginsburg, G. S. (2018). Community Study of Outcome Monitoring for Emotional Disorders in Teens (COMET): A comparative effectiveness trial of a transdiagnostic treatment and a measurement feedback system. Contemporary Clinical Trials, 74, 18–24. https://doi.org/10.1016/j.cct.2018.09.011
   PubMed Web of Science ®Google Scholar
• Kaplan, G. S. (2018, November 9). Building a culture of transparency in health care. Harvard Business Review. https://hbr.org/2018/11/building-a-culture-of-transparency-in-health-care
   Google Scholar
• Kendall, P. C., & Beidas, R. S. (2007). Smoothing the trail for dissemination of evidence-based practices for youth: Flexibility within fidelity. Professional Psychology: Research and Practice, 38(1), 13–20. https://doi.org/10.1037/0735-7028.38.1.13
   Web of Science ®Google Scholar
• Kendrick, T., El-Gohary, M., Stuart, B., Gilbody, S., Churchill, R., Aiken, L., Bhattacharya, A., Gimson, A., Brütt, A. L., de Jong, K., & Moore, M. (2016). Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. In The Cochrane Collaboration (Ed.), Cochrane database of systematic reviews. John Wiley & Sons, Ltd. https://doi.org/10.1002/14651858.CD011119.pub2
   Google Scholar
• King, D., & Hodges, K. (2013). Outcomes-driven clinical management and supervisory practices with youth with severe emotional disturbance. Administration in Social Work, 37(3), 312–324. https://doi.org/10.1080/03643107.2012.691080
   Web of Science ®Google Scholar
• Lambert, M. J. (2015). Progress feedback and the OQ-system: The past and the future. Psychotherapy, 52(4), 381–390. https://doi.org/10.1037/pst0000027
   PubMed Web of Science ®Google Scholar
• Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159–172. https://doi.org/10.1037/0022-006X.69.2.159
   PubMed Web of Science ®Google Scholar
• Levinson, W., Lesser, C. S., & Epstein, R. M. (2010). Developing physician communication skills for patient-centered care. Health Affairs, 29(7), 1310–1318. https://doi.org/10.1377/hlthaff.2009.0450
   PubMed Web of Science ®Google Scholar
• Lewis, C. C., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., Hoffman, M., Scott, K., Lyon, A., Douglas, S., Simon, G., & Kroenke, K. (2019). Implementing measurement-based care in behavioral health: A review. JAMA Psychiatry, 76(3), 324–335. https://doi.org/10.1001/jamapsychiatry.2018.3329
   PubMed Web of Science ®Google Scholar
• Lyon, A. R., Connors, E., Jensen-Doss, A., Landes, S. J., Lewis, C. C., McLeod, B. D., Rutt, C., Stanick, C., & Weiner, B. J. (2017). Intentional research design in implementation science: Implications for the use of nomothetic and idiographic assessment. Translational Behavioral Medicine: Practice, Policy, Research, 7(3), 567–580. https://doi.org/10.1007/s13142-017-0464-6
   PubMed Web of Science ®Google Scholar
• McLeod, B. D., Jensen-Doss, A., & Ollendick, T. H. (2013). Case conceptualization, treatment planning, and outcome monitoring. In B. D. McLeod, A. Jensen-Doss, & T. H. Ollendick (Eds.), Diagnostic and behavioral assessment in children and adolescents: A clinical guide (pp. 77–100). Guilford Press.
   Google Scholar
• Miller, S. D., Duncan, B. L., Brown, J., Sorrell, R., & Chalk, M. B. (2006). Using formal client feedback to improve retention and outcome: Making ongoing, real-time assessment feasible. Journal of Brief Therapy, 5(1), 5–22.
   Google Scholar
• Moltu, C., Veseth, M., Stefansen, J., Nøtnes, J. C., Skjølberg, Å., Binder, P.-E., Castonguay, L. G., & Nordberg, S. S. (2018). This is what I need a clinical feedback system to do for me: A qualitative inquiry into therapists’ and patients’ perspectives. Psychotherapy Research, 28(2), 250–263. https://doi.org/10.1080/10503307.2016.1189619
   PubMed Web of Science ®Google Scholar
• Muir, H. J., Coyne, A. E., Morrison, N. R., Boswell, J. F., & Constantino, M. J. (2019). Ethical implications of routine outcomes monitoring for patients, psychotherapists, and mental health care systems. Psychotherapy, 56(4), 459–469. https://doi.org/10.1037/pst0000246
   PubMed Web of Science ®Google Scholar
• National Patient Safety Foundation’s Lucian Leape Institute. (2015). Shining a light: Safer health care through transparency. National Patient Safety Foundation.
   Google Scholar
• National Quality Forum. (2018). Behavioral health and substance use fall 2017 cycle: CDP report. National Quality Forum. http://www.qualityforum.org/Publications/2018/07/Behavioral_Health_and_Substance_Use_Fall_2017_Final_Report.aspx
   Google Scholar
• Ofonedu, M. E., Belcher, H. M., Budhathoki, C., & Gross, D. A. (2017). Understanding barriers to initial treatment engagement among underserved families seeking mental health services. Journal of Child and Family Studies, 26(3), 863–876. https://doi.org/10.1007/s10826-016-0603-6
   PubMed Web of Science ®Google Scholar
• Porter, M. E., Larsson, S., & Lee, T. H. (2016). Standardizing patient outcomes measurement. New England Journal of Medicine, 374(6), 504–506. https://doi.org/10.1056/NEJMp1511701
   PubMed Web of Science ®Google Scholar
• Reese, R. J., Norsworthy, L. A., & Rowlands, S. R. (2009). Does a continuous feedback system improve psychotherapy outcome? Psychotherapy: Theory, Research, Practice, Training, 46(4), 418–431. https://doi.org/10.1037/a0017901
   PubMed Web of Science ®Google Scholar
• Riemer, M., & Bickman, L. (2011). Using program theory to link social psychology and program evaluation. In M. M. Mark, S. I. Donaldson, & B. Campbell (Eds.), Social psychology and program evaluation (pp. 102–139). Guilford Press.
   Google Scholar
• Riemer, M., Rosof-Williams, J., & Bickman, L. (2005). Theories related to changing clinician practice. Child and Adolescent Psychiatric Clinics of North America, 14(2), 241–254. https://doi.org/10.1016/j.chc.2004.05.002
   PubMed Web of Science ®Google Scholar
• Roter, D. L., Erby, L. H., Adams, A., Buckingham, C. D., Vail, L., Realpe, A., Larson, S., & Hall, J. A. (2014). Talking about depression: An analogue study of physician gender and communication style on patient disclosures. Patient Education and Counseling, 96(3), 339–345. https://doi.org/10.1016/j.pec.2014.05.006
   PubMed Web of Science ®Google Scholar
• Sachser, C., Berliner, L., Holt, T., Jensen, T. K., Jungbluth, N., Risch, E., Rosner, R., & Goldbeck, L. (2017). International development and psychometric properties of the Child and Adolescent Trauma Screen (CATS). Journal of Affective Disorders, 210, 189–195. https://doi.org/10.1016/j.jad.2016.12.040
   PubMed Web of Science ®Google Scholar
• Schnackenberg, A. K., & Tomlinson, E. C. (2016). Organizational transparency: A new perspective on managing trust in organization-stakeholder relationships. Journal of Management, 42(7), 1784–1810. https://doi.org/10.1177/0149206314525202
   Web of Science ®Google Scholar
• Schuster, J., Nikolajski, C., Kogan, J., Kang, C., Schake, P., Carney, T., Morton, S. C., & Reynolds, C. F., III. (2018). A payer-guided approach to widespread diffusion of behavioral health homes in real-world settings. Health Affairs, 37(2), 248–256. https://doi.org/10.1377/hlthaff.2017.1115
   Web of Science ®Google Scholar
• Scott, K., & Lewis, C. C. (2015). Using measurement-based care to enhance any treatment. Cognitive and Behavioral Practice, 22(1), 49–59. https://doi.org/10.1016/j.cbpra.2014.01.010
   PubMed Web of Science ®Google Scholar
• Semansky, R., Willging, C., Ley, D. J., & Rylko-Bauer, B. (2012). Lost in the rush to national reform: Recommendations to improve impact on behavioral health providers in rural areas. Journal of Health Care for the Poor and Underserved, 23(2), 842. https://doi.org/10.1353/hpu.2012.0059
   PubMed Web of Science ®Google Scholar
• Slade, K., Lambert, M. J., Harmon, S. C., Smart, D. W., & Bailey, R. (2008). Improving psychotherapy outcome: The use of immediate electronic feedback and revised clinical support tools. Clinical Psychology & Psychotherapy, 15(5), 287–303. https://doi.org/10.1002/cpp.594
   PubMed Web of Science ®Google Scholar
• The Joint Commission. (2018). Revised outcome measures standard for behavioral health care. R3 report: Requirement, rationale, reference. The Joint Commission.https://www.jointcommission.org/assets/1/18/R3_Outcome_measures_1_30_18_FINAL.pdf
   Google Scholar
• Trudeau, L.S. (2000). Effects of a clinical feedback system on client and therapist outcomes in a rural community mental health center. Retrospective Theses and Dissertations. 12367. https://lib.dr.iastate.edu/rtd/12367
   Google Scholar
• Unützer, J., Chan, Y.-F., Hafer, E., Knaster, J., Shields, A., Powers, D., & Veith, R. C. (2012). Quality improvement with pay-for-performance incentives in integrated behavioral health care. American Journal of Public Health, 102(6), e41–e45. https://doi.org/10.2105/AJPH.2011.300555
   PubMedGoogle Scholar
• Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K., Ugueto, A. M., Langer, D. A., & Hoagwood, K. E. (2011). Youth top problems: Using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79(3), 369–380. https://doi.org/10.1037/a0023307
   PubMed Web of Science ®Google Scholar
• Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003). Improving the effects of psychotherapy: The use of early identification of treatment and problem-solving strategies in routine practice. Journal of Counseling Psychology, 50(1), 59–68. https://doi.org/10.1037/0022-0167.50.1.59
   Web of Science ®Google Scholar