Effectiveness of routine psychotherapy: Method matters

References

• Acarturk, C., Cuijpers, P., Van Straten, A., & De Graaf, R. (2009). Psychological treatment of social anxiety disorder: A meta-analysis. Psychological Medicine, 39, 241–254. doi: 10.1017/S0033291708003590
   PubMed Web of Science ®Google Scholar
• Baldwin, S. A., Berkeljon, A., Atkins, D. C., Olsen, J. A., & Nielsen, S. L. (2009). Rates of change in naturalistic psychotherapy: Contrasting dose–effect and good-enough level models of change. Journal of Consulting and Clinical Psychology, 77, 203–211. doi: 10.1037/a0015235
   PubMed Web of Science ®Google Scholar
• Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects: Findings and methods. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 258–297). Hoboken, NJ: Wiley.
   Google Scholar
• Barkham, M., Connell, J., Stiles, W. B., Miles, J. N. V., Margison, F., Evans, C., & Mellor-Clark, J. (2006). Dose–effect relations and responsive regulation of treatment duration: The good enough level. Journal of Consulting and Clinical Psychology, 74, 160–167. doi: 10.1037/0022-006X.74.1.160
   PubMed Web of Science ®Google Scholar
• Barkham, M., Stiles, W. B., Connell, J., & Mellor-Clark, J. (2012). Psychological treatment outcomes in routine NHS services: What do we mean by treatment effectiveness? Psychology and Psychotherapy: Theory, Research and Practice, 85, 1–16. doi: 10.1111/j.2044-8341.2011.02019.x
   PubMed Web of Science ®Google Scholar
• Barkham, M., Stiles, W. B., Connell, J., Twigg, E., Leach, C., Lucock, M., … Angus, L. (2008). Effects of psychological therapies in randomized trials and practice-based studies. British Journal of Clinical Psychology, 47, 397–415. doi: 10.1348/014466508X311713
   PubMed Web of Science ®Google Scholar
• Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571. doi: 10.1001/archpsyc.1961.01710120031004
   PubMed Web of Science ®Google Scholar
• Center for Collegiate Mental Health. (2012). CCAPS 2012 technical manual. University Park, PA.
   Google Scholar
• Cribbie, R. A., & Arpin-Cribbie, C. A. (2009). Evaluating clinical significance through equivalence testing: Extending the normative comparisons approach. Psychotherapy Research, 19, 677–686. doi: 10.1080/10503300902926554
   PubMed Web of Science ®Google Scholar
• Cuijpers, P., Sijbrandij, M., Koole, S. L., Andersson, G., Beekman, A. T., & Reynolds, C. F. (2013). The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: A meta-analysis of direct comparisons. World Psychiatry, 12(2), 137–148. doi: 10.1002/wps.20038
   PubMed Web of Science ®Google Scholar
• Cuijpers, P., Smit, F., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). Efficacy of cognitive–behavioural therapy and other psychological treatments for adult depression: Meta-analytic study of publication bias. The British Journal of Psychiatry, 196, 173–178. doi: 10.1192/bjp.bp.109.066001
   PubMed Web of Science ®Google Scholar
• Cuijpers, P., Turner, E. H., Mohr, D. C., Hofmann, S. G., Andersson, G., Berking, M., & Coyne, J. (2014). Comparison of psychotherapies for adult depression to pill placebo control groups: A meta-analysis. Psychological Medicine, 44, 685–695. doi: 10.1017/S0033291713000457
   PubMed Web of Science ®Google Scholar
• Del Vecchio, T., & O’Leary, K. D. (2004). Effectiveness of anger treatments for specific anger problems: A meta-analytic review. Clinical Psychology Review, 24, 15–34. doi: 10.1016/j.cpr.2003.09.006
   PubMed Web of Science ®Google Scholar
• DiGiuseppe, R., & Tafrate, R. C. (2003). Anger treatment for adults: A meta-analytic review. Clinical Psychology: Science and Practice, 10, 70–84. doi:10.1093/clipsy.10.1.70
   Web of Science ®Google Scholar
• Dutra, L., Stathopoulou, G., Basden, S. L., Leyro, T. M., Powers, M. B., & Otto, M. W. (2008). A meta-analytic review of psychosocial interventions for substance use disorders. American Journal of Psychiatry, 165, 179–187. doi: 10.1176/appi.ajp.2007.06111851
   PubMed Web of Science ®Google Scholar
• Forand, N. R., Evans, S., Haglin, D., & Fishman, B. (2011). Cognitive-behavioral therapy in practice: Treatment delivered by trainees at an outpatient clinic is clinically effective. Behavior Therapy, 42, 612–623. doi: 10.1016/j.beth.2011.02.001
   PubMed Web of Science ®Google Scholar
• Gould, R. A., Otto, M. W., Pollack, M. H., & Yap, L. (1997). Cognitive behavioral and pharmacological treatment of generalized anxiety disorder: A preliminary meta-analysis. Behavior Therapy, 28, 285–305. doi: 10.1016/S0005-7894(97)80048-2
   Web of Science ®Google Scholar
• Hunsley, J., & Lee, C. M. (2007). Research-informed benchmarks for psychological treatments: Efficacy studies, effectiveness studies, and beyond. Professional Psychology: Research and Practice, 38, 21–33. doi: 10.1037/0735-7028.38.1.21
   Web of Science ®Google Scholar
• Imel, Z. E., Wampold, B. E., Miller, S. D., & Fleming, R. R. (2008). Distinctions without a difference: Direct comparisons of psychotherapies for alcohol use disorders. Psychology of Addictive Behaviors, 22, 533–543. doi: 10.1037/a0013171
   PubMed Web of Science ®Google Scholar
• Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. doi: 10.1037/0022-006X.59.1.12
   PubMed Web of Science ®Google Scholar
• Kendall, P. C., & Grove, W. M. (1988). Normative comparisons in therapy outcome. Behavioral Assessment, 10, 147–158.
   Google Scholar
• Kendall, P. C., Marrs-Garcia, A., Nath, S. R., & Sheldrick, R. C. (1999). Normative comparison for the evaluation of clinical significance. Journal of Consulting and Clinical Psychology, 67, 285–299. doi: 10.1037/0022-006X.67.3.285
   PubMed Web of Science ®Google Scholar
• Kraus, D. R., Bentley, J. H., Alexander, P. C., Boswell, J. F., Constantino, M. J., Baxter, E. E., & Castonguay, L. G. (2016). Predicting therapist effectiveness from their own practice-based evidence. Journal of Consulting and Clinical Psychology, 84(6), 473–483. doi: 10.1037/ccp0000083
   PubMed Web of Science ®Google Scholar
• Lambert, M. J., Hatfield, D. R., Vermeersch, D. A., Burlingame, G. M., Reisinger, C. W., & Brown, G. S. (2003). Administration and scoring manual for the OQ-30.1. East Setauket, NY: American Professional Credentialing Services.
   Google Scholar
• Lilienfeld, S. O. (2007). Psychological treatments that cause harm. Perspectives on Psychological Science, 2, 53–70. doi: 10.1111/j.1745-6916.2007.00029.x
   Web of Science ®Google Scholar
• Locke, B. D., Buzolitz, J. S., Lei, P., Boswell, J. F., McAleavey, A. A., Sevig, T. D., … Hayes, J. A. (2011). Development of the counseling center assessment of psychological symptoms-62 (CCAPS-62). Journal of Counseling Psychology, 58(1), 97–109. doi: 10.1037/a0021282
   PubMed Web of Science ®Google Scholar
• Locke, B. D., McAleavey, A. A., Zhao, Y., Lei, P., Hayes, J. A., Castonguay, L. G., … Lin, Y. (2012). Development and initial validation of the counseling center assessment of psychological symptoms-34. Measurement and Evaluation in Counseling and Development, 45(3), 151–169. doi: 10.1177/0748175611432642
   Web of Science ®Google Scholar
• Lutz, W., Schiefele, A.-K., Wucherpfennig, F., Rubel, J., & Stulz, N. (2016). Clinical effectiveness of cognitive behavioral therapy for depression in routine care: A propensity score based comparison between randomized controlled trials and clinical practice. Journal of Affective Disorders, 189, 150–158. doi: 10.1016/j.jad.2015.08.072
   PubMed Web of Science ®Google Scholar
• Mangardich, H., & Cribbie, R. A. (2014). Assessing clinical significance using robust normative comparisons. Psychotherapy Research, 25, 239–248. Advance online publication. doi: 10.1080/10503307.2014.889329
   PubMed Web of Science ®Google Scholar
• McAleavey, A. A., Lockard, A. J., Castonguay, L. G., Hayes, J. A., & Locke, B. D. (2015). Building a practice research network: Obstacles faced and lessons learned at the center for collegiate mental health. Psychotherapy Research, 25(1), 134–151. doi: 10.1080/10503307.2014.883652
   PubMed Web of Science ®Google Scholar
• McAleavey, A. A., Nordberg, S. S., Kraus, D., & Castonguay, L. G. (2012). Errors in treatment outcome monitoring: Implications for real-world psychotherapy. Canadian Psychology, 53, 105–114. doi: 10.1037/a0027833
   Web of Science ®Google Scholar
• McEvoy, P. M., & Nathan, P. (2007). Effectiveness of cognitive behavior therapy for diagnostically heterogeneous groups: A benchmarking study. Journal of Consulting and Clinical Psychology, 75, 344–350. doi: 10.1037/0022-006X.75.2.344
   PubMed Web of Science ®Google Scholar
• Merrill, K. A., Tolbert, V. E., & Wade, W. A. (2003). Effectiveness of cognitive therapy for depression in a community mental health center: A benchmarking study. Journal of Consulting and Clinical Psychology, 71(2), 404–409. doi: 10.1037/0022-006X.71.2.404
   PubMed Web of Science ®Google Scholar
• Minami, T., Davies, D. R., Tierney, S. C., Bettmann, J. E., McAward, S. M., Averill, L. A., & Wampold, B. E. (2009). Preliminary evidence on the effectiveness of psychological treatments delivered at a university counseling center. Journal of Counseling Psychology, 56, 309–320. doi: 10.1037/a0015398
   Web of Science ®Google Scholar
• Minami, T., Serlin, R. C., Wampold, B. E., Kircher, J. C., & Brown, G. S. (2008). Using clinical trials to benchmark effects produced in clinical practice. Quality & Quantity, 42, 513–525. doi: 10.1007/s11135-006-9057-z
   Web of Science ®Google Scholar
• Minami, T., Wampold, B. E., Serlin, R. C., Hamilton, E. G., Brown, G. S., & Kircher, J. C. (2008). Benchmarking the effectiveness of psychotherapy treatment for adult depression in a managed care environment: A preliminary study. Journal of Consulting and Clinical Psychology, 76, 116–124. doi: 10.1037/0022-006X.76.1.116
   PubMed Web of Science ®Google Scholar
• Minami, T., Wampold, B. E., Serlin, R. C., Kircher, J. C., & Brown, G. S. (2007). Benchmarks for psychotherapy efficacy in adult major depression. Journal of Consulting and Clinical Psychology, 75, 232–243. doi: 10.1037/0022-006X.75.2.232
   PubMed Web of Science ®Google Scholar
• Mitte, K. (2005). Meta-analysis of cognitive-behavioral treatments for generalized anxiety disorder: A comparison with pharmacotherapy. Psychological Bulletin, 131, 785–795. doi: 10.1037/0033-2909.131.5.785
   PubMed Web of Science ®Google Scholar
• Nasiakos, G., Cribbie, R. A., & Arpin-Cribbie, C. A. (2010). Equivalence based tests of clinical significance: Assessing treatments for depression. Psychotherapy Research, 20, 647–656. doi: 10.1080/10503307.2010.501039
   PubMed Web of Science ®Google Scholar
• Nordberg, S. S., McAleavey, A. A., Duszak, E., Locke, B. D., Hayes, J. A., & Castonguay, L. G. (2016). The counseling center assessment of psychological symptoms distress index: A pragmatic exploration of general factors to enhance a multidimensional scale. Counselling Psychology Quarterly, 1–16. doi: 10.1080/09515070.2016.1202809
   Web of Science ®Google Scholar
• R Core Team. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.R-project.org/
   Google Scholar
• Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting and Clinical Psychology, 82, 731–742. doi: 10.1037/a0036915
   PubMed Web of Science ®Google Scholar
• Ronk, F. R., Korman, J. R., Hooke, G. R., & Page, A. C. (2013). Assessing clinical significance of treatment outcomes using the DASS-21. Psychological Assessment, 25, 1103–1110. doi: 10.1037/a0033100
   PubMed Web of Science ®Google Scholar
• Saunders, J. B., Aasland, O. G., Babor, T. F., de la Fuente, J. R., & Grant, M. (1993). Development of the alcohol Use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption–II. Addiction, 88, 791–804. doi: 10.1111/j.1360-0443.1993.tb02093.x
   PubMed Web of Science ®Google Scholar
• Sheldrick, R. C., Kendall, P. C., & Heimberg, R. G. (2001). The clinical significance of treatments: A comparison of three treatments for conduct disordered children. Clinical Psychology: Science & Practice, 8, 418–430.
   Web of Science ®Google Scholar
• Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). New York, NY: Pearson.
   Google Scholar
• Thompson-Brenner, H., Glass, S., & Westen, D. (2003). A multidimensional meta-analysis of psychotherapy for bulimia nervosa. Clinical Psychology: Science and Practice, 10, 269–287.
   Web of Science ®Google Scholar
• Vocks, S., Tuschen-Caffier, B., Pietrowsky, R., Rustenbach, S. J., Kersting, A., & Herpertz, S. (2010). Meta-analysis of the effectiveness of psychological and pharmacological treatments for binge eating disorder. International Journal of Eating Disorders, 43, 205–217. doi:10.1002/eat.20696
   PubMed Web of Science ®Google Scholar
• Wade, W. A., Treat, T. A., & Stuart, G. L. (1998). Transporting an empirically supported treatment for panic disorder to a service clinic setting: A benchmarking strategy. Journal of Consulting and Clinical Psychology, 66(2), 231–239. doi: 10.1037/0022-006X.66.2.231
   PubMed Web of Science ®Google Scholar
• Weersing, V. R., & Weisz, J. R. (2002). Community clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology, 70(2), 299–310. doi: 10.1037/0022-006X.70.2.299
   PubMed Web of Science ®Google Scholar
• van Wieringen, K., & Cribbie, R. A. (2014). Evaluating clinical significance: Incorporating robust statistics with normative comparison tests. British Journal of Mathematical and Statistical Psychology, 67, 213–230. doi: 10.1111/bmsp.12015
   PubMed Web of Science ®Google Scholar