Practice Guidelines for the Assessment of Clinically Significant Treatment Outcomes in the Children's Mental Health System

References

• Ash, S. E., & Weis, R. (2009). Recovery among youths referred to outpatient psychotherapy: Reliable change, clinical significance, and predictors of outcome. Child and Adolescent Social Work Journal, 26, 399–413. doi:10.1007/s10560-009-0171-3.
   Google Scholar
• Atkins, D. C., Bedics, J. D., McGlinchey, J. B., & Beauchaine, T. P. (2005). Assessing clinical significance: Does it matter which method we use? Journal of Consulting and Clinical Psychology, 73, 982–989. doi:10.1037/0022-006X.73.5.982.
   PubMed Web of Science ®Google Scholar
• Bauer, S., Lambert, M. J., & Nielsen, S. L. (2004). Clinical significance methods: A comparison of statistical techniques. Journal of Personality Assessment, 82, 60–70. doi:10.1207/s15327752jpa8201_11.
   PubMed Web of Science ®Google Scholar
• Becker, B. J. (1988). Synthesizing standardized mean-change measures. British Journal of Mathematical and Statistical Psychology, 41, 257–278. doi:10.1111/j.2044-8317.1988.tb00901.x.
   Web of Science ®Google Scholar
• Beutler, L. E., & Moleiro, C. (2001). Clinical versus reliable and significant change. Clinical Psychology: Science and Practice, 8, 441–445. doi:10.1093/clipsy.8.4.441.
   Web of Science ®Google Scholar
• Bost, R. H., Wen, F. K., Basso, M. R., & Cates, G. R. (2008). Online tools for evaluating patient change: Statistical foundations, clinical applications, research relevance. Rehabilitation Psychology, 53, 313–320. doi:10.1037/a0012977.
   Web of Science ®Google Scholar
• Campbell, T. C. (2005). An introduction to clinical significance: An alternative index of intervention effect for group experimental designs. Journal of Early Intervention, 27, 210–227. doi:10.1177/105381510502700307.
   Web of Science ®Google Scholar
• Chelune, G. J., Naugle, R. I., Lüders, H., Sedlak, J., & Awad, I. A. (1993). Individual change after epilepsy surgery: Practice effects and base-rate information. Neuropsychology, 7, 41–52. doi:10.1037/0894-4105.7.1.41.
   Google Scholar
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. doi:10.1037/0033-2909.112.1.155.
   PubMed Web of Science ®Google Scholar
• Cribbie, R. A., & Arpin-Cribbie, C. A. (2009). Evaluating clinical significance through equivalence testing: Extending the normative comparisons approach. Psychotherapy Research, 19, 677–686. doi:10.1080/10503300902926554.
   PubMed Web of Science ®Google Scholar
• De Los Reyes, A., & Kazdin, A. E. (2006). Conceptualizing changes in behavior in intervention research: The range of possible changes model. Psychological Review, 113, 554–583. doi:10.1037/0033-295X.113.3.554.
   PubMed Web of Science ®Google Scholar
• Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., …Jacobson, N. S. (2006). Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology, 74, 658–670. doi:10.1037/0022-006X.74.4.658.
   PubMed Web of Science ®Google Scholar
• Eisen, S. V., Ranganathan, G., Seal, P., & Spiro, III, A. (2007). Measuring clinically meaningful change following mental health treatment. The Journal of Behavioral Health Services & Research, 34, 272–289. doi:10.1007/s11414-007-9066-2.
   PubMed Web of Science ®Google Scholar
• Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40, 532–538. doi:10.1037/a0015808.
   Web of Science ®Google Scholar
• Gladis, M. M., Gosch, E. A., Dishuk, N. M., & Crits-Christoph, P. (1999). Quality of life: Expanding the scope of clinical significance. Journal of Consulting and Clinical Psychology, 67, 320–331. doi:10.1037/0022-006X.67.3.320.
   PubMed Web of Science ®Google Scholar
• Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586. doi:10.1111/j.1469-7610.1997.tb01545.x.
   PubMed Web of Science ®Google Scholar
• Goodman, R. (1999). The extended version of the Strengths and Difficulties Questionnaire as a guide to child psychiatric caseness and consequent burden. Journal of Child Psychology and Psychiatry, 40, 791–799. doi:10.1111/1469-7610.00494.
   PubMed Web of Science ®Google Scholar
• Hageman, W. J., & Arrindell, W. A. (1999). Establishing clinically significant change: Increment of precision and the distinction between individual and group level of analysis. Behaviour Research and Therapy, 37, 1169–1193. doi:10.1016/S0005-7967(99)00032-7.
   PubMed Web of Science ®Google Scholar
• Hays, R. D., Brodsky, M., Johnston, M. F., Spritzer, K. L., & Hui, K. K. (2005). Evaluating the statistical significance of health-related quality-of-life change in individual patients. Evaluation and the Health Professions, 28, 160–171. doi:10.1177/0163278705275339.
   PubMed Web of Science ®Google Scholar
• Hermann, B. P., Wyler, A. R., Vanderzwagg, R., LeBailly, R. K., Whitman, S., Somes, G., & Ward, J. (1991). Predictors of neuropsychological change following anterior temporal lobectomy: Role of regression toward the mean. Journal of Epilepsy, 4, 139–148. doi:10.1016/S0896-6974(05)80039-8.
   Google Scholar
• Hiller, W., Schindler, A. C., & Lambert, M. J. (2012). Defining response and remission in psychotherapy research: A comparison of the RCI and the method of percent improvement. Psychotherapy Research, 22, 1–11. doi:10.1080/10503307.2011.616237.
   PubMed Web of Science ®Google Scholar
• Hsu, L. M. (1989). Reliable changes in psychotherapy: Taking into account regression toward the mean. Behavioral Assessment, 11, 459–467. Retrieved from http://www.researchgate.net/journal/0164-0305_Journal_of_Behavioral_Assessment.
   Google Scholar
• Hsu, L. M. (1999). A comparison of three methods of identifying reliable and clinically significant client changes: Commentary on Hageman and Arrindell. Behaviour Research and Therapy, 37, 1195–1202. doi:10.1016/S0005-7967(99)00033-9.
   PubMed Web of Science ®Google Scholar
• Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336–352. doi:10.1016/S0005-7894(84)80002-7.
   Web of Science ®Google Scholar
• Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999). Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. Journal of Consulting and Clinical Psychology, 67, 300–307. doi:10.1037/0022-006X.67.3.300.
   PubMed Web of Science ®Google Scholar
• Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. doi:10.1037/0022-006X.59.1.12.
   PubMed Web of Science ®Google Scholar
• Karpenko, V., Owens, J. S., Evangelista, N. M., & Dodds, C. (2009). Clinically significant symptom change in children with attention-deficit/hyperactivity disorder: Does it correspond with reliable improvement in functioning? Journal of Clinical Psychology, 65, 76–93. doi:10.1002/jclp.20549.
   PubMed Web of Science ®Google Scholar
• Kazdin, A. E. (1999). The meanings and measurement of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332–339. doi:10.1037/0022-006X.67.3.332.
   PubMed Web of Science ®Google Scholar
• Kazdin, A. E. (2001). Almost clinically significant (p < . 10): Current measures may only approach clinical significance. Clinical Psychology: Science and Practice, 8, 455–462. doi:10.1093/clipsy.8.4.455.
   Web of Science ®Google Scholar
• Kazdin, A. E., & Durbin, K. A. (2012). Predictors of child–therapist alliance in cognitive-behavioral treatment of children referred for oppositional and antisocial behavior. Psychotherapy, 49, 202–217. doi:10.1037/a0027933.
   PubMed Web of Science ®Google Scholar
• Keith, J. A., Verdeli, H., & Vousoura, E. (2011). Evaluating the clinical significance of depression treatment. International Journal of Mental Health, 40, 3–28. doi:10.2753/IMH0020-7411400301.
   Google Scholar
• Kelly, P. J. (2010). Calculating clinically significant change: Applications of the Clinical Global Impressions (CGI) Scale to evaluate client outcomes in private practice. Clinical Psychologist, 14, 107–111. doi:10.1080/13284207.2010.512015.
   Web of Science ®Google Scholar
• Kendall, P. C., & Grove, W. M. (1988). Normative comparisons in therapy outcome. Behavioral Assessment, 10, 147–158. Retrieved from http://www.researchgate.net/journal/0164-0305_Journal_of_Behavioral_Assessment.
   Google Scholar
• Kendall, P. C., Marrs-Garcia, A., Nath, S. R., & Sheldrick, R. C. (1999). Normative comparisons for the evaluation of clinical significance. Journal of Consulting and Clinical Psychology, 67, 285–299. doi:10.1037/0022-006X.67.3.285.
   PubMed Web of Science ®Google Scholar
• Kendall, P. C., & Sheldrick, R. C. (2000). Normative data for normative comparisons. Journal of Consulting and Clinical Psychology, 68, 767–773. doi:10.1037/0022-006X.68.5.767.
   PubMed Web of Science ®Google Scholar
• Lambert, M. J. (2013). Outcome in psychotherapy: The past and important advances. Psychotherapy, 50, 42–51. doi:10.1037/a0030682.
   PubMed Web of Science ®Google Scholar
• Lambert, M. J., Hansen, N. B., Umphress, V., Lunnen, K., Okiishi, J., Burlingame, G., …Reisinger, C. W. (1996). Administration and Scoring Manual for the Outcome Questionnaire (OQ 45.2). Wilmington, DE: American Professional Credentialing Services.
   Google Scholar
• Lindhiem, O., Kolko, D. J., & Cheng, Y. (2012). Predicting psychotherapy benefit: A probabilistic and individualized approach. Behavior Therapy, 43, 381–392. doi:10.1016/j.beth.2011.08.004.
   PubMed Web of Science ®Google Scholar
• Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181–1209. doi:10.1037/0003-066X.48.12.1181.
   PubMed Web of Science ®Google Scholar
• Maassen, G. H. (2000). Principles of defining reliable change indices. Journal of Clinical and Experimental Neuropsychology, 22, 622–632. doi:10.1076/1380-3395(200010)22:5;1-9;FT622.
   PubMed Web of Science ®Google Scholar
• Maassen, G. H. (2004). The standard error in the Jacobson and Truax Reliable Change Index: The classical approach to the assessment of reliable change. Journal of the International Neuropsychological Society, 10, 888–893. doi:10.1017/S1355617704106097.
   PubMed Web of Science ®Google Scholar
• Marsden, J., Eastwood, B., Wright, C., Bradbury, C., Knight, J., & Hammond, P. (2011). How best to measure change in evaluations of treatment for substance use disorder. Addiction, 106, 294–302. doi:10.1111/j.1360-0443.2010.03143.x.
   PubMed Web of Science ®Google Scholar
• McGlinchey, J. B., Atkins, D. C., & Jacobson, N. S. (2002). Clinical significance methods: Which one to use and how useful are they? Behavior Therapy, 33, 529–550. doi:10.1016/S0005-7894(02)80015-6.
   Web of Science ®Google Scholar
• Morris, S. B. (2000). Distribution of the standardized mean change effect size for meta-analysis on repeated measures. British Journal of Mathematical and Statistical Psychology, 53, 17–29. doi:10.1348/000711000159150.
   PubMed Web of Science ®Google Scholar
• Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11, 364–386. doi:10.1177/1094428106291059.
   Web of Science ®Google Scholar
• Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105–125. doi:10.1037/1082-989X.7.1.105.
   PubMed Web of Science ®Google Scholar
• National Center for Health Statistics, Centers for Disease Control and Prevention. (2001). Normative SDQ data from the USA. Retrieved from http://www.sdqinfo.org/USNorm.html.
   Google Scholar
• Nelson, T. D., & Steele, R. G. (2006). Beyond efficacy and effectiveness: A multifaceted approach to treatment evaluation. Professional Psychology: Research and Practice, 37, 389–397. doi:10.1037/0735-7028.37.4.389.
   Web of Science ®Google Scholar
• Ogles, B. M., Lunnen, K. M., & Bonesteel, K. (2001). Clinical significance: History, application, and current practice. Clinical Psychology Review, 21, 421–446. doi:10.1016/S0272-7358(99)00058-6.
   PubMed Web of Science ®Google Scholar
• Sanders, M. R. (1999). Triple P-positive parenting program: Towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clinical Child and Family Psychology Review, 2, 71–90. doi:10.1023/A:1021843613840.
   PubMedGoogle Scholar
• Sanders, M. R. (2008). Triple P-positive parenting program as a public health approach to strengthening parenting. Journal of Family Psychology, 22, 506–517. doi:10.1037/0893-3200.22.3.506.
   PubMed Web of Science ®Google Scholar
• Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752–760. doi:10.1037/0003-066X.32.9.752.
   PubMed Web of Science ®Google Scholar
• Speer, D. C. (1992). Clinically significant change: Jacobson and Truax (1991) revisited. Journal of Consulting and Clinical Psychology, 60, 402–408. doi:10.1037/0022-006X.60.3.402.
   PubMed Web of Science ®Google Scholar
• Speer, D. C., & Greenbaum, P. E. (1995). Five methods for computing significant individual client change and improvement rates: Support for an individual growth curve approach. Journal of Consulting and Clinical Psychology, 63, 1044–1048. doi:10.1037/0022-006X.63.6.1044.
   PubMed Web of Science ®Google Scholar
• Strauman, T. J., Vieth, A. Z., Merrill, K. A., Kolden, G. G., Woods, T. E., Klein, M. H., …Kwapil, L. (2006). Self-system therapy as an intervention for self-regulatory dysfunction in depression: A randomized comparison with cognitive therapy. Journal of Consulting and Clinical Psychology, 74, 367–376. doi:10.1037/0022-006X.74.2.367.
   PubMed Web of Science ®Google Scholar
• Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31, 25–32. doi:10.3102/0013189X031003025.
   Google Scholar
• Webster-Stratton, C., Hollinsworth, T., & Kolpacoff, M. (1989). The long-term effectiveness and clinical significance of three cost-effective training programs for families with conduct-problem children. Journal of Consulting and Clinical Psychology, 57, 550–553. doi:10.1037/0022-006X.57.4.550.
   PubMed Web of Science ®Google Scholar
• Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61, 671–689. doi:10.1037/0003-066X.61.7.671.
   PubMed Web of Science ®Google Scholar
• Weisz, J. R., Sandler, I. N., Durlak, J. A., & Anton, B. S. (2005). Promoting and protecting youth mental health through evidence-based prevention and treatment. American Psychologist, 60, 628–648. doi:10.1037/0003-066X.60.6.628.
   PubMed Web of Science ®Google Scholar
• Weisz, J. R., Weiss, B., Alicke, M. D., & Klotz, M. L. (1987). Effectiveness of psychotherapy with children and adolescents: A meta-analysis for clinicians. Journal of Consulting and Clinical Psychology, 55, 542–549. doi:10.1037/0022-006X.55.4.542.
   PubMed Web of Science ®Google Scholar
• Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117, 450–468. doi:10.1037/0033-2909.117.3.450.
   PubMed Web of Science ®Google Scholar