Intraclass Correlation Associated with Therapists: Estimates and Applications in Planning Psychotherapy Research

References

• *Abramowitz , J. S. , Foa , E. B. and Franklin , M. E. 2003 . Exposure and ritual prevention for obsessive-compulsive disorder: Effects of intensive versus twice-weekly sessions . Journal of Consulting and Clinical Psychology , 71 : 394 – 398 .
   PubMed Web of Science ®Google Scholar
• Baldwin , S. A. , Murray , D. M. and Shadish , W. R. 2005 . Empirically supported treatments or type I errors? Problems with the analysis of data from group-administered treatments . Journal of Consulting and Clinical Psychology , 73 : 924 – 935 .
   PubMed Web of Science ®Google Scholar
• *Baldwin , S. A. , Wampold , B. E. and Imel , Z. E. 2007 . Untangling the alliance–outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance . Journal of Consulting and Clinical Psychology , 75 : 842 – 852 .
   PubMed Web of Science ®Google Scholar
• Beck , A. T. , Steer , R. A. and Brown , G. K. 1996 . Manual for Beck Depression Inventory–II , San Antonio, TX : Psychological Corporation .
   Google Scholar
• Beck , A. T. , Ward , C. H. , Mendelson , M. , Mock , J. and Erbaugh , J. 1961 . An inventory for measuring depression . Archives of General Psychiatry , 4 : 561 – 571 .
   PubMed Web of Science ®Google Scholar
• Bergin , A. E. 1966 . Some implications of psychotherapy research for therapeutic practice . Journal of Abnormal Psychology , 71 : 235 – 246 .
   PubMed Web of Science ®Google Scholar
• Blitstein , J. L. , Hannan , P. J. , Murray , D. M. and Shadish , W. R. 2005 . Increasing the degrees of freedom in existing group randomized trials: The df* approach . Evaluation Review , 29 : 241 – 267 .
   PubMed Web of Science ®Google Scholar
• *Carlbring , P. , Bohman , S. , Brunt , S. , Buhrman , M. , Westling , B. E. , Ekselius , L. and Andersson , G. 2006 . Remote treatment of panic disorder: A randomized trial of Internet-based cognitive behavior therapy supplemented with telephone calls . American Journal of Psychiatry , 163 : 2119 – 2125 .
   PubMed Web of Science ®Google Scholar
• *Carroll , K. M. , Fenton , L. R. , Ball , S. A. , Nich , C. , Frankforter , T. L. , Shi , J. and Rounsaville , B. J. 2004 . Efficacy of disulfiram and cognitive behavior therapy in cocaine-dependent outpatients: A randomized placebo-controlled trial . Archives of General Psychiatry , 61 : 264 – 272 .
   PubMed Web of Science ®Google Scholar
• *Christensen , A. , Atkins , D. C. , Berns , S. , Wheeler , J. , Baucom , D. H. and Simpson , L. E. 2004 . Traditional versus integrative behavioral couple therapy for significantly and chronically distressed married couples . Journal of Consulting and Clinical Psychology , 72 : 176 – 191 .
   PubMed Web of Science ®Google Scholar
• Cornfield , J. 1978 . Randomization by group: A formal analysis . American Journal of Epidemiology , 108 : 100 – 102 .
   PubMed Web of Science ®Google Scholar
• Crits-Christoph , P. , Baranackie , K. , Kurcias , J. S. , Beck , A. T. , Carroll , K. , Perry , K. and Zitrin , C. 1991 . Meta-analysis of therapist effects in psychotherapy outcome studies . Psychotherapy Research , 1 : 81 – 91 .
   Google Scholar
• Crits-Christoph , P. and Mintz , J. 1991 . Implications of therapist effects for the design and analysis of comparative studies of psychotherapies . Journal of Consulting and Clinical Psychology , 59 : 20 – 26 .
   PubMed Web of Science ®Google Scholar
• Crits-Christoph , P. , Tu , X. and Gallop , R. 2003 . Therapists as fixed versus random effects—Some statistical and conceptual issues: A comment on Siemer and Joormann (2003) . Psychological Methods , 8 : 518 – 523 .
   PubMed Web of Science ®Google Scholar
• *Dinger , U. , Strack , M. , Leichsenring , F. , Wilmers , F. and Schauenburg , H. 2008 . Therapist effects on outcome and alliance in inpatient psychotherapy . Journal of Clinical Psychology , 64 : 344 – 354 .
   PubMed Web of Science ®Google Scholar
• Donner , A. 1986 . A review of inference procedures for the intraclass correlation coefficient in the one-way random effects model . International Statistics Review , 54 : 67 – 82 .
   Web of Science ®Google Scholar
• Donner , A. and Klar , N. 2000 . Design and analysis of cluster randomization trials in health research , London : Arnold .
   Google Scholar
• *Ehlers , A. , Clark , D. M. , Hackmann , A. , McManus , F. , Fennell , M. , Herbert , C. and Mayou , R. 2003 . A randomized controlled trial of cognitive therapy, a self-help booklet, and repeated assessments as early interventions for posttraumatic stress disorder . Archives of General Psychiatry , 60 : 1024 – 1032 .
   PubMed Web of Science ®Google Scholar
• Elkin , I. , Falconnier , L. , Martinovich , Z. and Mahoney , C. 2006 . Therapist effects in the National Institute of Mental Health Treatment of Depression Collaborative Research Program . Psychotherapy Research , 16 : 144 – 160 .
   Web of Science ®Google Scholar
• Fleiss , J. L. , Levin , B. and Paik , M. C. 2003 . Statistical methods for rates and proportions , 3rd ed , Hoboken, NJ : Wiley .
   Google Scholar
• Gail , M. H. , Mark , S. D. , Carroll , R. J. , Green , S. B. and Pee , D. 1996 . On design considerations and randomization-based inference for community intervention trials . Statistics in Medicine , 15 : 1069 – 1092 .
   PubMed Web of Science ®Google Scholar
• Gulliford , M. C. , Ukomunne , O. C. and Chinn , S. 1999 . Components of variance and intraclass correlation for the design of community-based surveys and intervention studies . American Journal of Epidemiology , 149 : 876 – 883 .
   PubMed Web of Science ®Google Scholar
• Hedges , L. V. and Hedberg , E. C. 2007 . Intraclass correlation values for planning group-randomized trials in education . Educational Evaluation and Policy Analysis , 29 : 60 – 87 .
   Web of Science ®Google Scholar
• Higgins , J. P. T. and Thompson , S. G. 2002 . Quantifying heterogeneity in a meta-analysis . Statistics in Medicine , 21 : 1539 – 1558 .
   PubMed Web of Science ®Google Scholar
• Kenny , D. A. , Kashy , D. A. and Bolger , N. 1998 . “ Data analysis in social psychology ” . In The handbook of social psychology , Edited by: Gilbert , D. T. , Fiske , S. T. and Lindzey , G. Vol. 1 , 233 – 265 . New York, NY : Oxford University Press .
   Google Scholar
• Kenny , D. A. , Mannetti , L. , Peirro , A. , Livi , S. and Kashy , D. A. 2002 . The statistical analysis of data from small groups . Journal of Personality and Social Psychology , 83 : 126 – 137 .
   PubMed Web of Science ®Google Scholar
• *Kim , D. -M. , Wampold , B. E. and Bolt , D. M. 2006 . Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data . Psychotherapy Research , 16 : 161 – 172 .
   Web of Science ®Google Scholar
• Kish , L. 1965 . Survey sampling , New York, NY : Wiley .
   Google Scholar
• *Koch , E. I. , Spates , C. R. and Himle , J. A. 2004 . Comparison of behavioral and cognitive-behavioral one-session exposure treatments for small animal phobias . Behaviour Research and Therapy , 42 : 1483 – 1504 .
   PubMed Web of Science ®Google Scholar
• *Kuyken , W. 2004 . Cognitive therapy outcome: The effects of hopelessness in a naturalistic outcome study . Behaviour Research and Therapy , 42 : 631 – 646 .
   PubMed Web of Science ®Google Scholar
• *Lange , A. , Rietdijk , D. , Hudcovicova , M. , van de Ven , J. P. , Schrieken , B. and Emmelkamp , P. M. 2003 . Interapy: A controlled randomized trial of the standardized treatment of posttraumatic stress through the Internet . Journal of Consulting and Clinical Psychology , 71 : 901 – 909 .
   PubMed Web of Science ®Google Scholar
• *Lincoln , T. M. , Rief , W. , Hahlweg , K. , Frank , M. , von Witzleben , I. , Schroeder , B. and Fiegenbaum , W. 2003 . Effectiveness of an empirically supported treatment for social phobia in the field . Behaviour Research and Therapy , 41 : 1251 – 1269 .
   PubMed Web of Science ®Google Scholar
• Lutz , W. , Leon , S. C. , Martinovich , Z. , Lyons , J. S. and Stiles , W. B. 2007 . Therapist effects in outpatient psychotherapy: A three-level growth curve approach . Journal of Counseling Psychology , 54 : 32 – 39 .
   Web of Science ®Google Scholar
• *Marijuana Treatment Project Research Group . 2004 . Brief treatments for cannabis dependence: Findings from a randomized multisite trial . Journal of Consulting and Clinical Psychology , 72 : 455 – 466 .
   PubMed Web of Science ®Google Scholar
• Martindale , C. 1978 . The therapist-as-fixed-effect fallacy in psychotherapy research . Journal of Consulting and Clinical Psychology , 46 : 1526 – 1530 .
   PubMed Web of Science ®Google Scholar
• Maxwell , S. E. and Delaney , H. D. 2004 . Designing experiments and analyzing data: A model comparison approach , 2nd ed. , Mahwah, NJ : Erlbaum .
   Google Scholar
• *Merrill , K. A. , Tolbert , V. E. and Wade , W. A. 2003 . Effectiveness of cognitive therapy for depression in a community mental health center: A benchmarking study . Journal of Consulting and Clinical Psychology , 71 : 404 – 409 .
   PubMed Web of Science ®Google Scholar
• Murray , D. M. 1998 . Design and analysis of group-randomized trials , New York, NY : Oxford University Press .
   Google Scholar
• Murray , D. M. and Blitstein , J. L. 2003 . Methods to reduce the impact of intraclass correlation in group-randomized trials . Evaluation Review , 27 : 79 – 103 .
   PubMed Web of Science ®Google Scholar
• Murray , D. M. , Hannan , P. J. and Baker , W. L. 1996 . A Monte Carlo study of alternative responses to intraclass correlation in community trials. Is it ever possible to avoid Cornfield's penalties? . Evaluation Review , 20 : 313 – 337 .
   PubMed Web of Science ®Google Scholar
• Murray , D. M. , Pals , S. P. , Blitstein , J. L. , Alfano , C. M. and Lehman , J. 2008 . Design and analysis of group-randomized trials in cancer: A review of current practices . Journal of the National Cancer Institute , 100 : 483 – 491 .
   PubMedGoogle Scholar
• Murray , D. M. , Rooney , B. L. , Hannan , P. J. , Peterson , A. V. , Ary , D. V. , Biglan , A. and Schinke , S.P. 1994 . Intraclass correlation among common measures of adolescent smoking: Estimates, correlates, and applications in smoking prevention studies . American Journal of Epidemiology , 140 : 1038 – 1050 .
   PubMed Web of Science ®Google Scholar
• Murray , D. M. , Varnell , S. P. and Blitstein , J. L. 2004 . Design and analysis of group-randomized trials: A review of recent methodological developments . American Journal of Public Health , 94 : 423 – 432 .
   PubMed Web of Science ®Google Scholar
• Okiishi , J. C. , Lambert , M. J. , Nielsen , L. and Ogles , B. M. 2003 . Waiting for supershrink: An empirical analysis of therapist effects . Clinical Psychology and Psychotherapy , 10 : 361 – 373 .
   Web of Science ®Google Scholar
• Pals , S. P. , Murray , D. M. , Alfano , C. M. , Shadish , W. R. , Hannan , P. J. and Baker , W. L. 2008 . Individually randomized group treatment studies: Are the most frequently used analytic models misleading? . American Journal of Public Health , 98 : 1418 – 1424 .
   PubMed Web of Science ®Google Scholar
• Pinheiro , J. C. and Bates , D. M. 2000 . Mixed-effects models in S and S-Plus , New York, NY : Springer .
   Google Scholar
• Roberts , C. 1999 . The implications of variation in outcome between health professionals for the design and analysis of randomized controlled trials . Statistics in Medicine , 18 : 2605 – 2615 .
   PubMed Web of Science ®Google Scholar
• Roberts , C. and Roberts , S. A. 2005 . Design and analysis of clinical trials with clustering effects due to treatment . Clinical Trials , 2 : 152 – 162 .
   PubMed Web of Science ®Google Scholar
• Satterthwaite , F. W. 1946 . An approximate distribution of estimates of variance components . Biometrics Bulletin , 2 : 110 – 140 .
   Google Scholar
• Schnurr , P. P. , Friedman , M. J. , Lavori , P. W. and Hsieh , F. Y. 2001 . Design of Department of Veterans Affairs Cooperative Study No. 420: Group treatment of posttraumatic stress disorder . Controlled Clinical Trials , 22 : 74 – 88 .
   PubMedGoogle Scholar
• Snedecor , G. W. and Cochran , W. G. 1989 . Statistical methods , 8th ed. , Ames, IA : Iowa State University Press .
   Google Scholar
• StataCorp (2009). Stata Statistical Software (Version 11) [Computer software]. College Station, TX: Author
   Google Scholar
• Swallow , W. H. and Monahan , J. F. 1984 . Monte Carlo comparison of ANOVA, MIVQUE, REML, and ML estimators of variance components . Technometrics , 26 : 47 – 57 .
   Web of Science ®Google Scholar
• *Szapocznik , J. , Feaster , D. J. , Mitrani , V. B. , Prado , G. , Smith , L. , Robinson-Batista , C. and Robbins , M. S. 2004 . Structural ecosystems therapy for HIV-seropositive African American women: Effects on psychological distress, family hassles, and family support . Journal of Consulting and Clinical Psychology , 72 : 288 – 303 .
   PubMed Web of Science ®Google Scholar
• *Taylor , S. , Thordarson , D. S. , Maxfield , L. , Fedoroff , I. C. , Lovell , K. and Ogrodniczuk , J. 2003 . Comparative efficacy, speed, and adverse effects of three PTSD treatments: Exposure therapy, EMDR, and relaxation training . Journal of Consulting and Clinical Psychology , 71 : 330 – 338 .
   PubMed Web of Science ®Google Scholar
• *Trepka , C. , Rees , A. , Shapiro , D. A. , Hardy , G. E. and Barkham , M. 2004 . Therapist competence and outcome of cognitive therapy for depression . Cognitive Therapy and Research , 28 : 143 – 157 .
   Web of Science ®Google Scholar
• *van Minnen , A. , Hoogduin , K. A. , Keijsers , G. P. , Hellenbrand , I. and Hendriks , G. J. 2003 . Treatment of trichotillomania with behavioral therapy or fluoxetine: A randomized, waiting-list controlled study . Archives of General Psychiatry , 60 : 517 – 522 .
   PubMed Web of Science ®Google Scholar
• Varnell , S. , Murray , D. M. , Janega , J. B. and Blitstein , J. L. 2004 . Design and analysis of group-randomized trials: A review of recent practices . American Journal of Public Health , 94 : 393 – 399 .
   PubMed Web of Science ®Google Scholar
• Verma , V. and Le , T. 1996 . An analysis of sampling errors for the demographic and health surveys . International Statistical Review , 64 : 265 – 294 .
   PubMed Web of Science ®Google Scholar
• Wampold , B. E. 2001 . The great psychotherapy debate , Mahwah, NJ : Erlbaum .
   Google Scholar
• *Wampold , B. E. and Brown , G. S. 2005 . Estimating variability in outcomes attributable to therapists: A naturalistic study of outcomes in managed care . Journal of Consulting and Clinical Psychology , 73 : 914 – 923 .
   PubMed Web of Science ®Google Scholar
• Wampold , B. E. and Serlin , R. C. 2000 . The consequence of ignoring a nested factor on measures of effect size in analysis of variance . Psychological Methods , 5 : 425 – 433 .
   PubMed Web of Science ®Google Scholar
• *Watson , J. C. , Gordon , L. B. , Stermac , L. , Kalogerakos , F. and Steckley , P. 2003 . Comparing the effectiveness of process-experiential with cognitive-behavioral psychotherapy in the treatment of depression . Journal of Consulting and Clinical Psychology , 71 : 773 – 781 .
   PubMed Web of Science ®Google Scholar
• Winer , B. J. , Brown , D. R. and Michels , K. 1991 . Statistical principles in experimental design , New York, NY : McGraw-Hill .
   Google Scholar
• Zucker , D. M. 1990 . An analysis of variance pitfall: The fixed effects analysis in a nested design . Educational and Psychological Measurement , 50 : 731 – 738 .
   Web of Science ®Google Scholar