Cochrane’s risk of bias tool in the context of psychotherapy outcome research

References

• Abraha, I., Cherubini, A., Cozzolino, F., De Florio, R., Luchetta, M. L., Rimland, J. M., & Orso, M. (2015). Deviation from intention to treat analysis in randomised trials and treatment effect estimates: Meta-epidemiological study. BMJ, 350, h2445–h2445. doi: 10.1136/bmj.h2445
   PubMed Web of Science ®Google Scholar
• Armijo-Olivo, S., Ospina, M., da Costa, B. R., Egger, M., Saltaji, H., Fuentes, J., & Cummings, G. G. (2014). Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials. PLoS One, 9(5), e96920. doi: 10.1371/journal.pone.0096920
   PubMed Web of Science ®Google Scholar
• Barkowski, S., Schwartze, D., Strauss, B., Burlingame, G. M., Barth, J., & Rosendahl, J. (2016). Efficacy of group psychotherapy for social anxiety disorder: A meta-analysis of randomized-controlled trials. Journal of Anxiety Disorders, 39, 44–64. doi: 10.1016/j.janxdis.2016.02.005
   PubMed Web of Science ®Google Scholar
• Barth, J., Munder, T., Gerger, H., Nüesch, E., Trelle, S., Znoj, H., & Cuijpers, P. (2013). Comparative efficacy of seven psychotherapeutic interventions for patients with depression: A network meta-analysis. PLoS Medicine, 10(5), e1001454. doi: 10.1371/journal.pmed.1001454
   PubMed Web of Science ®Google Scholar
• Baskin, T. W., Tierney, S. C., Minami, T., & Wampold, B. E. (2003). Establishing specificity in psychotherapy: A meta-analysis of structural equivalence of placebo controls. Journal of Consulting and Clinical Psychology, 71, 973–979. doi: 10.1037/0022-006X.71.6.973
   PubMed Web of Science ®Google Scholar
• Basoglu, M., Marks, I., Livanou, M., & Swinson, R. (1997). Double-blindness procedures, rater blindness, and ratings of outcome: Observations from a controlled trial. Archives of General Psychiatry, 54, 744–748. doi: 10.1001/archpsyc.1997.01830200078011
   PubMedGoogle Scholar
• Bradley, H., Rucklidge, J., & Mulder, R. (2017). A systematic review of trial registration and selective outcome reporting in psychotherapy randomized controlled trials. Acta Psychiatrica Scandinavica, 135, 65–77. doi: 10.1111/acps.12647
   PubMed Web of Science ®Google Scholar
• Chalmers, I. (1993). The Cochrane collaboration: Preparing, maintaining, and disseminating systematic reviews of the effects of health care. Annals of the New York Academy of Sciences, 703, 156–165. doi: 10.1111/j.1749-6632.1993.tb26345.x
   PubMed Web of Science ®Google Scholar
• Chalmers, T. C., Smith, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., & Ambroz, A. (1981). A method for assessing the quality of a randomized control trial. Controlled Clinical Trials, 2, 31–49. doi: 10.1016/0197-2456(81)90056-8
   PubMedGoogle Scholar
• Chan, A.-W., & Altman, D. G. (2005). Epidemiology and reporting of randomised trials published in PubMed journals. The Lancet, 365, 1159–1162. doi: 10.1016/S0140-6736(05)71879-1
   PubMed Web of Science ®Google Scholar
• Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. JAMA, 291, 2457–2465. doi: 10.1001/jama.291.20.2457
   PubMed Web of Science ®Google Scholar
• Colagiuri, B. (2010). Participant expectancies in double-blind randomized placebo-controlled trials: Potential limitations to trial validity. Clinical Trials, 7, 246–255. doi: 10.1177/1740774510367916
   PubMed Web of Science ®Google Scholar
• Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: The new medical research council guidance. BMJ, 337, a1655. doi: 10.1136/bmj.a1655
   PubMed Web of Science ®Google Scholar
• Cuijpers, P., Li, J., Hofmann, S. G., & Andersson, G. (2010). Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: A meta-analysis. Clinical Psychology Review, 30, 768–778. doi: 10.1016/j.cpr.2010.06.001
   PubMed Web of Science ®Google Scholar
• Cuijpers, P., Smit, F., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). Efficacy of cognitive–behavioural therapy and other psychological treatments for adult depression: Meta-analytic study of publication bias. British Journal of Psychiatry, 196, 173–178. doi: 10.1192/bjp.bp.109.066001
   PubMed Web of Science ®Google Scholar
• Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S., & Andersson, G. (2010). The effects of psychotherapy for adult depression are overestimated: A meta-analysis of study quality and effect size. Psychological Medicine, 40, 211–223. doi: 10.1017/S0033291709006114
   PubMed Web of Science ®Google Scholar
• da Costa, B. R., Beckett, B., Diaz, A., Resta, N. M., Johnston, B. C., Egger, M., & Armijo-Olivo, S. (2017). Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: A prospective study. Systematic Reviews, 6, 376. doi: 10.1186/s13643-017-0441-7
   Google Scholar
• De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., & Overbeke, A. J. P. (2004). Clinical trial registration: A statement from the international committee of medical journal editors. New England Journal of Medicine, 351, 1250–1251. doi: 10.1056/NEJMe048225
   PubMed Web of Science ®Google Scholar
• Falkenström, F., Markowitz, J. C., Jonker, H., Philips, B., & Holmqvist, R. (2013). Can psychotherapists function as their own controls? Meta-analysis of the “crossed therapist” design in comparative psychotherapy trials. Journal of Clinical Psychiatry, 74, 482–491. doi: 10.4088/JCP.12r07848
   PubMed Web of Science ®Google Scholar
• Fleiss, J. L., Levin, B., & Paik, M. C. (2013). Statistical methods for rates and proportions. Hoboken, NJ: Wiley.
   Google Scholar
• Harrison, A., de la Cruz, L. F., Enander, J., Radua, J., & Mataix-Cols, D. (2016). Cognitive-behavioral therapy for body dysmorphic disorder: A systematic review and meta-analysis of randomized controlled trials. Clinical Psychology Review, 48, 43–51. doi: 10.1016/j.cpr.2016.05.007
   PubMed Web of Science ®Google Scholar
• Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., & Sterne, J. A. C. (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ, 343, d5928. doi: 10.1136/bmj.d5928
   PubMed Web of Science ®Google Scholar
• Higgins, J. P. T., & Green, S. (2011). Cochrane handbook for systematic reviews of interventions (Version 5.1.0). Retrieved from http://handbook-5-1.cochrane.org/
   Google Scholar
• Higgins, J. P. T., Savović, J., Page, M. J., & Sterne, J. A. C. (2016). Background information and detailed guidance on using the RoB 2.0 tool. Retrieved from https://sites.google.com/site/riskofbiastool//welcome/rob-2-0-tool.
   Google Scholar
• Jordan, V. M., Lensen, S. F., & Farquhar, C. M. (2017). There were large discrepancies in risk of bias tool judgments when a randomized controlled trial appeared in more than one systematic review. Journal of Clinical Epidemiology, 81, 72–76. doi: 10.1016/j.jclinepi.2016.08.012
   PubMed Web of Science ®Google Scholar
• Jüni, P., Altman, D. G., & Egger, M. (2001). Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ, 323, 42–46. doi: 10.1136/bmj.323.7303.42
   PubMed Web of Science ®Google Scholar
• Jüni, P., Witschi, A., Bloch, R., & Egger, M. (1999). The hazards of scoring the quality of clinical trials for meta-analysis. JAMA, 282, 1054–1060. doi: 10.1001/jama.282.11.1054
   PubMed Web of Science ®Google Scholar
• Kaptchuk, T. J. (2001). The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology, 54, 541–549. doi: 10.1016/S0895-4356(00)00347-4
   PubMed Web of Science ®Google Scholar
• Kirsch, I. (1990). Changing expectations: A key to effective psychotherapy. Pacific Grove, CA: Brooks/Cole.
   Google Scholar
• Laird, K. T., Tanner-Smith, E. E., Russell, A. C., Hollon, S. D., & Walker, L. S. (2017). Comparative efficacy of psychological therapies for improving mental health and daily functioning in irritable bowel syndrome: A systematic review and meta-analysis. Clinical Psychology Review, 51, 142–152. doi: 10.1016/j.cpr.2016.11.001
   PubMed Web of Science ®Google Scholar
• Leichsenring, F., Abbass, A., Hilsenroth, M., Leweke, F., Luyten, P., Keefe, J., & Steinert, C. (2017). Biases in research: Risk factors for non-replicability in psychotherapy and pharmacotherapy research. Psychological Medicine, 47, 1000–1011. doi: 10.1017/S003329171600324X
   PubMed Web of Science ®Google Scholar
• Leichsenring, F., & Rabung, S. (2008). Effectiveness of long-term psychodynamic psychotherapy: A meta-analysis. JAMA, 300, 1551–1565. doi: 10.1001/jama.300.13.1551
   PubMed Web of Science ®Google Scholar
• Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P., … Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine, 6(7), e1000100. doi: 10.1371/journal.pmed.1000100
   PubMed Web of Science ®Google Scholar
• Liebherz, S., Schmidt, N., & Rabung, S. (2016). How to assess the quality of psychotherapy outcome studies: A systematic review of quality assessment criteria. Psychotherapy Research, 26, 573–589. doi: 10.1080/10503307.2015.1044763
   PubMed Web of Science ®Google Scholar
• Morin, C. M., Colecchi, C., Brink, D., Astruc, M., Mercer, J., & Remsberg, S. (1995). How “blind” are double-blind placebo-controlled trials of benzodiazepine hypnotics? Sleep, 18, 240–245. doi: 10.1093/sleep/18.4.240
   PubMed Web of Science ®Google Scholar
• Munder, T., Brütsch, O., Leonhart, R., Gerger, H., & Barth, J. (2013). Researcher allegiance in psychotherapy outcome research: An overview of reviews. Clinical Psychology Review, 33, 501–511. doi: 10.1016/j.cpr.2013.02.002
   PubMed Web of Science ®Google Scholar
• Nüesch, E., Trelle, S., Reichenbach, S., Rutjes, A. W., Tschannen, B., Altman, D. G., & Jüni, P. (2010). Small study effects in meta-analyses of osteoarthritis trials: Meta-epidemiological study. BMJ, 341, c3515–c3515. doi: 10.1136/bmj.c3515
   PubMed Web of Science ®Google Scholar
• Olfson, M., Mojtabai, R., Sampson, N. A., Hwang, I., Druss, B., Wang, P. S., & Kessler, R. C. (2009). Dropout from outpatient mental health care in the United States. Psychiatric Services, 60, 898–907. doi: 10.1176/ps.2009.60.7.898
   PubMed Web of Science ®Google Scholar
• Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829–841. doi: 10.1037/0022-006X.75.6.829
   PubMed Web of Science ®Google Scholar
• Schulz, K. F., & Grimes, D. A. (2002a). Allocation concealment in randomised trials: Defending against deciphering. Lancet, 359, 614–618. doi: 10.1016/S0140-6736(02)07750-4
   PubMed Web of Science ®Google Scholar
• Schulz, K. F., & Grimes, D. A. (2002b). Blinding in randomised trials: Hiding who got what. Lancet, 359, 696–700. doi: 10.1016/S0140-6736(02)07816-9
   PubMed Web of Science ®Google Scholar
• Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
   Google Scholar
• Shiell, A., Hawe, P., & Gold, L. (2008). Complex interventions or complex systems? Implications for health economic evaluation. BMJ, 336, 1281–1283. doi: 10.1136/bmj.39569.510521.AD
   PubMed Web of Science ®Google Scholar
• Shinohara, K., Tajika, A., Imai, H., Takeshima, N., Hayasaka, Y., & Furukawa, T. A. (2015). Protocol registration and selective outcome reporting in recent psychiatry trials: New antidepressants and cognitive behavioural therapies. Acta Psychiatrica Scandinavica, 132, 489–498. doi: 10.1111/acps.12502
   PubMed Web of Science ®Google Scholar
• Spielmans, G., & Flückiger, C. (2018). Moderators in psychotherapy meta-analyses. Psychotherapy Research.
   Web of Science ®Google Scholar
• Sterne, J. A., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., & Carpenter, J. R. (2009). Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls. BMJ, 338, b2393. doi: 10.1136/bmj.b2393
   PubMed Web of Science ®Google Scholar
• Swift, J. K., & Greenberg, R. P. (2012). Premature discontinuation in adult psychotherapy: A meta-analysis. Journal of Consulting and Clinical Psychology, 80, 547–559. doi: 10.1037/a0028226
   PubMed Web of Science ®Google Scholar
• Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632–638. doi: 10.1177/1745691612463078
   Web of Science ®Google Scholar
• Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). New York, NY: Routledge.
   Google Scholar
• Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., & Sterne, J. A. C. (2008). Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: Meta-epidemiological study. BMJ, 336, 601–605. doi: 10.1136/bmj.39465.451748.AD
   PubMed Web of Science ®Google Scholar