A primer on meta-analysis of correlation coefficients: The relationship between patient-reported therapeutic alliance and adult attachment style as an illustration

Abstract

The aim of this article is twofold: to offer an introduction to meta-analysis using correlation coefficients to facilitate greater understanding of meta-analytic findings and to guide those interested in conducting meta-analyses. The authors review calculations for a weighted average effect size, the statistical significance of this effect, a test of homogeneity, confidence intervals, and file drawer analysis. They provide a running example of the relationship between patient-reported therapeutic alliance and adult attachment style. Results (k = 12, N = 581.17, weighted average r = .17, p < .0001, 95% confidence interval=.13–.21) indicated a positive, statistically significant relationship, suggesting that greater attachment security is associated with stronger therapeutic alliances, whereas greater attachment insecurity is associated with weaker therapeutic alliances. File drawer results suggested that some caution is warranted in terms of the size of the effect.

ABSTRACT

Dieser Artikels verfolgt zwei Ziele: erstens eine Einführung für die Durchführung von Meta-Analysen unter Verwendung von Korrelationskoeffizienten vorzustellen, um ein besseres Verständnis meta-analytischer Ergebnisse zu ermöglichen, und zweitens diejenigen anzuleiten, die an der Durchführung von Meta-Analysen interessiert sind. Die Autoren bewerten die Berechnung von gewichteten durchschnittlichen Effektgrößen, die statistische Signifikanz dieser Effekte, ein Homogenitätstest, Konfidenzintervalle und eine ,,File Drawer“-Analyse. Sie geben ein fortlaufendes Beispiel des Zusammenhangs zwischen der von den Patienten berichteten therapeutischen Arbeitsbeziehung und dem Bindungsstil im Erwachsenenalter. Die Ergebnisse (k = 12, N = 581.17, r = .17, p < .0001, 95% Konfidenzintervall = .13–.21) zeigten eine positive, statistisch signifikante Beziehung, welche andeutet, dass größere Bindungssicherheit mit einer stabileren therapeutischen Arbeitsbeziehung zusammenhängt, wohingegen größere Bindungsunsicherheit mit einer schwächeren therapeutischen Arbeitsbeziehung assoziiert war. Die Ergebnisse der ,,File Drawer“-Analyse weisen daraufhin, dass Vorsicht bei der Interpretation der Größe der Effekte geboten ist.

RÉSUMÉ

Le but de cet article est double : offrir une introduction à la méta-analyse utilisant des coefficients de corrélation pour faciliter une meilleure compréhension des résultats méta-analytiques et guider les personnes intéressées dans la conduite d'une méta-analyse. Les auteurs revoient les calculs pour les tailles de l'effet moyennes pondérées, la signification statistique de cet effet, un test d'homogénéité, les intervalles de confiance, et l'analyse des données pas exploitées (‘file drawer effect’. Ils fournissent un exemple de la relation entre l'alliance thérapeutique évaluée par le patient et le style d'attachement adulte. Les résultats (k = 12, N = 581.17, r = .17, p < .0001, 95% de l'intervalle de confiance = .13–.21) indiquent une relation positive, statistiquement significative, suggérant qu'une plus grande sécurité de l'attachement est associée avec une alliance thérapeutique plus forte, alors qu'une plus grande insécurité de l'attachement est associée avec une alliance thérapeutique plus faible. Les résultats de l'analyse des données non exploitées suggèrent qu'une certaine précaution est justifiée en termes de taille de l'effet.

Resumo

O objective deste artigo divide-se em: fornecer uma introdução às meta-análises usando coeficientes de correlação para facilitar o melhor conhecimento de resultados meta-analíticos, e guiar os interessados na condução de meta-análises. Os autores revêem cálculos de magnitude do efeito com médias ponderadas, a significância estatística deste efeito; o teste da homogeneidade; intervalos de confiança e file drawer analysis. Providenciam ainda um exemplo corrente da relação entre aliança terapêutica relatada pelo paciente e estilos de vinculação no adulto. Os resultados (k = 12, N = 581.17, r = .17, p < .0001, 95% intervalo de confiança= .13–.21) indicaram uma relação positive e estatisticamente significativa, sugerindo que vinculação segura maior está associada a alianças terapêuticas mais fortes, enquanto estilos de vinculação inseguros estão associados a alianças terapêuticas mais fracas. File drawer results sugerem que é necessário cuidados particulares com a magnitude do efeito.

ABSTRACT

L'articolo presentato ha un duplice obiettivo: offrire una introduzione alla meta-analisi riportando i coefficienti di correlazione per facilitare una maggiore comprensione dei risultati e orientare coloro che sono interessati a condurre meta-analisi. Gli autori revisionano l'effetto di una media ponderata, la significatività statistica di questo effetto, un test di omogeneità delle medie, l'intervalli di confidenza e la drawer analysis. Tutto sarà facilitato dalla presentazione di esempi che si basano sul rapporto tra terapeuta e paziente e stile di attaccamento adulto. I risultati (k = 12, N = 581,17, r = .17, p <,0001, intervallo di confidenza 95% = ,13 -. 21) indicano una relazione statisticamente significativa la quale suggerisce che un maggiore attaccamento è associato ad una più forte alleanza terapeutica, considerando che una maggiore attaccamento insicuro è associato ad una più debole alleanza terapeutica. I risultati derivanti dalla drawer analysis sottolineano la necessità di avere una certa prudenza nella generalizzazione dei risultati in termini di dimensione degli effetti.

Keywords:

• meta-analysis
• correlation coefficients
• attachment
• alliance

Acknowledgements

This article is based in part on the doctoral dissertation of Marc Diener. An earlier version of this study was presented at the annual meeting of the Division of Psychoanalysis (39) of the American Psychological Association, New York, April 2008. We thank Drs. Larry Hedges, Blair Johnson, Gregory Meyer, Robert Rosenthal, and James Sexton for their helpful suggestions on a number of statistical issues. We also thank Dr. William Gottdiener for his comments and suggestions on an earlier version of this article. We are grateful to Dr. Rosemarie Vala Stewart for her assistance in reviewing the literature for relevant studies. Finally, we thank Drs. Diane Arnkoff, Dennis Kivlighan, Brent Mallinckrodt, Margaret Parish, and Eric Sauer for providing additional information on the studies included in the analyses.

Notes

1. Computation programs, including one created by Marc Diener using Excel and available for download at the supplemental materials web site (http://www.informaworld.com/mpp/uploads/metaanalysisprogramv.3.4.xls), exist that permit calculation of a number of different meta-analytic methods.

2. The approach used here to aggregate multiple effect sizes from a single study follows that of Martin, Garske, and Davis (2000). See Hunter and Schmidt (2004) for a discussion of various options in dealing with multiple effects. Still other approaches to aggregating multiple effect sizes from a single study have been presented in the literature (e.g., Gleser & Olkin, 1994; Rosenthal & Rubin, 1988), although many of these require data that may not be typically available to the meta-analytic researcher (Rosenthal, 1991).

3. We discuss calculations of statistical significance because it is standard in both primary and meta-analytic research to report such calculations. To avoid misrepresentation, however, of the Hunter–Schmidt approach, we refer the reader to Hunter and Schmidt (1990) and to Schmidt and Hunter's (1999) discussion of statistical significance testing.

4. Another critique of homogeneity tests refers to their potential lack of sufficient power to demonstrate heterogeneity as a consequence of including only a small number of individual studies in the meta-analysis (National Research Council, 1992; see also Hedges & Pigott, 2001).

5. We discuss calculation of confidence intervals because these are commonly reported in both primary and meta-analytic research. To avoid misrepresentation of the Hunter–Schmidt approach, however, the reader is referred to Hunter and Schmidt (2004) and Schmidt and Hunter (1999) for a discussion of the relative importance of confidence versus “credibility” intervals.

6. See Hunter and Schmidt's (2004) discussion of an alternative approach to evaluating publication bias (or other forms of availability bias) using funnel plot graphs.

7. We should note that three of four correlations in Bruck, Winston, Aderholt, and Muran (2006) were reported as nonsignificant; the actual effect sizes were not reported and the original data are presently unavailable from the authors (J.C. Muran, personal communication, November 13, 2007). In order to be most conservative, we followed the procedure used by Horvath and Symonds (1991) as well as Martin et al. (2000) to code these nonsignificant effect sizes as r = 0. However, we also ran secondary analyses excluding these nonsignificant effect sizes, which yielded results that were essentially similar; however, the weighted average effect size was slightly larger, with =.19.