A Meta-Analysis on Differences and Associations between Alliance Ratings in Child and Adolescent Psychotherapy

ABSTRACT

Objective

The alliance in child and adolescent psychotherapy is widely recognized as an important factor in therapy. Studies on the alliance have increasingly focused on assessment of the alliance as a dyadic construct, measuring both client and therapist alliance ratings. However, cross-informant reports of the alliance in child psychotherapy have not yet been subjected to meta-analysis. Therefore, the present meta-analysis aims to increase knowledge on the degree of convergence and divergence between child, parent, therapist, and observer alliance ratings in child and adolescent psychotherapy.

Methods

A series of three-level meta-analyses of 78 studies was conducted to investigate differences and associations between child, parent, therapist, and observer alliance ratings in child and adolescent psychotherapy.

Results

Findings indicated that children and parents in general rated the alliance more positively than their therapists (d = 0.35, d = 0.72, respectively), and that child-therapist and parent-therapist alliance ratings were moderately correlated (r = .32, r = .23, respectively). Associations between child and therapist ratings and observer ratings were moderate to large (r = .43, r = .53, respectively).

Conclusions

It can be concluded that children and parents generally report more positively on the alliance compared to their therapists, which is consistent with research on the alliance in adult populations. The small to moderate associations between alliance ratings indicate that individuals to some extent have a shared perspective on their alliance, and that the various perspectives on alliance should be acknowledged when dealing with children and parents in therapy. Implications for future research are discussed.

The therapeutic alliance – also referred to as working alliance – is widely recognized as an important factor in psychotherapy with adults, but also with children and their parents (Horvath et al., 2011; Karver et al., 2018; McLeod, 2011; Norcross & Lambert, 2018). The therapeutic alliance consists of three interdependent aspects: the personal and emotional bond between client and therapist, the agreement on therapy goals, and the agreement on tasks of therapy (Bordin, 1979, 1994). Most definitions focus upon these three aspects of the client-therapist alliance in adult as well as child and adolescent psychotherapy (Elvins & Green, 2008; Zack et al., 2007). However, it has been proposed that children have a different understanding of the alliance than adult clients (for a conceptual review, see Shirk et al., 2010; Zack et al., 2007), although it is unclear to what extent a multidimensional alliance construct is relevant in therapy with children and their parents, and which defining elements of the child-therapist and parent-therapist alliance are most important (Elvins & Green, 2008; Nuñez et al., 2021; Ryan et al., 2021).

Self-report alliance measures are widely used in clinical practice as part of client feedback and to monitor the alliance during treatment (Rober et al., 2020). Actively monitoring the alliance by addressing the degree of alliance agreement or divergence is recognized as important to improve treatment efficacy (Horvath et al., 2011). In alliance research, increasing attention has been given to assessing both client and therapist alliance ratings and the degree of divergence or agreement to properly assess the alliance as a dyadic construct instead of using ratings of a single actor (i.e., client or therapist).

Recent studies have found that alliance discrepancies as well as alliance agreement are important predictors of therapeutic outcomes in adult and youth psychotherapy literature (Fjermestad et al., 2016; Friedlander et al., 2018; Zandberg et al., 2015; Zilcha-Mano et al., 2017). Several meta-analyses on the alliance-outcome association in child populations have found that alliance ratings by different informants (i.e., child, parent, therapist, and observer) are differentially related to treatment outcomes of the child, although effect sizes for both perspectives have been shown to range from small to moderate (r = .14 to r = .28; Karver et al., 2019; McLeod, 2011). Additionally, the parent-therapist alliance may impact child outcomes through increased engagement in treatment, session attendance, and may also be associated with the child-therapist alliance, such that if a parent has a good alliance with the therapist, the child may also develop a positive alliance with the therapist (De Greef et al., 2017; Kazdin et al., 2006; Welmers-Van de Poll et al., 2020). Contrarily, the child- and parent-therapist alliances should be seen as distinct, such that a good child-therapist alliance can co-exist with poor parent-therapist alliance and vice versa (Hawley & Garland, 2008). Notably, studies in family therapy have increasingly focused on the operationalization and measurement of the alliance as a systemic construct (Welmers-Van de Poll et al., 2020).

These findings indicate that the alliance in child and adolescent psychotherapy should be understood from different perspectives, and should ideally be measured as a dyadic (or even systemic) construct, that is, as a product of alliance ratings from multiple perspectives. Increasing knowledge on how alliance ratings of different actors are related is therefore important. Also, knowledge on the relation between child, parent, therapist, and observer alliance ratings is of methodological interest in studying how the alliance is related to therapy process factors and treatment outcomes. Therefore, the purpose of the present meta-analysis is to synthesize the empirical literature on the differences and associations between child, parent, therapist, and observer alliance ratings in child and adolescent psychotherapy.

The Child-Therapist Alliance in Child and Adolescent Psychotherapy

There are several differences that have to be considered when examining the child-therapist alliance in child and adolescent psychotherapy compared to the alliance in individual adult psychotherapy. First, children and adolescents are often not self-referred for therapy. Instead, they are referred by their parents, teachers, or other authorities, such as social services, because of perceived problem behavior (DiGiuseppe et al., 1996; Shirk et al., 2010). As a consequence, children may show more resistance to therapy, which makes it more difficult for therapists to establish a therapeutic alliance. Also, developmental issues may complicate the alliance between child and therapist. Younger children may have age-related cognitive limitations in understanding the need of treatment (Shirk & Saiz, 1992). Adolescents may be in conflict with their parents over the nature of the problem and the need of treatment, and additionally experience problems with accepting authority (DiGiuseppe et al., 1996; Shirk et al., 2010). Moreover, adolescents orient themselves more toward their peers than parents or other adults (Harris, 1998). These developmental issues may cause resistance to therapeutic change, and may complicate the alliance with an adult professional (DiGiuseppe et al., 1996; Shirk & Karver, 2003). Thus, in child and adolescent therapy, a therapist needs to form a separate alliance with parents and possibly with other family members too, in addition to the alliance with the child, in order to create a therapeutic environment (Accurso et al., 2013; Green, 2006). Developmental differences between children and adults, and children’s resistance to treatment may complicate the formation of a therapeutic alliance between therapist and children, possibly resulting in lower alliance ratings of the child than their therapists compared to adult clients.

Despite factors that may result in lower alliance ratings of children, research on the alliance in child populations has shown that child ratings of the alliance are often relatively high compared to therapist ratings (Shelef et al., 2005; Shirk et al., 2010). Alternatively, some studies have found no significant differences between child and therapist alliance ratings, and some studies found lower child alliance ratings compared to therapist alliance ratings (Duppong Hurley et al., 2015; Fjermestad et al., 2016). In adult psychotherapy, studies have found that clients also rate the alliance higher than their therapist (Doran, 2016; Simmonds, 2016; Tryon et al., 2007). A meta-analysis on the difference between client and therapist ratings in adult populations showed that client ratings of the alliance were overall substantially higher than therapist ratings (d = 0.63, Tryon et al., 2007). Further, Tryon et al.’s (2007) meta-analysis on alliance ratings in adult psychotherapy showed that client and therapist alliance ratings were moderately and positively correlated (r = .36). This suggests that client and therapist perspectives overlap to some extent, but they can be seen as distinct perspectives too.

It has been proposed that systematic differences in alliance scores between a client and therapist may reflect different interpretations of the alliance construct (Horvath, 2018; Tryon et al., 2007). Large discrepancies (i.e., divergence) between scores indicate that clients consistently report the alliance with their therapist as more positive or negative than the therapist, which raises the question how both actors interpret and value various aspects of the alliance. Also, a high level of congruence (i.e., convergence) between client and therapist alliance scores indicates that, in general, higher alliance scores of the client are related to higher scores of the therapist. Notably, in research examining divergence or convergence between alliance ratings of different informants generally pertains to examining absolute or relative differences between scores, respectively, which can provide different information on discrepancies and similarities in alliances, but does not always or necessarily lead to different interpretations of study results.

Research on cross-informant assessments in mental health care has consistently shown small to moderate associations (De Los Reyes et al., 2015, 2019; De Los Reyes & Ohannessian, 2016; De Los Reyes et al., 2013). Research suggests that informant characteristics are often not related to the degree of convergence or divergence between informant ratings, instead the context in which psychological phenomena are assessed by different informants (e.g., parent vs. teacher reports on child behavior) may explain discrepancies between ratings. In alliance research, different informants assess their alliance in which they jointly participate. Also, different informants experience the alliance in the same context. Therefore, it would be plausible to suggest that cross-informant ratings of the alliance would be moderately correlated. Considering the increasing amount of studies on the alliance in child and adolescent psychotherapy, a meta-analytic study on differences and associations between child, parent, and therapist ratings is important to gain insight in the magnitude of these associations, but also to examine potential moderators of these differences and associations.

Measuring the Alliance in Child and Adolescent Psychotherapy

Self-report alliance measures are a direct assessment of the perception of the alliance by children and their parents, and therefore important tools used in clinical practice for client feedback and monitoring of treatment progress (Mihalo & Valenti, 2018). On a conceptual level, children, parents, and therapists may have different expectations, understandings, and experiences of their alliance with each other, meaning that there may be different defining aspects of the alliance that are relevant to each perspective, possibly resulting in difficulties regarding measurement of the child-therapist or parent-therapist alliance. Children and adolescents report a positive alliance mostly as a function of professional characteristics and behaviors such as kindness, trustworthiness, and emotional support (Baylis et al., 2011; Campbell & Simmonds, 2011; Hawks, 2015; Houlding, 2014). It has also been proposed that alliance perceived by children may primarily be an affective (personal bond oriented) instead of a cognitive construct (task and goal oriented; Nuñez et al., 2021; Ormhaug et al., 2015; Zack et al., 2007). It is plausible to suggest that parents and therapists may be more focused on attitudes and behaviors related to the collaborative aspect of the alliance, such as open communication, sharing information, and working on treatment goals. However, empirical research in this regard is currently underdeveloped.

Several studies on self-report measures of alliance in child populations have found a different underlying factor structure than self-report measures in adults, indicating that the theoretically proposed three-factor alliance model by Bordin (i.e., personal bond, agreement on tasks, and agreement on goals) is not well supported empirically in children (Anderson et al., 2012; Cirasola et al., 2021; DiGiuseppe et al., 1996; Ormhaug et al., 2015). These studies show that the alliance concept in children may be viewed as a one-dimensional construct instead of the theoretically proposed alliance dimensions in adults, which could be explained by the child’s incapacity to distinguish between the personal bond, task, and goal dimensions of alliance (DiGiuseppe et al., 1996; Zack et al., 2007). Studies on self-report measures to assess the parent-therapist alliance also indicate that the alliance construct could be best represented by two factors: personal bond and collaboration on therapeutic tasks (Accurso et al., 2013; Anderson et al., 2012). These findings indicate that assessment of the alliance between therapists, children, and parents using self-report measures may not capture all relevant aspects of the alliance from different perspectives, which could potentially lead to systematic errors in measurement and interpretation of alliance scores.

Several observer alliance measures have been developed for use in child and adolescent psychotherapy, of which the Therapy Process Observational Coding System for Child Psychotherapy – Alliance Scale (TPOCS-A, McLeod & Weisz, 2005) is the most widely used. Although observer-based measures do not assess the perception of the client directly, they are important to address several methodological issues in alliance research. Using observer measures requires training of multiple observers and is time consuming, but allows for a more in-depth and objective assessment of the alliance. Observer alliance measures are often used in studies on the alliance in relation to therapeutic outcomes or other therapy process factors to overcome common-method bias (e.g., use of self-report alliance ratings in relation to self-reported outcomes) or when measuring alliance repeatedly within-session. Using observer alliance ratings can overcome shortcomings of self-report alliance measures, such as potential ceiling effects or treatment effects reflected in client-rated alliance scores (McLeod & Weisz, 2005; Shelef et al., 2005). Studies have shown positive correlations between child and parent ratings and observer ratings of the alliance (Langer et al., 2011; McLeod & Weisz, 2005). A strong correlation between observer alliance measures and self-report alliance measures would indicate that observer measures are valid measures to study the alliance in child psychotherapy. A meta-analytic examination of the association between observer rated alliance and self-reported alliance in child therapy is therefore important to examine the degree of convergence of these ratings.

The Present Study

To summarize, the differences and associations between alliance ratings of different actors in child and adolescent psychotherapy (i.e., child, parent, therapist, and observer) have not yet been subjected to meta-analysis. To the best of our knowledge, the present meta-analysis is the first to examine the convergence and divergence of alliance ratings in child and adolescent psychotherapy. As the number of studies on the alliance in child and adolescent psychotherapy has substantially increased over the past decades, it is important to meta-analytically examine the associations between child, parent, therapist, and observer alliance ratings as well as the differences between these ratings.

The main aim of this study was to increase knowledge on the differences and associations between alliance ratings reported by child, parent, and therapist alliance ratings. We investigated whether study characteristics, such as age category, type of problem behavior, and timing of alliance moderated the differences and associations between alliance ratings. It was expected that children and parents would rate the alliance more positively than their therapists, and that child and therapist alliance ratings as well as parent and therapist ratings would demonstrate small to modest associations.

Methods

Sample of Studies

The following selection criteria were formulated in order to find relevant studies for inclusion in the separate meta-analyses: (a) the alliance construct had to be operationalized as a collaborative relationship, consisting of a bond aspect and a collaborative aspect; (b) the mean age of the child had to be below 18 years; (c) the child and/or parents had to receive a therapeutic intervention; (d) the child and/or parent had to be the unit of measurement (instead of the family as a whole); (e) the therapist with whom an alliance was formed had to be clearly specified (as opposed to forming an alliance with multiple therapists); and (f) the study had to include a validated (formal) measure of the alliance between therapist and child, adolescent, or parent; (g) studies had to focus on at least two perspectives of the alliance, such as of therapists, children, parents, or observers, respectively. Studies were excluded if subjects were treated for a physical problem, if the alliance concept was not clearly described, or if the timing of alliance assessment was not clearly described.

In an attempt to find all relevant published articles, various stepwise search procedures were carried out. First, an online database search covering up to June 1^st 2021 was conducted on Sciencedirect, Wiley Online Library, Taylor and Francis Online, SAGE Journals, Web of Science, and Springerlink. Also, a search was conducted in Proquest in order to identify possibly overlooked published studies and unpublished dissertations. In doing so, the terms alliance, therapeutic alliance, working alliance, helping alliance, treatment alliance, therapeutic relationship, treatment, therapy, and psychotherapy were crossed with youth, child, adolescent, parent and used in various combinations for titles, abstracts, and keywords, resulting in 2,225 references (script available upon request). Second, relevant research reviews and prior meta-analyses on the alliance in child and adolescent psychotherapy (e.g., Elvins & Green, 2008; Karver et al., 2018, 2019; McLeod, 2011; Murphy & Hutton, 2018; Shirk & Karver, 2003; Shirk et al., 2011) were used to identify possibly overlooked studies. Google Scholar was used to retrieve studies that were found in reference sections of included studies or through review articles, which were not found in the database search. Twelve additional studies were identified by this method. In order to retrieve several studies and dissertations, authors were contacted. Studies were screened based on title and abstract for relevance, based on aforementioned inclusion criteria.

This search method resulted in a pool of 127 studies of which the full text was reviewed. A total of 49 studies were excluded; primary reasons were that studies used a different operationalization of the alliance construct, studies reported on the same sample, or the therapist was not clearly defined. If studies that assessed the alliance from multiple perspectives (e.g., child and therapist) did not report means and standard deviations of these ratings or a product-moment correlation between ratings, authors were contacted in order to obtain these results. In total, 11 authors were contacted to obtain additional information regarding 11 studies. One author responded to our inquiry and was able to provide the information requested.

Studies included in the meta-analysis are listed in Appendix A. A flow chart of our search strategy and screening process is depicted in Figure B1 in Appendix B. The search resulted in a total pool of 78 studies, of which 56 studies were included in separate meta-analyses to examine differences between child and therapist alliance ratings, parent and therapist ratings, or child and parent ratings. Also, 64 studies were included in the meta-analyses on the association between alliance ratings across raters (between child, parent, therapist, or observer).

Coding of the Studies

We considered a number of variables to include in moderator analyses, which are divided into study characteristics, sample characteristics, alliance characteristics, and treatment characteristics. The inter-rater reliability proved to be satisfactory, with Kappa’s ranging from .78 to 1.00, and intraclass correlations (average measures) ranging from .73 to 1.00.

Study Characteristics

Information about the study was coded for each manuscript: author name(s), publication status, journal and journal impact factor, year of publication, and the country in which the study was conducted (United States, Europe, other).

Sample Characteristics

Information about the following variables was coded for each study: child mean age and age category (mean age above and below 12 years), child gender (percentage male), child race (percentage Caucasian), referral source (recruited, help-seeking, mandated, and not reported), and whether children had a DSM diagnosis. Regarding age category, studies were coded based on mean age of children above and below 12 years. The following information regarding child problem behavior was coded: target problem of the child for which the sample received treatment (internalizing problems, externalizing problems, mixed problems, substance abuse problems, and eating disorders).

Alliance Characteristics

Each alliance measure used in the studies was coded into the following categories: type of alliance (child-therapist alliance and parent-therapist alliance), alliance rater (child-report, parent-report, therapist-report, and observer-report), timing of alliance assessment (early, midtreatment, late in treatment, and averaged scores), type of alliance measure (questionnaire and observer measure), and whether the alliance measure was specifically developed for children or adults. For the operationalization of timing of alliance measurement, assessment of alliance within one-third of treatment (often within several sessions) was coded as early in treatment, assessment of alliance at or around midtreatment was coded as midtreatment, and assessment of alliance near the end of treatment or at posttreatment was coded as late in treatment. Further, several studies reported on averaged alliance scores, in which case timing of alliance was coded as averaged scores.

The therapist alliance measure was coded into whether the measure assessed the child’s or therapist’s perspective on the alliance. The therapist version of the TASC (Therapeutic Alliance Scale for Children; Shirk & Saiz, 1992) uses the therapist rating of the child’s perspective of alliance instead of the therapist’s own perspective (with the exception of a small number of studies in which the measure was adapted), whereas the items of the WAI (Working Alliance Inventory, Horvath & Greenberg, 1989) are based on both perspectives. It is important to account for the difference between therapist versions of the alliance measure, and to examine whether the perspective from which the alliance is assessed moderates the magnitude of the differences and associations between child-therapist alliance ratings. Methodological differences in how the alliance is assessed may account for the discrepancies (or congruence) between informant ratings, which can be considered a methodological issue in research on cross-informant assessment in other areas of psychology too (De Los Reyes & Ohannessian, 2016). Also, internal consistency coefficients (Cronbach’s alpha) were coded for child, parent, and therapist versions of the alliance measure. If a study did not provide this information, the mean Cronbach’s alpha value was used of the specific version of an alliance measure, based on the available alpha values in the total study sample.

Treatment Characteristics

Treatments described in the studies were coded into several treatment characteristics: treatment setting (outpatient, inpatient, community/home-based, school, or not defined), use of a treatment manual, and treatment type (CBT and non-CBT). The studies that met the inclusion criteria were coded by the first author. Twenty percent of the studies was independently coded by the second author using a coding manual in order to calculate intercoder agreement. Reliability was computed with Cohen’s kappa for categorical variables and intraclass correlations coefficients (ICCs) for continuous variables.

Statistical Analyses

For the studies that reported means and standard deviations of different raters, we calculated the standardized mean difference between raters. Cohen’s d was calculated based on means and standard deviations (Rosenthal & DiMatteo, 2001, p. 71). To calculate the standardized mean difference between child or parent and therapist ratings, the mean score of the therapist’s alliance rating was subtracted from the mean score of the child or parent alliance rating, and divided by the pooled standard deviation of both ratings. The standardized mean difference between child and parent ratings was calculated by subtracting the parent’s mean alliance score from the child’s mean alliance score. A positive d-value was assigned if mean child or parent alliance scores were higher than their mean therapist scores, and if child alliance ratings were higher than parent alliance ratings.

We examined the difference between child, parent, and therapist ratings when the same alliance measure and scoring format was used. Observer alliance measures are not equivalent to self-report measures, in that a different format and number of questions are used. Therefore, mean scores of the alliance derived from observer measures were not compared to child, parent, or therapist alliance ratings.

Given that the therapist version of the alliance measure differed across studies (i.e., some measures assessed the therapist’s rating of the child’s perception of the alliance, whereas other measures assessed the therapist’s own perception of the alliance), it was necessary to account for these separate therapist versions in the analyses. We therefore chose to include studies that used both types of therapist measures to calculate an overall ES, but also reported separate ES estimates, and conducted moderator analyses for both types of measures separately.

With regard to the studies that reported correlations between raters, almost all studies reported Pearson’s r correlations and two studies reported Spearman’s rho. One study that reported intraclass correlation coefficients (ICCs) was excluded, because ICCs are based on both the absolute and relative difference between scores, whereas Pearson’s r correlations are based in the relative difference between scores. If an association between alliance scores of a dyad was in the expected direction (i.e., higher levels of child alliance scores were positively related to therapist alliance scores), a positive r-value was assigned, whereas a negative r-value was assigned to associations that were not in the expected direction.

We checked for outliers by calculating standardized scores of effect sizes in order to identify standardized scores larger than 3.29 or smaller than −3.29 (Tabachnik & Fidell, 2013). One effect size exceeding 3.29 was identified as an outlier. To reduce the impact of the outlier, the raw d value of this outlier was substituted by a new d value equal to the highest effect size within the normal range. Sensitivity analyses were conducted to test whether the outlier had an effect on the overall effect size estimate, by conducting the analysis including the outlier.

Each correlation was transformed to Fisher’s Z before combined effect sizes were calculated and moderator analyses were conducted (Mullen, 1989) and transformed back into Pearson r after analyses for reporting and interpretation. The resulting effect sizes were interpreted following Cohen’s (1988) guidelines for approximate thresholds; r is a small effect when at least .10, r is a medium effect when at least .30, and r is a large effect when at least .50. The effect size Cohen’s d for standardized mean differences is considered small when at least 0.20, d is a moderate effect when at least 0.50, and d is a large effect when at least 0.80.

Most studies reported on multiple raters of alliance and multiple times of measurement and therefore, more than one effect size could be calculated. It is assumed that the subjects in the study samples are independent (Hedges & Olkin, 1985) and calculating multiple effect sizes per study violates the assumption of non-independence (Lipsey & Wilson, 2001). To deal with this dependency we used a three-level random effects model (Cheung, 2014; Van den Noortgate et al., 2013, 2014) to calculate overall effect sizes and conduct moderator analyses. This approach models three sources of variance: sampling variance of the observed effect sizes (level 1), variance between effect sizes from the same study (level 2), and variance between studies (level 3). An important advantage of this three-level approach to meta-analysis is that (dependent) effect sizes extracted from the same study can be included in the analysis. By using all available effect sizes, all information can be preserved and more statistical power can be achieved compared to traditional approaches to meta-analysis.

For the statistical analyses we used the function “rma.mv” of the metafor package (Viechtbauer, 2010) in the R environment (version 3.4.1; R Core Team, 2015). The R syntax and protocol (Assink & Wibbelink, 2016) were based on procedures outlined by Van den Noortgate et al. (2013, 2014), modeling the three sources of variance. The t-distribution was used for testing individual regression coefficients of the meta-analytic models and for calculating the corresponding confidence intervals (Knapp & Hartung, 2003). This approach accounts for uncertainty of the amount of residual variance, which leads to a more accurate estimate of the standard errors and fewer type-I errors.

Iterative maximum likelihood estimation (MLE) procedures were applied to estimate unknown parameters. The intercept only model (without moderators) through MLE is equivalent to the traditional random-effects model by Hedges and Olkin (1985). In the overall model, covariates can be added to test potential moderators. Van den Noortgate and Onghena (2003) compared multilevel meta-analysis to the traditional meta-analytic approach and concluded that the results obtained by the maximum likelihood multilevel approach are not substantially different from the results of the traditional random-effects approaches for intercept only models. Moreover, the MLE procedure is in general superior to the traditional fixed-effects approaches (Van den Noortgate & Onghena, 2003).

For all associations, forest plots were generated based on the guidelines outlined by Fernández-Castilla et al. (2020) extended for use in three-level meta-analysis. Forest plots in traditional meta-analysis provide a visual representation of the effect size of a study based on the sample size and confidence interval of effect sizes. The extended forest plot contains additional confidence intervals based on the sampling variance of individual observed effect sizes within the study and the number of effect sizes within the study. Therefore the forest plots provide information about the variability in effect sizes among studies and the relative contribution to the overall effect size estimate.

When models were extended with categorical moderators consisting of three or more categories, the omnibus test of the null hypothesis that all group mean effect sizes are equal, followed an F-distribution. To determine whether the variance between effect sizes from the same study (level 2), and the variance between studies (level 3) were significant, two separate one-tailed log-likelihood-ratio-tests were performed in which the deviance of the full model was compared to the deviance of a model excluding one of the variance parameters. The sampling variance of observed effect sizes (level 1) was estimated by using the formula of Cheung (2014). All model parameters were estimated using the restricted maximum likelihood estimation method and before moderator analyses were conducted, each continuous variable was centered around its mean and dichotomous dummy variables were created for all categorical variables (Tabachnik & Fidell, 2013). In multilevel regression analyses, the intercept is the reference category, while the dummy variables test if, and to what extent, the other categories deviate from the reference category. The log-likelihood-ratio-tests were performed one-tailed and all other tests were performed two-tailed. We considered p-values < .05 as statistically significant.

Publication Bias

The tendency of selected publication by journals to accept papers that report significant associations – referred to as publication bias – can influence the overall estimates of effect sizes in a meta-analysis and therefore its conclusions (Rosenthal, 1979; Rothstein, 2008). This problem was designated as the ‘file drawer problem’ by Rosenthal (1979). We obtained all unpublished material as best as possible, which is the simplest solution to the problem of publication bias (Mullen, 1989).

We applied three methods to address potential publication bias. First, we used Egger regression (Egger et al., 1997; Fernández-Castilla et al., 2021), which tests the degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates (effect size divided by its standard error) against the estimate’s precision (the inverse of the standard error). A significant Egger regression test indicates funnel plot asymmetry. Following Fernández-Castilla et al. (2021), an adapted version of the Egger’s test was used in which we investigated the relation between the effect size and the standard error in order to account for dependency of effect sizes. In doing so, the standard error of the effect size was included as a moderator in the regression model. Consequently, the degree of funnel plot asymmetry was interpreted via the moderator’s regression weight and associated p-value. A second method to address publication bias was the use of an extension of the funnel plot test for use in three-level meta-analysis (Fernández-Castilla et al., 2021).

For all associations, both funnel plots of all effect sizes and plots of study effects are depicted, following guidelines by Fernández-Castilla et al. (2020) on the use of funnel plots in three-level meta-analyses. Funnel plots of all effect sizes commonly used in meta-analysis to examine whether publication bias or selective reporting bias might be present (missing effect sizes at the lower-left part of the funnel plot). In a funnel plot of study effects, separate random-effects meta-analyses are conducted on each study, resulting in a dot based on the sample size and the number of effect sizes within the study.

We also performed a trim and fill procedure for all associations (Duval & Tweedie, 2000), testing whether effect sizes are missing on the left side of the distribution – since publication bias would only be likely to occur in case of non-significant or unfavorable (i.e., negative) results, indicating that the overall estimate found in the meta-analysis is an overestimation of the true effect. Alternatively, the trim and fill procedure could also indicate missing studies on the right side of the distribution, indicating that the overall estimate found in the meta-analysis is an underestimation of the true effect due to possible selection bias.

Previous simulation studies have shown that effect size estimates based on imputation of effect sizes after the trim-and-fill procedure may not be accurate (Fernández-Castilla et al., 2021; Peters et al., 2007). Therefore, we used the trim-and-fill procedure as outlined by Fernández-Castilla et al. (2021), which estimates the number of effect sizes imputed at the right side or left side of the distribution, to examine whether the overall effect size estimates were sensitive to potential presence of publication bias. Fernández-Castilla et al. (2021) have proposed a method in which the estimated number of effect sizes on the left side of the funnel plot distribution is related to a cutoff value of the estimator of the trim-and-fill method, based on the population ES (effect size) and power (number of effect sizes). If the number of imputed studies exceeds the cutoff value, this may be indicative of publication bias.

Results

Descriptive Statistics of the Study Sample

Table B1 (see Appendix B) shows the characteristics of the study sample included in the meta-analyses. The sample contains a total of 78 studies, which is divided into different subgroups. The studies were completed between 1992 and 2021. Fifty-six studies reported means and standard deviations of child, parent, and therapist alliance ratings, producing 170 effect sizes. Sixty-four studies reported on the association between alliance ratings of several informants (e.g., child-therapist, parent-therapist), producing 145 effect sizes (ESs). Of all studies included in the meta-analyses, only one study used parent self-report alliance ratings and an observer measure of the parent-therapist alliance (McLeod & Weisz, 2005). Therefore, the association between parent alliance ratings and observer alliance ratings of the parent-therapist alliance was not subjected to meta-analysis. The association between observer ratings of child-therapist alliance and observer ratings of the parent-therapist alliance was reported in several studies. Although observers are not involved in the alliance, we included this association in the review in order to examine the extent to which observer scores of both types of alliance are related. This may be of interest to investigate whether observer measures used to assess both alliances demonstrate conceptual overlap.

Meta-analyses on Differences between Alliance Ratings

Three separate meta-analyses were carried out on the mean difference between child-therapist alliance ratings, parent-therapist ratings as well as child-parent ratings (Table 1). Forty-one studies consisting of 47 independent samples reporting 92 effect sizes (ESs) presented means and standard deviations of child and therapist alliance ratings of 7,080 children and therapists. The mean overall estimate was significant (d = 0.346, 95% CI = 0.222, 0.469, p < .001), indicating a small difference between raters, showing that children reported higher alliance ratings than their therapists. Sensitivity analysis including one outlier did not significantly change the overall ES (d = 0.351, 95% CI = 0.222, 0.480, p < .001).

Table 1. Results for the overall mean effect sizes for differences between child, parent, and therapist alliance ratings.

Type of difference	# studies^a	# es	Mean d (se)	95% ci	p	σ^2level2	σ^2level3	% var. level 1	% var. level 2	% var. level 3
Child ratings – therapist ratings	47	92	0.346 (0.062)	0.222, 0.469	<.001***	0.000	0.146***	17.8	0.0	82.2
Parent ratings – therapist ratings	23	47	0.715 (0.094)	0.527, 0.903	<.001***	0.000	0.176***	14.6	0.0	85.4
Child ratings – parent ratings	18	31	−0.406 (0.080)	−0.569, −0.243	<.001***	0.000	0.081**	32.2	0.0	67.8

ci = confidence interval; es = effect size; σ²_level2 = variance between effect sizes (within studies); σ²_level3 = variance between effect sizes (between studies); % var. = percentage of variance explained; ^aThe number of studies reflects the number of independent samples.

Separate ES were calculated for study samples that used different versions of the therapist alliance measure (i.e., some measures assessed the therapist’s rating of the child’s perception of the alliance, whereas other measures assessed the therapist’s perception of the alliance). The overall ES for the difference based on the therapist version assessing the child’s perception was d = 0.351, 95% CI = 0.203, 0.500, p < .001, based on 28 independent samples, with 65 ESs. The overall ES for the difference based on the therapist version assessing the therapist’s perception was d = 0.347, 95% CI = 0.123, 0.570, p = .004, based on 19 independent samples, including 27 ESs.

Twenty-three studies (including 23 independent samples and 47 ESs) reported on parent and therapist alliance ratings of 4,076 parents and therapists. A significant mean overall effect size was found (d = 0.715, 95% CI = 0.527, 0.903, p < .001). This indicates a moderate to large difference between raters, showing that parents reported higher alliance ratings than their therapists. Thirteen studies consisting of 18 independent samples reporting 31 ESs presented means and standard deviations of child and parent alliance ratings of 2,176 children and parents. The mean overall effect size was significant (d = −0.406, 95% CI = −0.569, −0.243, p < .001), indicating a small difference between raters. This finding suggests that parents reported higher alliance ratings of the parent-therapist alliance compared to child ratings of the child-therapist alliance. Forest plots of all associations are depicted in Appendix C.

Meta-analyses on Associations between Alliance Ratings

Next, we examined separate associations between child, parent, therapist, and observer alliance ratings (see Table 2). We found a significant moderate mean overall effect size (r = .321, 95% CI = .275, .365, p < .001) for the correlation between child and therapist alliance ratings of 3,316 dyads (43 independent samples, 59 ESs), indicating a small to moderate association between child and therapist alliance ratings. As noted previously, the therapist version of the alliance measure differed across studies. The overall ES for the therapist measure assessing the child’s perception was r = .295, 95% CI = .237, .350, p < .001, based on 26 independent samples and 36 ESs. The overall ES for the therapist measure assessing the therapist’s perception was r = .310, 95% CI = .244, .375, p = .011, based on 17 independent samples reporting 23 ESs.

Table 2. Results for the overall mean effect sizes for the association between child, parent, therapist, and observer ratings.

Type of association	# studies^a	# es	mean r (se)	95% ci	p	σ^2level2	σ^2level3	% var. level 1	% var. level 2	% var. level 3
Child ratings – therapist ratings	43	59	.321 (0.025)	.275, .365	<.001***	0.019**	0.000	42.2	57.8	0.0
Parent ratings – therapist ratings	18	25	.225 (0.034)	.158, .290	<.001***	0.003	0.009	49.0	13.1	37.9
Child ratings – parent ratings	16	27	.256 (0.037)	.184, .327	<.001***	0.013	0.000	58.5	41.5	0.0
Child ratings – observer ratings	15	19	.425 (0.071)	.296, .539	<.001***	0.000	0.049*	36.7	0.0	63.3
Therapist ratings – observer ratings	7	10	.533 (0.062)	.426, .626	<.001***	0.000	0.000	99.9	0.0	0.1
Observer ratings parent alliance – observer ratings child alliance	5	5	.183 (0.135)	−.186, .507	.240	0.026	0.026	39.8	30.1	30.1

ci = confidence interval; es = effect size; σ²_level2 = variance between effect sizes (within studies); σ²_level3 = variance between effect sizes (between studies); % var. = percentage of variance explained; ^aThe number of studies reflects the number of independent samples.

Further, an overall significant effect size (r = .225, 95% CI = .158, .290, p < .001) was found for the correlation between parent and therapist alliance ratings of 1,719 dyads (18 independent samples, 25 ESs), indicating a small to moderate association between parent and therapist alliance ratings. Also, an overall significant effect size (r = .256, 95% CI = .184, .327, p < .001) was found for the correlation between child ratings of the child-therapist alliance and parent ratings of the parent-therapist alliance of 1,127 dyads (16 independent samples, 27 ESs), indicating a small to moderate association between child and parent ratings of the alliance with their therapist.

Further, correlations with observer ratings were investigated. We examined correlations between child and observer ratings of the alliance (15 independent samples, 19 ESs). The mean overall estimate was significant (r = .425, 95% CI = .296, .539, p < .001), indicating a moderate to large association between child ratings and observer ratings of the child-therapist alliance. Nine studies specifically focused on the association between the TASC (Shirk & Saiz, 1992) and TPOCS-A (McLeod & Weisz, 2005). The mean effect size of these studies was significant (r = .343, 95% CI = .187, .482, p < .001). Six studies (seven independent samples, reporting 10 ESs) presented correlations between therapist and observer ratings of the child-therapist alliance. The mean overall estimate was significant (r = .533, 95% CI = .426, .626, p < .001), indicating a large association between therapist and observer ratings of the child-therapist alliance. Finally, a non-significant effect size (r = .183, 95% CI = −.186, .507, p = .240) was found for the association between observer rated child-therapist alliance and observer rated parent-therapist alliance (5 studies reporting on 5 ESs).

Moderator Analyses

The results of the overall effect size estimates of differences between alliance ratings, associations between child-therapist alliance ratings, and observer and child ratings showed that there was significant variability in effect sizes within studies (level 2), as well as between studies (level 3). This variability stressed the need for moderator analyses to explain this variance. We considered various moderators related to study characteristics, sample characteristics, alliance characteristics, and treatment characteristics (see Appendix D, Tables D1–D5).

Moderator Analyses on Differences between Alliance Ratings

Moderator analyses on the difference between child and therapist alliance ratings (Table D1) indicated no significant moderating effects for study characteristics, sample characteristics, alliance characteristics, and treatment characteristics. Given that the therapist version of the alliance measure assessed either the child’s or therapist’s perception of the alliance, moderator analyses were conducted for separate study samples. Results of moderator analyses for both types of therapist measures were also non-significant. Moderator analyses on the difference between parent and therapist ratings (Table D2) showed that the reliability of the alliance measure of the therapist was a significant moderator (β¹ = 4.30; F(1, 41) = 6.69, p = .013), indicating larger differences between parent and therapist ratings when the reliability coefficient of the therapist measure was larger. No significant moderating effects were found for study characteristics, sample characteristics, and treatment characteristics. Moderator analyses on the difference between child and parent alliance ratings (Table D3) indicated that reliability of the alliance measure of the parent was a significant moderator (β¹ = 3.82; F(1, 17) = 8.58, p = .009), indicating larger differences between parent and child ratings when the reliability coefficient of the parent measure was larger. No significant moderating effects were found for study characteristics, sample characteristics, and treatment characteristics.

Moderator Analyses on Associations between Alliance Ratings

Moderator analyses on the association between child and therapist alliance ratings (Table D4) revealed that manualized treatment was a significant moderator, showing larger associations between child and therapist ratings for manualized (r = .37) than for non-manualized treatment (r = .27; F(1, 57) = 6.17, p = .016). Also, larger associations between child and therapist ratings were found for CBT (r = .38) compared to non-CBT (r = .28; F(1, 57) = 4.50, p = .038). Again, due to the different therapist versions of the alliance measure assessing either the child’s or therapist’s perception of the alliance, moderator analyses were conducted for separate study samples. Results indicated that for studies that used a therapist measure that assessed the therapist’s rating of the child’s perception of the alliance, stronger associations were found between child and therapist ratings for manualized (r = .38) than for non-manualized treatment (r = .21; F(1, 34) = 9.89, p = .003). Also, stronger associations were found between child and therapist ratings for CBT (r = .39) compared to non-CBT (r = .24; F(1, 34) = 6.42, p = .016). For the study sample in which the therapist measure assessed the therapist’s own perception of the alliance, these moderators analyses were non-significant. However, reliability of the therapist measure proved to be a significant moderator (β¹ = 1.69; F(1, 16) = 5.16, p = .037). No significant moderating effects were found for study characteristics and sample characteristics. For the association between child ratings and observer ratings of the child-therapist alliance (Table D5), a significant moderating effect was found for age category, indicating larger associations between child and observer alliance ratings for children aged 13 years and older (r = .61) compared to children aged 12 and younger (r = .36; F(1, 17) = 4.59, p = .047). No significant moderating effects were found for study characteristics and treatment characteristics.

Publication Bias

Three methods were applied to address publication bias. First, we used extended versions of Egger regression (Egger et al., 1997) and the funnel plot test adapted from Fernández-Castilla et al. (2021) to test the degree of funnel plot asymmetry. We also conducted trim-and-fill analyses (Duval & Tweedie, 2000; Fernández-Castilla et al., 2021). The z-statistics and p-values of the Egger regression tests, funnel plot tests, and trim-and-fill analyses for all meta-analyses are depicted in Table 3. Funnel plots of all associations are depicted in Appendix E. For each meta-analysis, a funnel plot of all ESs is presented (Figure a) and a funnel plot of study effects (Figure b).

Table 3. Results for publication bias analyses.

				Trim-and-fill analyses
Type of association	# k (# es)	Funnel plot test	Egger test	# es imp. l	# es imp. r	# es > cutoff
Child ratings – therapist ratings (d)	47 (92)	z = 0.01, p = .99	z = 0.56, p = .58	8	3	Yes
Parent ratings – therapist ratings (d)	23 (47)	z = 0.71, p = .48	z = −0.79, p = .43	3	0	No
Child ratings – parent ratings (d)	18 (31)	z = 1.56, p = .12	z = −1.39, p = .16	2	0	No
Child ratings – therapist ratings (r)	43 (59)	z = 0.55, p = .58	z = 0.21, p = .83	0	1	No
Parent ratings – therapist ratings (r)	18 (25)	z = −0.55, p = .58	z = 0.79, p = .43	0	0	No
Child ratings – parent ratings (r)	16 (27)	z = 0.00, p = 1.00	z = −1.53, p = .13	0	0	No
Child ratings – observer ratings (r)	15 (19)	z = 0.20, p = .84	z = −0.03, p = .98	0	0	No
Therapist ratings – observer ratings (r)	7 (10)	z = −1.65, p = .10	z = 1.55, p = .12	3	0	Yes
Observer ratings parent alliance – observer ratings child alliance (r)	5 (5)	z = −0.72, p = .47	z = 0.52, p = .60	0	0	No

#k = number of studies; #es = number of effect sizes; imp. l = number of effect sizes on the left side of the funnel plot distribution; imp. r. = number of effect sizes on the right side of the funnel plot distribution; #es > cutoff = the number of imputed es on the left side in relation to the cutoff value of the estimator of the trim-and-fill method proposed by Fernández-Castilla et al. (2021). this estimate is based on the population es and power (number of effect sizes).

The results of both the Egger regression analyses and funnel plot tests revealed no indications of funnel plot asymmetry. Results of the trim-and-fill procedures indicated that for several associations, effect sizes had to be imputed on the left side of the plot. Based on the guidelines by Fernández-Castilla et al. (2021), the number of imputed effect sizes on the left side of the plot exceeded the cutoff value for the difference between child and therapist alliance ratings (8 effect sizes) and the association between therapist ratings and observer ratings (3 effect sizes). These findings indicate that the magnitude of these meta-analytic associations might present some overestimation of the true effect size, possibly indicating presence of publication bias.

Discussion

A series of meta-analyses was conducted to examine differences and associations between alliance ratings by different informants in child and adolescent psychotherapy. We found that children and parents in general rated the alliance more positively than their therapists, and that associations between child, therapist, and parent alliance ratings were small to moderate. The associations between child and observer ratings of the child-therapist alliance as well as the therapist and observer ratings of the child-therapist alliance were moderate to large. Several moderators were identified. Associations between child and therapist alliance ratings were stronger for CBT versus non-CBT and for manualized treatment versus non-manualized treatment, but only if the therapist version of the alliance measure assessed the therapist’s rating of the child’s perception of the alliance. Also, reliability of therapist and parent alliance measures moderated various associations, indicating larger differences or stronger associations between alliance scores when the reliability of the measure was higher. Further, the association between child alliance ratings and observer ratings was stronger for adolescents than for children.

The findings on the difference between child and therapist alliance ratings as well as the difference between parent and therapist ratings were in line with our expectations. Previous studies on differences between alliance ratings in adult psychotherapy have also found that clients rate the alliance more positively than their therapists (Tryon et al., 2007, 2008). In their meta-analysis, Tryon et al. (2007) found a moderate difference between adult client and therapist alliance ratings (d = 0.63). Our findings yielded smaller differences between child and therapist alliance ratings (d = 0.35; 95% CI: 0.22, 0.47), and larger differences between parent-therapist ratings (d = 0.72; 95% CI: 0.53, 0.90). The higher alliance ratings of children and parents compared to their therapist could be explained by their frame of reference. Clients normally have no prior experiences with therapists, and may compare the nature of their interaction with the therapist to their interaction with friends or family members. When clients experience their interaction with their therapist as positive and helpful, meeting their expectations and therapeutic needs, this would result in a positive rating of the alliance. Therapists, on the other hand, may compare the interaction with the client to their experiences with previous clients (Hartmann et al., 2015; Tryon et al., 2007). In child psychotherapy, children and adolescents may evaluate their interaction with the therapist against their parents or other supportive adults, with whom they often disagree on the nature of the problem (Garland et al., 2004; Hawley & Weisz, 2003). In some cases, children and their parents may have had prior experiences with mental health care professionals, and may compare the interaction with their current therapist to previous therapists, which may also result in a positive rating of the alliance with their current therapist. However, to our knowledge there is currently no empirical evidence to fully justify this explanation.

Parents reported more positively on the alliance with the therapist than their children. Further, studies in which both parent-therapist and child-therapist alliance scores were reported (based on the same measure) also showed higher parent ratings of the alliance with the therapist than child ratings. An explanation for these findings may be that in child and adolescent therapy, parents often initiate treatment and may have had several contacts with the therapist before the start of treatment, which parents may have experienced as collaborative. Also, it has been proposed that, in conjoint family treatment, the therapist may be more focused on the alliance with parents as a result of previous interactions with the parent, which may result in the relatively positive evaluation of the alliance by parents (Welmers-Van de Poll et al., 2020).

Recent studies indicate that split alliances in conjoint family therapy (i.e., discrepancies between alliance ratings among family members with the therapist) may result in poorer treatment outcomes (Escudero et al., 2021; Friedlander et al., 2018). However, the number of studies on alliance discrepancies between child, parent, and therapist ratings in child and adolescent psychotherapy and conjoint family therapy is relatively small, and more studies are needed to understand how the alliance in therapy with children and their parents develops, and what factors contribute to the occurrence of alliance discrepancies in client-therapist dyads and split alliances in the context of family therapy.

The correlation between child and therapist alliance ratings found in the present study (r = .32; 95% CI: .28, .37) is comparable to a previous meta-analysis in adult populations (r = .36, Tryon et al., 2007), whereas the effect size for the association between parent and therapist ratings was somewhat smaller (r = .23; 95% CI: .16, .29). The significant associations between child, parent, and therapist alliance ratings indicate that they have a shared perspective on their alliance to some extent. However, given that the correlation was small to moderate, the perspectives of the different raters are not fully congruent. Given that children, parents, and therapists are part of the same collaborative relationship, it is plausible to assume that correlations between ratings could have been stronger (and differences smaller). However, research on cross-informant assessments in mental health care has consistently shown small to moderate associations between different informants (De Los Reyes et al., 2015; De Los Reyes & Ohannessian, 2016; De Los Reyes et al., 2013). Although it has been proposed that cross-informant ratings are higher for psychological phenomena that are measured in the same context, when the construct is relatively easy to observe, and when measured with a continuous scale (De Los Reyes & Ohannessian, 2016), cross-informant associations for the alliance with children and parents proved to be small to moderate. Another methodological complexity in alliance research is that child ratings of the alliance often show a restricted range in scores in the upper end of the scale, also referred to as ‘ceiling effect’ (Shirk et al., 2010). The restricted range of alliance scores could also be an explanation for the small to moderate associations between child, parent, and therapist ratings.

Another explanation for the moderate degree of convergence between child, parent, and therapist alliance ratings could be that there are fundamental differences in how children, parents, and therapists perceive their alliance with each other, and that the alliance measures currently used do not capture all aspects of the alliance that are valued by children, parents, and therapists. The child-therapist and parent-therapist alliance have mainly been defined according to the pantheoretical concept of alliance by Bordin (1979), consisting of the dimensions personal bond, agreement on tasks, and agreement on goals. Literature on the conceptual understanding of the alliance construct in therapy with children and their parents is scarce. A recent study by Ryan et al. (2021) investigated how children, parents, and therapists perceived the alliance using qualitative methods. The results indicated that participants had different views on several aspects of the alliance, such as the nature of the personal bond, which therapeutic techniques were important, and even the role of the parent in therapy. Other qualitative studies have found that trust, kindness, doing activities together, transparency (e.g., sharing information, open communication), and shared decision making are highly valued among children and parents (Baylis et al., 2011; Crom et al., 2020; Nooteboom et al., 2020; Nuñez et al., 2021). These findings illustrate the complexity and contextuality of the alliance in child therapy compared to individual adult psychotherapy.

It has to be noted that there are differences between the therapist versions of alliance measures assessing the child-therapist alliance, in that some measures assess the therapist’s own perception of the alliance with the child, whereas other measures assess the therapist rating of the child’s perspective of alliance instead of the therapist’s own perspective. Assessment of the therapist’s rating of his or her own perspective of the alliance leads to different conclusions than the assessment of the therapist’s rating of the child’s perspective. For instance, if the therapist’s rating of the child’s perspective of alliance is lower than the client’s rating, this might indicate that the therapist underestimates the client’s value of the alliance, and discrepancies between child-therapist alliance scores may reflect that the child is not being accurately understood by the therapist. However, it is not an indication of how the therapist values the alliance with the client. The differences in therapist versions was only present in studies that used the TASC, as there are several versions of this measure in use. The majority of studies did report clearly which therapist version of the TASC was used, which allowed us to examine the distinction between therapist alliance measures in our meta-analyses.

We tested whether the association between child and therapist ratings was stronger for studies using a therapist version of a measure that assessed the child’s perspective of alliance, but results were non-significant for the overall ES estimate. Moderator analyses for separate study samples revealed that CBT and manualized treatment were significant moderators of the associations between child and therapist alliance ratings, indicating stronger associations compared to non-CBT and non-manualized treatment. Of note, this moderating effect was only found for studies in which the therapist measure assessed the alliance from the perspective of the child. A possible explanation for this stronger association is that manualized treatment is highly structured and aims to clarify the trajectory of treatment for children, particularly in the early phase of treatment. Children may respond positively to this approach, and if recognized by the therapist, possibly resulting in a higher degree of convergence of alliance scores. The use of treatment manuals in psychotherapy has been criticized to potentially hinder the establishment of a therapeutic alliance, although empirical studies have repeatedly found no differences in strength of the child-therapist alliance between manualized treatments, mostly CBT, and treatment as usual (Langer et al., 2011; Ormhaug et al., 2014).

Our results show some empirical support for convergent validity of alliance measurements. The associations between child self-report ratings and observer ratings (r = .43; 95% CI: .30, .54) and therapist and observer ratings (r = .53; 95% CI: .43, .63) of the child-therapist alliance showed moderate to strong associations between these perspectives. This finding underlines that use of observer alliance measures in addition to self-report measures in alliance research is important to overcome measurement artifacts, such as a restricted range of self-report measures or relying on single-source ratings of alliance and outcome measures. Specifically, common rater variance (i.e., common method variance) is widely recognized as problematic in alliance studies (Kazdin, 2005; Kazdin et al., 2006). Age category proved to be a significant moderator of the association between child alliance ratings and observer ratings, indicating stronger associations for adolescents (age 13 and older) compared to children. Although the study samples were relatively small (four adolescent studies compared to 11 child studies), this finding may be of interest in future research on self-report and observer alliance measures for children. It has been proposed that measuring alliance in young children is particularly difficult due to age-limited cognitive abilities and limited understanding of the alliance process in therapy, which could be an explanation for self-report alliance ratings of children failing to converge with alliance measures of other informants (McLeod et al., 2017).

Empirical research on divergence or convergence of the alliance pertains to examining absolute or relative differences in alliances scores, and in general does not make a sharp theoretical distinction between alliance discrepancies and agreement based on different operationalizations of what may be distinctive of a weak (‘bad’) or strong (‘good’) alliance, respectively. Notably, we know from, for example, attachment research (Zeegers et al., 2017) and studies of psychological phenomena in general (Baumeister et al., 2001) that the influence of a bad relationship is greater than that of a good relationship, but it seems unlikely that the meaning of a weak (‘bad’) or strong ('good') alliance is fully captured by absolute or relative differences or similarities in alliance scores. This may have consequences for studies that examine associations between divergence or convergence of alliances and youth outcomes, including the strength of these associations. Notably, a negative or positive alliance encompasses supposedly more than quantitative differences or similarities in alliance scores. We therefore argue that alliance research may need to partly shift its focus to comparing alliances of children, parents, and therapists in terms of distinctive qualitative discrepancies and similarities between child, parent, and therapist perspectives of the alliance, because examining quantitative differences in alliance scores only tells part of the story.

Limitations

Studies on the alliance have increasingly focused on assessment of the alliance as a dyadic construct, measuring both client and therapist alliance ratings as opposed to assessing the alliance from a single perspective. Although alliance research in child and adolescent psychotherapy has been found to lag behind the alliance literature in adults (Karver et al., 2019), it is interesting to note that alliance studies in children and adolescents have also begun to examine the alliance as a dyadic construct (e.g., Fjermestad et al., 2016; Van Benthem et al., 2020; Zandberg et al., 2015). Future studies using assessments of alliance from multiple perspectives (i.e., child, parent, therapist) should explore whether the alliance can be operationalized as a product of these multiple perspectives and studied as a dyadic or systemic construct.

An important methodological limitation of the present study is that the analyses regarding the association between observer ratings and therapist ratings of the child-therapist alliance contained few studies, which makes it difficult to draw a valid and reliable conclusion from the results of these analyses. Further, only one study in the present meta-analysis used parent self-report alliance ratings and an observer measure of the parent-therapist alliance, suggesting that more research on the parent-therapist alliance using both parent self-report and observer measures is warranted. Another limitation is that some categorical moderators contained relatively few studies, which possibly resulted in insufficient statistical power to detect significant effects. Studies on youth with externalizing problems, substance use problems, and treatment in home-based or community settings were underrepresented in the present meta-analysis, which limits the possibility of generalization to these groups. Also, testing multiple moderators can result in chance capitalization. Therefore, effects that are just significant should be interpreted with caution.

To investigate the robustness of our findings, we applied several methods to address publication bias. It should be noted that studies included in the present meta-analysis did not focus on associations or differences between alliance ratings as a specific study aim, but as preliminary analyses. It could therefore be argued that publication bias or selective reporting in a strict sense cannot bias results of the present meta-analysis. However, in any meta-analysis, publication or selection bias analyses are important to examine whether the overall effect size estimates can be affected by particular research or publication tendencies in the scientific community that may affect (even unanticipated) study outcomes. The Egger regression test indicated no presence of publication bias and the result of the funnel plot test indicated no funnel plot asymmetry. However, the trim-and-fill procedure indicated possible presence of publication bias for two associations. It must be noted that every method has its limitations, therefore we used multiple methods to examine potential presence of publication bias. Moreover, for multilevel meta-analysis, specific methods to take into account dependency of effect sizes in the assessment of publication bias are still under development (Assink & Wibbelink, 2016; Fernández-Castilla et al., 2021; Rodgers & Pustejovsky, 2021).

Despite these limitations, several strengths can be noted. To the best of our knowledge, this is the first meta-analytic study on the alliance in child and adolescent psychotherapy that examined divergence and convergence of child, parent, therapist, and observer alliance ratings. We used an advanced method of multilevel meta-analysis, through which all available effect sizes of included studies could be used, increasing statistical power for moderator analyses. Consequently, the present meta-analysis provides an elaborate picture of the current state of alliance research in child and adolescent psychotherapy specifically focusing on differences and associations between alliance raters.

Implications for Future Research and Clinical Practice

Several implications for future research should be considered. First, at a conceptual level, research on the alliance with children could benefit from investigations into the conceptual understanding of the alliance by children compared to parents and therapists. Given our findings that children and parents, on average, rate the alliance more positively than their therapists, this may indicate that alliance scores between a client and therapist may reflect different interpretations of the alliance construct (Horvath, 2018; Tryon et al., 2007).

Different types of alliances may need different definitions (or defining elements) of the alliance construct. Youth, parent, and therapist perspectives on the operationalization of the alliance may also depend on the treatment setting, as individual therapy in an out-patient setting is very different compared to home-based treatment or inpatient treatment. A clear understanding of how children, parents, and therapists perceive the alliance could contribute to our knowledge on how to measure these types of alliances, how alliances develop during treatment and how they are related to therapy process factors and therapeutic outcomes.

Qualitative research in this area is scarce relative to the number of quantitative studies on the alliance with children, adolescents, and parents. Recently, several qualitative studies have been conducted specifically focusing on the alliance (e.g., Altimir et al., 2017; Gilson & Abela, 2021; Nooteboom et al., 2020; Nuñez et al., 2022, 2021; Ryan et al., 2021), which may indicate a renewed interest in understanding how children and parents experience the alliance. More rigorous qualitative studies as well as qualitative syntheses of this literature are needed to advance knowledge on child, parent, and therapist perceptions of the alliance. Additionally, a qualitative inquiry into how divergence between perceptions of the alliance by children, parents, and therapists are related to building and monitoring the alliance, alliance rupture-repair processes, and the process and outcome of therapy may advance alliance research in child and adolescent therapy.

The parent’s perception of the child-therapist alliance has not yet received much attention in research. For instance, it is important to investigate which elements of the alliance are valued by parents with regard to their own alliance with the therapist and the alliance between their child and the therapist. Also, the therapist’s perception of the alliance could be better understood by addressing the therapist’s own perception of the alliance in addition to their perception of the child’s (or parent’s) perspective of the alliance.

Second, alliance discrepancies and tensions or ruptures in the alliance seem relatively unexplored territories in alliance research in child populations (Baillargeon et al., 2012). Valid and reliable measurement of disagreement on and negative perceptions of the alliance by child, parent, and therapist could also advance alliance research in child psychotherapy. Third, recent studies on the alliance in adult psychotherapy have focused on the alliance at a dyadic level, taking into account the interdependency of alliance ratings between client and therapist, as well as within and between person variance (Friedlander et al., 2012; Kivlighan, 2007; Zilcha-Mano et al., 2017). These relatively new analytic approaches in alliance research should be applied to child and adolescent psychotherapy in order to better understand the child-therapist and parent-therapist alliance as an interpersonal process.

Our findings have some implications for clinical practice as well. Our findings show that associations between child, parent, and therapist alliance ratings are only small to moderate, which underlines the importance of acknowledging the various perspectives on alliance when dealing with children and parents in therapy. Also, it is widely recognized that the alliance is a dynamic interpersonal process, and that positive alliances in child and adolescent therapy contribute to engagement in treatment and positive treatment outcomes (Karver et al., 2019). Collecting client feedback on the alliance through child and parent self-report measures is seen as an important method to increase treatment efficacy (Duncan et al., 2010; Rober et al., 2020), and alliance measures are frequently used in measurement-based care (MBC; Connors et al., 2021; Lambert et al., 2018; Tam & Ronan, 2017). Although MBC has been found to be difficult to implement and possibly burdensome for practitioners and clients (Connors et al., 2021; Van Sonsbeek et al., 2021), routine evaluation of the alliance seems feasible in this regard (either by using formal assessments or through ongoing informal discussion), because it relates to the direct interaction between participants in therapy.

Enabling children and parents to give feedback on the alliance with their therapist and to actively discuss and reflect upon this process could provide therapists and their clients insight into how children and parents perceive and experience the alliance. This may be especially important in the early stages of treatment, during which a positive alliance may contribute to positive treatment response (and positive treatment response may subsequently contribute to positive alliance, Flückiger et al., 2020). It is therefore an important task of the therapist to establish a working framework early in treatment, within which the alliance in terms of collaboration and building rapport can be routinely discussed throughout treatment. In case of discrepancies between child, parent, and therapist perceptions of the alliance, these may be explored through joint reflection, which could be therapeutic in itself. Also, monitoring and discussing the alliance among all participants in therapy could prove helpful to deal with strengths and difficulties during the therapeutic process, and to prevent strains and ruptures in the alliance.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Supplementary Material

Supplemental material for this article can be accessed online at https://doi.org/10.1080/15374416.2022.2093210

Additional information

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.