It's the therapist and the treatment: The structure of common therapeutic relationship factors

Abstract

Objective: Prior research has established that common therapeutic relationship factors are potent predictors of change in psychotherapy, but such factors are typically studied one at a time and their underlying structure when studied simultaneously is not clear. We assembled empirically validated relationship factors (e.g., therapist empathy; patient expectations; agreement about goals) into a single instrument and subjected it to factor analysis. Method: The instrument was applied to patients (N = 332) undergoing intensive psychotherapy of different types for depressive disorders, anxiety disorders, eating disorders, and childhood trauma in an inpatient specialized mental health setting. In order to examine the psychometric properties of the scale, we used half the sample (N=164) to conduct exploratory factor analysis (EFA) and parallel analysis before we tested the solution using exploratory structural equation modeling (ESEM) on the second half of the sample (N=168). Measurement invariance analysis was conducted to examine the stability of the factor structure. Results: The analysis yielded two factors, which were termed 1. “Confidence in the therapist” and 2. “Confidence in the treatment.” Discussion: When assessed simultaneously, patients differentiate between their evaluation of the therapist and of the treatment. The results indicate that there is substantial overlap among previously established relationship factors.

Trial registration: ClinicalTrials.gov identifier: NCT03503981.

Key words:

• psychotherapy
• common factors
• therapeutic relationship
• factor analysis
• ESEM

Clinical or methodological significance of this article: We collected items reflecting empirically validated common relationship factors (such as treatment goals and therapist empathy) and constructed a patient-rated measurement scale to simultaneously investigate these items. We then assessed the underlying structure of this newly developed scale. The results suggest that patients differentiate between their (1). confidence in the therapist, and their (2). confidence in the treatment. The study also highlights that many of the constructs we measure in psychotherapy are dependent on each other.

Psychotherapy is typically seen as composed of specific ingredients, such as interventions associated with particular therapeutic models, and non-specific, common factors that exist across models (e.g., Lambert & Bergin, 1994). The concept of common factors was first mentioned by Rosenzweig in his seminal 1936 article, and the idea was further popularized by Frank (Frank, 1961). Their idea was that the common factors were likely responsible for much of the beneficial effect of psychotherapy. Later empirical investigations have supported this claim (e.g., Imel & Wampold, 2008; Lambert & Barley, 2002; Wampold, 2015a).

Over the years, many different common factors have been proposed, leading to a proliferation of such factors, giving rise to the need to classify them. Indeed, there have been several attempts at classifying the common factors (e.g., Frank, 1982; Goldfried, 1980; Karasu, 1986; Marmor, 1976; Orlinsky & Howard, 1987; Rosenzweig, 1936; Wampold & Imel, 2015b). Different conceptualizations and studies have proposed varying numbers of factors, from one to 89 distinctly different common factors. This assortment of factors can be explained in several ways. First, different researchers use different levels of conceptualization (Grencavage & Norcross, 1990), including technical interventions (such as interpretation and exposure) and assumed mechanisms of change (such as catharsis and desensitization) (e.g., Lambert & Bergin, 1994). Others include factors at a more abstract and general level (such as client or therapist level; Bromberg, 1962; Hynan, 1981). As well, large number of factors might exist because different terms have been used for similar constructs (such as “warmth” vs “empathy,” “alliance” vs “goal collaboration,” “expectation vs hope,” and “catharsis” vs “emotional processing”).

Typically, taxonomies of common factor use heuristically derived categories and comprehensive models of how the common factors produce benefits are rare, with some exceptions (e.g., Frank & Frank, 1993; Garfield, 1995; Grawe, 2004; Tschacher et al., 2014). One model which seeks to explain how therapeutic factors lead to change, is the contextual model, which is a meta-theory explaining how all psychotherapies produce therapeutic results (Wampold & Imel, 2015b). The model describes three ways in which the therapeutic relationship works to produce change: First, benefits accrue from an empathic, trusting relationship (sometimes referred to as the Rogerian conditions); the real relationship, or epistemic trust. Second, benefits derive in part from the patient's beliefs in the treatment and the concomitant therapeutic actions (i.e., the tasks of therapy), primarily through expectations that the treatment is an effective means to achieve therapeutic goals. Third, benefits are produced through the specific ingredients, which induce the patient to engage in healthy actions. The contextual model is based on evolved characteristics of humans healing in a social context and is supported by research evidence from randomized clinical trials and psychotherapy process research (Wampold & Imel, 2015b).

In their comprehensive investigation of effective factors pertaining to the psychotherapeutic relationship, Norcross and Lambert (2018) concluded that the working alliance, collaboration, goal consensus, positive regard and affirmation, therapist empathy and using feedback, are demonstrably effective. Further, treatment credibility, congruence/genuineness, real relationship, emotional expression, cultivating positive expectations, managing countertransference, repairing alliance ruptures are evaluated as “probably effective.” Recent meta-analyses on alliance, empathy, and expectations support the notion that these factors are central components of change (see e.g., Constantino et al., 2018; Elliott et al., 2018; Flückiger et al., 2018; Norcross & Lambert, 2018).

To sum, even though decades of psychotherapy research indicate the presence of some promising common factors, how they are organized and how they work, is not yet clear. Norcross and Lambert (2018) noted that “too few studies exist to allow meta-analytic reviews of multiple relationship elements (e.g., measures of the therapeutic alliance, therapist empathy, and client expectations for improvement)” (p. 312). Traditionally, the common factors are typically examined in isolation. This is an issue, because as Norcross and Lambert (2018) noted, the overlap among the factors is bound to be substantial and their contribution to therapeutic progress should be investigated simultaneously to avoid an overestimation of their individual importance. Thus, an important step toward an empirically validated, comprehensive theory of the common factors and their relationship to outcome is to assess the structure of the common factors; that is, what are the latent factors that underlie the various factors proposed and discussed in the literature.

There exists a wide range of instruments that measure different common factors. However, many of these suggest a high level of interconnectedness and complexity of common factors structures. As an illustration of the complexity, several studies have found that theoretically proposed models for common factor structures did not necessarily correspond to the factor structure found in analyses of patients’ reports (e.g., Elvins & Green, 2008; Hatcher & Barends, 1996). For instance, Agnew-Davies et al. (1998) investigated the factor structure of the Agnew Relationship Measure (ARM). Interestingly, they found that the factor structure identified by factor analysis of patient reports did not match the conceptual ARM model proposed by the authors. Hatcher and Barends (1996), who studied three measures of the therapeutic alliance, have also reported similar results. Other instruments have shown good psychometric qualities, such as The Bern Post Session Report (Flückiger et al., 2010) and the scale for the Multiperspective Assessment of General Change Mechanisms in Psychotherapy; the MULTI (Mander et al., 2013). However, these studies approached the phenomenon of common factors from the theoretical standpoint of Klaus Grawe (2004), and so did do not correspond with the contextual model nor the empirically supported psychotherapy relationship factors proposed by Norcross and Lambert (2018).

An important but often ignored issue that arises when instruments are administered repeatedly is whether the factor structure is stable over time (e.g., Falkenström et al., 2015). Such stability in factor structure over time is called longitudinal measurement invariance and is crucial for affirming validity and reliability of a scale (Vandenberg & Lance, 2000). If a scale does not achieve longitudinal measurement invariance, one cannot compare aggregate scores across time, as it would be an indication that the patientś interpretation of the factors or items change over time. For example, Falkenström et al. (2015) argued that without longitudinal measurement invariance, changes in means of the observed variables may not reflect changes in the latent construct, but rather changes in the way the patient's understand the item content. They further specify that the invariance of factor loadings and indicator intercepts are the most important types of invariance, as these are the parameters that represent the measurement part of a factor analysis model. If the factor loadings and intercepts vary over time, it is not possible to interpret changes in the latent factors. In short, longitudinal measurement invariance allows for examination of the stability of psychological constructs even though the ratings of these constructs may vary (i.e., patients may score empathy differently at the beginning or the end of treatment but the perception of the key features of empathy remains the same). In turn, this would suggest that the perception of common factors are independent of the phase of therapy, which would be a reflection of their robustness and ultimately their validity.

The objective of the present study was to evaluate an instrument covering central common relationship factors embedded in the contextual model (i.e., Wampold, 2001; Wampold & Imel, 2015b), which have received strong empirical support in regard to treatment outcomes, as demonstrated by Norcross and Lambert (2018). We first constructed a patient rated instrument that included vital relationship factors from existing common factor instruments. Based on patients’ responses, we examined the latent structure of this instrument using factor analysis. Finally, we investigated the stability of the factor structure over the course of therapy to investigate temporal stability of the factor structure.

Method

Design and Treatment Context

The patients in this study had been admitted to a public inpatient treatment facility offering specialized treatment for patients with depression, anxiety disorders, eating disorders or longstanding relational trauma (i.e., Modum Bad, Vikersund, Norway). Patients received treatment packages consisting of individual therapy and, in varying degree across units, group therapy. Patients were treated with different psychotherapeutic methods according to the policy at four treatment units, among them cognitive behavioral therapy, compassion-focused therapy, trauma-focused therapy, short-term psychodynamic therapy and metacognitive therapy. The duration of treatment varied across units from eight to 14 weeks. An important inclusion criterion is that the patient must have exhausted local treatment options. Patients were excluded if they had current suicidal risk, substance abuse that required immediate treatment, or if simultaneous treatment could interfere with the treatment. The project is registered with Clinicaltrials.gov (Identifier: NCT03503981).

Participants

Patients

Two inpatient samples, collected consecutively, were used in the factor analytic procedure; the first sample (sample 1) contained 164 patients and the second (sample 2) consisted of 168 patients. All patients were adults, 74% women and 26% men. The patients were examined by a clinical psychologist or a medical doctor who used standardized diagnostic instruments, such as the M.I.N.I (Sheehan et al., 1998) and SCID-II (Lobbestael et al., 2011). Among the patients, 23% presented with a primary anxiety disorder, 30% with a primary depressive disorder, 16% with a primary eating disorder, and 30% with a primary trauma disorder. The samples were characterized by a high degree of comorbidity, with 58% of the participants presenting with more than one clinical diagnosis.

Therapists

There were 67 (64.2% female) therapists included in this study. The therapist groups comprised psychologists (41.8%), clinical psychologists with specialization (i.e., minimum five years clinical experience, 20.9%) medical doctors under specialization (14.9%), psychiatrists (11.9%), clinical nurses and social workers with specialization in clinical work (6%), and psychology students in clinical training (4.5%). The therapists received regular supervision according to the treatment model offered at their unit. Their experience varied from being newly educated to several decades of clinical experience.

Measurement Development

The common factors questionnaire, which we termed MPOQ—Common Factor Scale (Modum Bad Research Institute, 2017) was constructed through several steps. First, a team of clinicians and researchers selected a large sample of items (i.e., 142 questions) that measure relationship factors which have been proposed as important treatment predictors (such as agreement on tasks and goals, therapist empathy, therapist expertise, patients’ hope and expectations, and treatment credibility, e.g., Norcross & Lambert, 2018; Wampold & Imel, 2015b). In all, 30 items from validated measurement scales were used, i.e., from the Working Alliance Inventory (Horvath & Greenberg, 1989), California Psychotherapy Alliance Scale (Gaston, 1991), the Real relationship Inventory (Kelley et al., 2010), the Barrett-Lennard Relationship Inventory (Barrett-Lennard, 1962), and the Credibility/Expectations questionnaire (Devilly & Borkovec, 2000). Items from existing instruments that did not exist in Norwegian (e.g., on treatment credibility) were translated. We conducted the translation via the conventional use of translation/back-translation. Bilingual members of the research staff provided a first translation from English to Norwegian. Then a bilingual researcher made a back translation to English. Throughout the process, language, grammar, and cultural discrepancies that might influence the interpretation of the questionnaire, were taken into account. All translators were also experts in psychotherapy.

Next, the preliminary questionnaire consisting of 30 items was administrated to a cohort of patients (n = 80). These items were all selected from the validated forms mentioned above, and the expertise questions (see below). Based on their original scales, the items measured (the patients’ evaluations of) agreement on goals and tasks, expectations, credibility, genuineness/empathy, and therapist expertise. The data from this administration were analyzed and the psychometric properties were evaluated using generalizability theory and factor analysis. The aim of these analyses was to optimize the measure and reduce the number of items without narrowing the construct domain (e.g., Brennan, 2011). While generalizability theory provides reliability estimates for different measurement designs (e.g., different number of items), factor analysis can point to the specific items that should be retained in a new format (Brennan, 2011). Further, a group of patients functioned as a focus group, providing feedback on the questionnaire.

Subsequently, the instrument was revised in accordance with results from the above-mentioned analyses, and feedback from the focus group. Decisions on which items and constructs to include in the final version were based on several factors. First, one goal was to attain a psychometrically sound instrument. To ensure this, we took into consideration the preliminary analysis so that we could balance item reduction while maintaining reliability in the generalizability analysis, and secondly, retain items with high factor loadings in a confirmatory factor analysis, while also ensuring acceptable content validity. Also, as a main strategy, we turned to already familiar/validated questionnaires when selecting items. Another goal was to attain an instrument that is clinically meaningful to therapists and patients, and to include only constructs that we were empirically promising (according to Norcross & Lambert, 2018). Therefore, we opened for including items from different relationship measurements (in accordance with Hatcher & Barends, 1996) and to include items we developed ourselves. Thus, the final revisions were informed by—and balanced between—feedback from patients, clinicians and statistical analyses.

Following this revision, the instrument was administrated to a new cohort of patients, and the present study is based on this sample. The final version included 14 items reflecting six different domains of common relationship factors; (1) agreement on tasks (2 items); (2) agreement on goals; (2 items); (3) therapist empathy (3 items), (4) therapist expertise (3 items), (5) treatment credibility (2 items) and 6. expectations (2 items). Of the 14 items, four items were developed by us (the authors) for the purpose of this questionnaire (item 6 and items 12-14). See Table I for items included in the final version. Patients rated their answers on a 1–7 Likert scale, where 1=“strongly disagree” and 7=“strongly agree.”

Table I. The MPOQ- Common factor questionnaire.

Scale	Item		Adapted from
Agreement on tasks	1	I think what we do in treatment will help me achieve the changes I want.	WAI (Horvath & Greenberg, 1989)
	2	What I do in treatment gives me new ways of understanding my problems.	WAI (Horvath & Greenberg, 1989)
Agreement on goals	3	My therapist understand what I hope to get out of therapy.	WAI (Horvath & Greenberg, 1989)
	4	My therapist and I are working towards mutually agreed upon goals.	WAI (Horvath & Greenberg, 1989)
Expectations	5	Overall, I expect much improvement as a result of treatment.	Attitudes and Expectations questionnaire (Mooney et al., 2014)
	6	If I do what's expected of me in therapy, I have a good chance to deal with my problems.	Developed by researchers at Modum Bad
Credibility	7	At this point, therapy seems logical to me.	The credibility/expectancy questionnaire (Devilly & Borkovec, 2000)
	8	I would recommend this treatment to a friend who has problems similar to mine.	The credibility/expectancy questionnaire (Devilly & Borkovec, 2000)
Empathy	9	My therapist is dedicated to help me overcome my difficulties.	CALPAS (Gaston, 1991)
	10	My therapist and I are able to have an open and honest relationship.	The Real Relationship Inventory (Kelley et al., 2010)
	11	I feel accepted and respected by my therapist.	CALPAS (Gaston, 1991)
Expertise	12	My therapist has the expertise to help me.	Developed by researchers at Modum Bad
	13	My therapist explains what we are doing in therapy in a way I understand.	Developed by researchers at Modum Bad
	14	My therapist explains my problems in a way I understand.	Developed by researchers at Modum Bad

Procedure

The patients were recruited at treatment commencement and gave written consent to participate in the study. The questionnaire was administered electronically to patients weekly throughout the duration of treatment (as a feedback system that was a part of their treatment). The data were collected from December 2017 until March 2020. The study was approved by the Regional Ethics Committee (REK) prior to data collection (REK number = 2017/2124).

Statistical Analyses

The data set consisted of two samples, allowing for exploratory examinations in sample 1, and confirmatory analyses in sample 2. All analyses were conducted on data from the first week of treatment, except for analysis on invariance where we used data from week 1 and week 6 to assess stability of the factor structure over time. Week 6 was chosen in order to include as many as possible of the patients in the sample, because patientś treatment vary in length. Due to the familiarity of the items and constructs, as most of them were collected from already validated questionnaires, the content validity of the questionnaire was judged satisfactory.

Sample 1

In order to determine the number of latent factors in the data, we conducted an exploratory factor analysis with maximum likelihood estimation and oblique rotation (promax). Sample characteristics, such as factor loadings, cross-loadings and explained variance were examined. To examine the internal consistency of the scales derived from the factor analysis, coefficient alphas were calculated for each factor. Further, we conducted a parallel analysis (Horn, 1965; O’Connor, 2000), which is recommended as it obtains a more accurate estimate of the numbers of factors to retain, compared to other methods, such as screen tests (Cattell, 1966) and the K1 (Kaiser, 1960). For the parallel analysis, we utilized the rawpar.sps script developed by O’Connor (2000).¹ One thousand datasets were generated based on permutations of the raw data, using a principal component approach. This procedure generates eigenvalues from the raw data along with the mean eigenvalues and eigenvalues representing the 95th percentile based on the Monte Carlo simulation. Factors are retained when the eigenvalue from the actual data set is higher than the eigenvalue from the randomly generated data set. All analyses were conducted in SPSS, version 25.

Sample 2

We wanted to test whether the factor structure from sample 1 would replicate in a different sample. Thus, we used exploratory structural equation modeling (ESEM; Asparouhov & Muthén, 2009; Marsh et al., 2014) in sample 2. ESEM is a statistical framework that combines features from exploratory and confirmatory factor analysis (EFA and CFA). We used ESEM because it handles some of the issues that have been noticed concerning CFA; for instance, due to more restrictive features of CFA, it has been difficult to replicate factor structures from exploratory analysis with CFA (see Morin et al., 2013). Further, when modeling psychological constructs one can argue for the use of more flexible models since these are more suited to capture complex psychological structures compared to standard CFA (Marsh et al., 2009). Using ESEM allows for items to cross-load as in EFA, while also allowing for typical CFA parameters and statistical advances, such as estimation of fit indices, standard error and tests of invariance (Morin et al., 2013). Thus, ESEM can be used as a confirmatory tool (Marsh et al., 2014). Here, ESEM was conducted by specifying the extraction of two factors and item loadings as freely estimated. All analyses of sample 2 were conducted in Mplus 8 with maximum likelihood estimator with robust estimation (MLR; Muthén & Muthén, 1998–2017). The factors were correlated using oblique geomin rotation (Muthén & Muthén, 1998–2017).

Goodness of fit of the factor model was assessed by means of chi square (χ²), comparative fit index (CFI), root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR; Schweizer, 2010). For model fit, we used the following criteria: cut-offs for acceptable and good model fit on the RMSEA were set to below .08 or .05 respectively (in line with Marsh et al., 2010). For CFI, we used cut-offs for acceptable and good model fit, i.e., ≤.90 and ≤.95, and for SRMR, values were expected to be below .08 (Hu & Bentler, 1999). The Akaike information criterion (AIC) was used to compare model fit between models.

As the common factor questionnaire was developed with the purpose of measuring common factors throughout the treatment process, invariance testing across time was conducted to test the stability of the selected model (e.g., Marsh et al., 2014). This provides additional evidence for the psychometric validity of the scales. Hence, we tested invariance across weeks 1 and 6, and for each invariance test (see below), we first tested model fit for the two groups. Next, we successively tested configural, weak, strong and strict invariance (Liu et al., 2017; Meredith, 1993; Meredith & Teresi, 2006). Assessing invariance was done utilizing the procedure outlined in Morin et al. (2013):

• (1). Configural invariance tests if the latent factors have the same pattern of fixed and free loadings across time (i.e., in our case, at session one and session six). The factor structures are estimated separately in each group, but the number of factors is set to be the same in both groups. Latent variances are fixed to 1 and latent means to 0 in both groups, thus allowing for free estimation of all factor loadings and intercepts.

• (2). Weak factorial invariance is conducted to check if each item contributes to the latent factors to a similar degree across time. This is tested by adding equality constraints on the factor loadings across groups, and by fixing factor variance to 1 in week 1, while freely estimating it in week 6.

• (3). Strong invariance tests whether the intercepts (in addition to factor loadings) are invariant from week 1 to week 6. It specifies whether individuals with the same score on a latent factor answer the items in a similar way. This assumption is tested by adding equivalence constraint on the item intercept at the two time points.

• (4). Strict factorial invariance is conducted to test whether residual variance (i.e. unique variance and error variance) is similar across time. This is tested by constraining the item residuals to be the same at the two time points.

The invariance analysis is conducted stepwise, and if for any step invariance is not established, further analysis is not conducted. To compare difference between models, comparative fit index (CFI) and root mean square error of approximation (RMSEA) were used (Chen, 2007). For CFI, a difference smaller or equal to .01 supports invariance (Chen, 2007; Cheung & Rensvold, 2002). For RMSEA, a difference smaller or equal to .015 supports invariance (Chen, 2007).

Results

Sample 1

See Table II for descriptive results including means, standard deviations, skewness and kurtosis for scores on the common relationship questionnaire from week 1 and 6.

Table II. Descriptive statistics for week 1 and 6.

	Week 1			Week 6
Item	M (SD)	Skewness (SD)	Kurtosis (SD)	M (SD)	Skewness (SD)	Kurtosis (SD)
1	5.92 (1.21)	−1.22 (.17)	1.68 (.34)	6.25 (1.06)	−2.25 (.23)	6.96 (.45)
2	5.92 (1.27)	−1.10 (.17)	0.83 (.34)	6.25 (1.10)	−1-88 (.23)	3.71 (.45)
3	5.50 (1.46)	−0.95 (.17)	0.47 (.34)	5.36 (1.13)	−0.52 (.23)	0.34 (.45)
4	5.68 (1.24)	−1.27 (.17)	2.02 (.34)	5.64 (0.98)	−0.81 (.23)	1.15 (.45)
5	5.94 (1.08)	−1.28 (.17)	2.34 (.34)	5.89 (1.10)	−1.24 (.23)	1.85 (.45)
6	6.16 (1.28)	−1.74 (.17)	3.11 (.34)	6.12 (1.29)	−1.81 (.23)	3.35 (.45)
7	5.80 (1.14)	−0.76 (.18)	0.17 (.35)	5.70 (1.05)	−1.13 (.23)	2.71 (.45)
8	5.99 (1.24)	−1.21 (.17)	0.89 (.34)	6.29 (1.09)	−1.82 (.23)	3.42 (.45)
9	6.36 (1.01)	−1.38 (.17)	0.51 (.34)	6.40 (1.04)	−1.96 (.23)	3.53 (.45)
10	6.40 (1.03)	−1.70 (.17)	2.02 (.34)	6.48 (0.91)	−2.19 (.23)	5.56 (.45)
11	6.02 (1.19)	−0.92 (.17)	−0.37 (.34)	6.25 (1.04)	−1.60 (.23)	2.47 (.45)
12	5.98 (1.26)	−1.32 (.18)	1.69 (.35)	6.21 (1.06)	−1.79 (.23)	3.80 (.45)
13	6.20 (1.13)	−1.32 (.18)	0.68 (.35)	6.43 (0.95)	−2.19 (.23)	5.44 (.45)
14	5.84 (1.18)	−0.63 (.17)	−0.67 (.34)	5.99 (1.04)	−1.13 (.23)	1.35 (.45)

The exploratory factor analysis suggested a two-factor structure (see factor loadings in Table III). Factor 1 comprised of eight items (α = .92), explained 48.4% of the variance, with factor loadings ranging from .68-.87. Factor 2 comprised of six items (α = .83), explained 13.9% of the variance, with factor loadings ranging from .38-.85. The correlation between the two factors was .53. As can be observed in Table IV, item 8 (“I would recommend this treatment to a friend who has similar problems to mine”) displayed low factor loading in the exploratory factor analysis (.38). Consequently, we decided to remove this item from the analysis in sample 2.

Table 3. Factor loadings from exploratory factor analysis (sample 1, week 1) and ESEM (sample 2, week 1), both with maximum likelihood estimation and oblique rotation

		EFA		ESEM
	Item	F1	F2	F1	F2
3	My therapist understand what I hope to get out of therapy	.78	.07	.75	.10
4	My therapist and I are working towards mutually agreed upon goals	.68	.08	.71	.09
10	My therapist and I are able to have an open and honest relationship	.77	.01	.85	-.03
11	I feel accepted and respected by my therapist	.83	.07	.89	-.05
9	My therapist is dedicated to helping me overcome my difficulties	.80	.11	.87	-.03
14	My therapist explains my problems in a way I understand	.87	.06	.81	.00
13	My therapist explains what we are going to do in therapy in a way I understand	.76	.05	.70	.13
12	My therapist has the expertise to help me	.74	.06	.77	.09
5	Overall, I expect much improvement as a result of treatment	.15	.84	-.11	.81
6	If I do what's expected of me in therapy, I have a good chance to deal with my problems	.08	.77	.05	.71
7	At this point, therapy seems logical to me	.21	.42	.27	.40
8	I would recommend this treatment to a friend who has similar problems to mine	.29	.38	-	-
1	I think what we do in treatment will help me achieve the changes I want	.01	.85	.00	.87
2	What I do in treatment gives me new ways of understanding my problems	.24	.50	.08	.76

Table IV. Results from parallell analysis: Raw data eigenvalues, mean and percentile random data eigenvalues.

Root	Raw Data	Means	Prcntyle
1	6.773	1.500	1.618
2	1.938	1.378	1.460
3	.958	1.285	1.353
4	.761	1.207	1.264
5	.642	1.136	1.188
6	.555	1.070	1.118
7	.468	1.010	1.059
8	.417	0.952	0.999
9	.338	0.893	.944
10	.309	0.834	.881
11	.258	0.778	.826
12	.230	0.718	.767
13	.180	0.656	.709
14	.166	0.582	.641

The two-factor structure was supported by the parallel analysis (see Table III), which indicated that the two factors had larger eigenvalues than would be expected by chance. The two factors were termed 1.“Confidence in therapist” (including items such as “I feel accepted and respected by my therapist” and “My therapist explains my problems in a way I understand”) and 2. “Confidence in treatment” (including items such as “At this point, therapy seems logical to me” and “What I do in treatment gives me new ways of understanding my problems”), see below.

Sample 2

The confirmatory ESEM confirmed the two-factor solution (see Table IV for factor loadings and Table V for model fit). Invariance tests were conducted for responses from weeks 1 and week 6 of the inpatient treatment (see Table V). The model showed adequate fit for week 1 (χ2 (64) = 86.166, p<0.01; RMSEA=0.063, 90% C.I. [0.038, 0.087]; CFI=0.966; SRMR=0.033), and week 6 (χ2 (53) = 88.162, p<0.01; RMSEA=0.071, 90% C.I. [0.044, 0.097]; CFI=0.946; SRMR=0.035).

Table V. Summary of goodness of fit statistics.

	χ2 (df)	RMSEA	90% C.I.	CFI	SRMR	AIC	Δ RMSEA	Δ CFI
ESEM W1	86.166* (53)	0.063	[0.038, 0.087]	0.966	0.033	4947.216
ESEM W6	88.162* (53)	0.071	[0.044, 0.097]	0.946	0.035	3133.688
			Invariance week 1 week 6
Configural	382.369*(258)	0.055	[0.043, 0.067]	0.940	0.065	7980.029
Weak	407.071*(280)	0.054	[0.042, 0.065]	0.938	0.111	7970.511	0.001	0.002
Strong	435.327*(291)	0.056	[0.045, 0.067]	0.930	0.114	7979.511	−0.002	0.008
Strict	573.539*(304)	0.075	[0.065, 0.084]	0.869	0.200	8148.089	−0.019	0.061

Note. Estimator is maximum likelihood (ML); ESEM: Exploratory structural equation modeling; RMSEA: root mean square error of approximation; C.I.: confidence interval; CFI: comparative fit index; SRMR: standardized root mean square residual; AIC: akaike information criterion; Δ=difference previous model.

*p<0.01; ESEM estimated with geomin oblique rotation; Bifactor ESEM estimated with bi-geomin orthogonal rotation.

For the tests of longitudinal measurement invariance, the goodness of fit indices implied good model fit for configural, weak and strong invariance, and changes in the goodness of fit indices were within the guidelines indicating strong support for measurement invariance (Chen, 2007; Cheung & Rensvold, 2002). The model fit and changes in fit did not suggest that strict invariance was achieved. However, achieving strict invariance is not crucial for this study, as the residuals are not part of the latent factor, thus this is less important for the interpretation of latent mean differences (e.g., Meredith & Teresi, 2006; Putnick & Bornstein, 2016). To conclude, the structure found in sample 1 was replicated in sample 2 and longitudinal measurement invariance was also achieved.

Discussion

Common factors pertaining to the therapeutic relationship are considered among the most important predictors of therapeutic outcome, and it is essential to better understand these mechanisms of change in psychotherapy. Prior research indicates that well-known and often-studied relationship factors overlap considerably (Wampold & Imel, 2015b). This causes uncertainty regarding their relative importance for the therapeutic progress, and how they potentially interact throughout the treatment process (Norcross & Lambert, 2018). To understand these dynamics, it is critical to develop reliable and valid measurement. Hence, in this study, we developed a new common factor relationship instrument (i.e., the MPOQ Common Factor Scale, Modum Bad Research Institute, 2017) reflecting common factors pertaining to the therapeutic relationship that have received strong empirical (Norcross & Lambert, 2018) and theoretical support in the literature (Wampold, 2015a; Wampold & Imel, 2015b). This made it possible to investigate the underlying structure of these constructs simultaneously. Moreover, we sought to assess the structure of the constructs over the course of therapy.

The results in the current study provide information about how patients perceive and evaluate common relationship factors in psychotherapy. The six domains (i.e., agreement on goals, agreement on tasks, therapist empathy, client expectations, therapist expertise, and treatment credibility) that were included in the instrument seem to be best explained by two underlying factors, which were termed “Confidence in the therapist” (Factor 1) and “Confidence in the treatment” (Factor 2). The two latent factors each showed more than adequate internal consistency (i.e., coefficient alphas of .924 and .828 respectively), and were replicated by the use of ESEM in a second sample, supporting factor robustness and reliability. Further, there was evidence for measurement invariance over the course of therapy, indicating stability of the factor structure throughout the therapeutic process. The two factors were interrelated but also showed relative independence (correlation of .525 between the two factors).

Factor one, “Confidence in the therapist,” represents the patient's view of their therapist's qualities and perception of their therapist's ability to understand, help, and collaborate with the patient. Questions that originally tapped into therapist empathy, expertise, and agreement on goals are included in this factor. The integration of these constructs into a more global perception of the therapist and the therapeutic collaboration is interesting. We observe similar findings from Agnew-Davies et al. (1998), which made the authors interpret that clients (but not therapists) “consider professional competence as integral to the emotional bond” (p.163). Our findings push toward similar conclusions; patients do not necessarily differentiate between different therapist qualities, such as empathy and expertise.

The second factor, “Confidence in the treatment,” describes the patients’ experience of the treatment as a meaningful remedy for their problems, and positive expectations of improvement. Thus, this factor taps into outcome expectancy (Constantino et al., 2018) and treatment credibility (Devilly & Borkovec, 2000), as well as agreement on therapeutic tasks (Horvath & Greenberg, 1989), and thus seems to reflect a general expression of whether the patient has “bought into” the treatment.

The two-factor solution revealed in this study corresponds with existing conceptualizations of psychotherapy, including the contextual meta-theory of Wampold and Imel (2015b). The two factors suggest two potential change pathways, which align with those of the contextual model. Pathway 1, Confidence in the therapist, describes the experience of being engaged in a purposive therapeutic relationship with an empathic and competent therapist. This pathway corresponds with the assumption of the contextual model that the therapeutic relationship entails three change pathways, where empathy is considered a central mechanism. Pathway 2, Confidence in the treatment, corresponds with the contextual mode's assumption of expectations as a second change pathway. The contextual model assumes that expectations are created through the presence of a meaningful explanation accompanied by specific ingredients that correspond with both the patient's cultural beliefs and the explanation for the patient's problems (Wampold & Imel, 2015b). Factor 2 parallels this assumption, by incorporating both expectations, treatment credibility and a sense of the therapy having meaningful tasks/actions. This interpretation of the two-factor solution is also consistent with the notion that the therapeutic relationship might influence the therapeutic process directly through the therapeutic relationship, and indirectly, through the specific ingredients in the treatment (e.g., Zilcha-Mano, 2017).

The finding that patients form two main evaluations of the therapeutic relationship is also found in medical literature on the placebo effect. In their article, Howe et al. (2019) identified two dimensions of an effective medical relationship: warmth and competence, which they referred to as the doctor “Gets It” and “Gets you,” which correspond well to the two factors identified in the present study.

This distinction between therapist and treatment, from the patient's perspective, might be an important piece of the puzzle in regards to understanding the therapeutic process. The two factors appear clinically meaningful and relevant for the daily use in therapy, and it is natural to assume that the therapist is able to influence processes related to both factor 1 and to factor 2 (Constantino et al., 2012; Coyne et al., 2021; Vîslă et al., 2019; Zilcha-Mano et al., 2019). For instance, a therapist might interpret low scores on therapist confidence as an indication of having to practice his/her interpersonal skills (i.e., expressing empathy, communicating in a competent and verbally fluent manner and enabling direction in the therapeutic process). On the other side, therapeutic skills, such as psychoeducation, explaining the therapeutic model, connecting the therapeutic model to the patient's specific problems, and promoting hope and expectations of improvement (see Constantino et al., 2012; Coyne et al., 2021), are related to factor two. Skills associated with factor two might also tap into the therapist's capability to take into account certain patient characteristics, such as ambivalence and hopelessness, and other barriers to progress and engagement in treatment.

Limitations and Further Research

Despite the strengths of this study, such the use of advanced statistical analyses, the large and heterogeneous sample and variation in treatments, our study has some shortcomings that should be taken into account. First, the number of items included in the questionnaire might have affected the result from the factor analysis. The questionnaire included 14 items and might not have ensured enough variation, which may have masked actual differences between constructs. Despite this, a manageable number of items is also a desired characteristic of the questionnaire, to make it more user-friendly for the patients.

The measurement invariance analysis indicated that the factor structure remained stable across therapy as assessed at the between patient-level. However, our analysis did not take into account that there might be within-patient variability in conceptualization of the factor structure across time. Lately, the research community has become more aware of the issues associated with confusing within- and between- patient variability (e.g., McAleavey et al., 2020). Therapists often are interested in how their patient is changing over the course of therapy rather than how they compare to other patients, although it is also important to understand how a particular patient compares to others (i.e., is this degree of symptoms abnormal?). Our results suggest that the conceptualization (i.e., factor structure) of the common factors measured does not change over time, so a focus on how the level of each of the two factors change may over time would be useful. That is, the scores on the two dimensions profitably could be used in a within-patient analysis to determine how variability over time within a patient (across time) is related to the patient's symptom change. However, this study investigated the latent structure of these phenomena at the between-patient level. It is possible that future research will identify nuances to this finding at the within-patient level, but that kind of analysis is beyond the scope of this paper.

Another concern might be that the two factors are expressions of the patient's global evaluation of progress in therapy or that the results are due to semantics (i.e., that the two constructs are nested with the phrasing of the items). This should be examined further. There are examples of constructs measured in therapy being influenced by the phrasing of the items or by already experienced symptom relief although it seems clear that important constructs, such as the alliance, are not simply an epiphenomenon of symptom reduction (e.g., Flückiger et al., 2020; Mooney et al., 2014; Zilcha-Mano et al., 2014). However, in this study the measures were examined from week one of treatment, meaning that the likelihood of the scores being confounded with progress in treatment, is likely reduced.

Finally, the study was conducted on an inpatient sample, making generalizability a possible limitation. Despite this the patients included in this sample are comparable to ordinary outpatient clinics (see e.g., Hoffart et al., 2013; Vrabel et al., 2015), the main difference being that the inpatient treatment allows for intensive, high-dose treatment. However, the factor structure might be influenced by the treatment setting. To further understand the implications and significance of these findings, the two factors should be replicated in yet different samples, and be investigated as predictors of outcome to further ensure their criterion/predictive validity.

Notes

1 Due to space considerations, researchers are referred to an updated SPSS script allowing for permutations of raw data at https://people.ok.ubc.ca/brioconn/nfactors/rawpar.sps.