Reactivity and stability in facial expressions as an indicator of therapeutic alliance strength

Abstract

Objective: Aspects of our emotional state are constantly being broadcast via our facial expressions. Psychotherapeutic theories highlight the importance of emotional dynamics between patients and therapists for an effective therapeutic relationship. Two emotional dynamics suggested by the literature are emotional reactivity (i.e., when one person is reacting to the other) and emotional stability (i.e., when a person has a tendency to remain in a given emotional state). Yet, little is known empirically about the association between these dynamics and the therapeutic alliance. This study investigates the association between the therapeutic alliance and the emotional dynamics of reactivity and stability, as manifested in the facial expressions of patients and therapists within the session. Methods: Ninety-four patients with major depressive disorder underwent short-term treatment for depression (N = 1256 sessions). Results: Both therapist reactivity and stability were associated with the alliance, across all time spans. Patient reactivity was associated with the alliance only in a short time span (1 s). Conclusions: These findings may potentially guide therapists in the field to attenuate not only their emotional reaction to their patients, but also their own unique presence in the therapy room.

Keywords:

• therapeutic alliance
• emotional reactivity
• emotional stability
• facial expressions

Clinical or methodological significance of this article: Our facial behavior is constantly broadcasting our internal emotional state to the outside world. For the therapeutic relationship, facial behavior is a central channel of communication, yet research on this subject is lacking. This study is a proof-of-concept of not only the automatic assessment of facial behavior within the therapeutic session, but also of a systematic exploration of the interdependence in facial behavior between patients and therapists, and what it may indicate for the therapeutic relationship.

Introduction

Aspects of our emotional state are constantly being broadcast via our facial expressions (Ekman & Oster, 1979; Matsumoto et al., 2008). This broadcast occurs in every relationship, including the therapeutic relationship, most commonly conceptualized as the therapeutic alliance. In psychotherapy, therapist’s reactivity to their patient’s facial expressions can follow many possible dynamics (e.g., Rasting & Beutel, 2005): A mirroring dynamic may emerge when the therapist expresses sadness in reaction to their patient’s sadness over a recent breakup; A complementing dynamic may emerge when the therapist expresses compassion in reaction to their patient’s anger over a conversation with their parents. Therapists can also demonstrate emotional stability in response to patient distress (Ford, 2021). Emotional reactivity (i.e., when one person is reacting to the other) and emotional stability (i.e., when a person has a tendency to remain in a given emotional state) have already been empirically documented in the literature of interpersonal emotional dynamics (Butler, 2011). However, it is still an open theoretical question whether they are associated with a stronger therapeutic alliance. The therapeutic alliance refers to the collaborative relationship between patients and therapists, and encompasses the emotional bond between them, as well as the agreement between them on the goals and tasks of therapy (Bordin, 1979). A strong therapeutic alliance is one of the most consistent predictors of therapy success (Flückiger et al., 2012). Given the importance of the therapeutic alliance, exploration of this question has potential clinical significance, as therapists may be able to observe the emergence of emotional dynamics within sessions and interpret their meaning for the therapy process based on empirical data. Despite the potential benefits of this avenue of research, little is known about whether and which types of emotional dynamics are associated with the strength of the therapeutic alliance.

Much is known about emotions and emotional dynamics from the literature of emotion outside the field of psychotherapy. While disagreements abound between emotion researchers, the synthesis of the literature suggests that there are five distinct components that are involved in emotions: feeling, cognitive, motivational, somatic, and motor (Moors, 2009; Scherer, 2005). These components may manifest in the context of the therapeutic session, for example, when the patient decides to share a shameful experience (cognitive). Sharing this experience may cause them to feel anxious (feeling), and even though they want to share this information (motivational), their body may become tense (somatic), and their facial expression may show that they are distressed (motor). While the construct of facial expressions does not completely overlap with emotions (Ekman, 1993; Ekman & Friesen, 1969), facial expressions are conceptualized as a part of a single component in the process of emotion (Moors, 2009). In the context of psychotherapy, facial expressions can be specifically important as they encompass a central channel of communication between patients and therapists (Bänninger-Huber, 1992), and were shown to correlate with conscious aspects of emotional experience (Gentsch et al., 2015) as well as unconscious aspects (Dimberg et al., 2000). Psychotherapy sessions entail two (or more) people, each with their own individual emotional systems. When these emotional systems interact with one another, interpersonal emotional dynamics emerge.

Interpersonal emotional dynamics can be defined as a state in which the level of activity in one component of emotion of a person is dependent upon the level of activity in another component of emotion of another person (Butler, 2011; Sels et al., 2016). This interdependence in activity of emotion components between two people can bring about different types of interpersonal emotional dynamics. While there is an open question regarding which types of emotional dynamics exist, there is a consensus among researchers on the existence of at least two types of dynamics: emotional reactivity and emotional stability. Emotional reactivity is defined as one person’s emotional reaction to another person’s behavior (Butler, 2011), as can be manifested by their facial expressions. Emotional stability is defined as a tendency to remain in a given emotional state (also termed inertia; Butler, 2011), as can be manifested by the person’s facial expressions. There is no consensus, however, regarding the time it would take for one partner’s emotional state to influence the other partner’s emotional state (i.e., the time span of the emotional dynamic); the literature exploring emotional dynamics does not converge on a single time span. Published studies have explored emotional dynamics in time spans of anywhere between a handful of seconds (Bar-Kalifa et al., 2019; Bänninger-Huber, 1992; Frank et al., 1993; Schmidt et al., 2006) to a few minutes (Main et al., 2016; Rasting & Beutel, 2005; Tal et al., 2023; Tschacher & Meier, 2020). Moreover, studies that explored acoustic markers of emotional state throughout the conversation could not have controlled for the time span in which emotional dynamics were analyzed, as the timing of samples was dependent upon when participants chose to speak (Bryan et al., 2018; Reich et al., 2014).

In recent years, the field of psychotherapy research has witnessed a rising trend in exploring the emotional dynamics that emerge within therapeutic sessions. These studies assessed nonverbal synchrony, defined as any concurrent association (Atzil-Slonim et al., 2023) or shared temporal organization (Kleinbub et al., 2020) between patients’ and therapists’ nonverbal behavior. Conceptualized as a possible manifestation of the therapeutic relationship (Koole & Tschacher, 2016; Ramseyer & Tschacher, 2014), researchers are looking for predictors of nonverbal synchrony (e.g., McParlin et al., 2022) as well as its meaning to the therapeutic process and outcome (e.g., Tschacher & Meier, 2020). As its definition implies, synchrony describes a broader phenomenon than reactivity or stability, one which probably includes many possible types of emotional dynamics (Butler, 2011; for a review of this literature see Atzil-Slonim et al., 2023; and Kleinbub, 2017 for physiological synchrony specifically). Despite this currently flourishing research avenue of nonverbal behavior in psychotherapy, however, only a limited number of empirical studies have examined the association between types of emotional dynamics and the therapeutic alliance, and results have been mixed. These studies have evaluated the emotional states of patients and therapists by measuring nonverbal behavior in different modalities: physiology (Bar-Kalifa et al., 2019; Tschacher & Meier, 2020; see Tourunen et al., 2020 for an example of couples therapy), acoustic properties (Bryan et al., 2018; Reich et al., 2014), and facial expressions (Yokotani et al., 2020). While some studies found a positive association between the emergence of emotional dynamics and alliance strength (Bar-Kalifa et al., 2019; Bryan et al., 2018; Tschacher & Meier, 2020), another study found a negative association (Reich et al., 2014). One methodological factor that may have contributed to these mixed results is the disentangling of the different types of emotional dynamics that are at play during the therapeutic session. Despite the consensus regarding the existence of reactivity and stability as types of emotional dynamics in interpersonal relationships (Butler, 2011), to our knowledge, only a single study in psychotherapy disentangled these two emotional dynamics when testing the association with the alliance. Bryan et al. (2018) analyzed acoustic interdependence within the session and explored reactivity and stability of both patients and therapists. They reported that reactivity of either side (i.e., patient or therapist), was correlated with a stronger therapeutic bond. Stability of either patient or therapist was not found to correlate with the alliance in that study.

Another methodological factor that could contribute to the mixed results regarding the association between emotional dynamics and the alliance, additionally to disentangling of emotional dynamic types, is the chosen time span in which to explore these dynamics. Researchers focusing on physiological behavior have explored emotional dynamics in different time spans, such as 10 (Bar-Kalifa et al., 2019), 60 (Tschacher & Meier, 2020), and 90 (Tal et al., 2023) seconds, but, as mentioned before, researchers focused on acoustic behavior were not able to keep the time span of analyzed emotional dynamics constant, as time spans were dependent upon when patients and therapists decided to talk (Bryan et al., 2018; Reich et al., 2014). Taking these methodological factors into consideration, it is preferable to design a methodology that can: (1) distinguish between the emotional dynamics of reactivity and stability, (2) and can systematically control for the time span in which these dynamics are explored. Exploring facial expressions in the therapeutic session can potentially meet these methodological requirements.

Facial expressions are a universal, reliable marker of emotional state, that covaries with subjective experience, and serves many interpersonal and social regulatory functions (Ekman & Oster, 1979; Matsumoto et al., 2008). In recent years, the number of published studies focusing on facial expressions has increased, largely due to technological advances that allow automatic assessment of facial behavior (e.g., Baltrusaitis et al., 2018). Despite this increase, still only a handful of studies have explored facial expressions in psychotherapy. These studies suggest that the therapeutic context is characterized by lower levels of emotional expressiveness, as compared with everyday life (Merten, 2003). Furthermore, it was suggested that therapists are even less emotionally expressive with their facial expressions than their patients (Merten, 2003, 2005; Merten & Brunnhuber, 2004; Rasting & Beutel, 2005). This difference may be explained by patients being generally more active in the session (Rasting & Beutel, 2005). The emotional dynamics between patients and therapists, as they emerge via facial expressions, are perceived as part of a dynamic and dyadic emotional pattern that can vary in its complexity, as well as in the specific emotions that are expressed (Merten, 2003). Empirical evidence suggests that these dynamics can emerge as similar to, as well as complementing, emotions between patients and therapists (Bänninger-Huber, 1992; Rasting & Beutel, 2005).

Published research establishes the potential of exploring facial expressions in the therapeutic session by demonstrating what emotional dynamics may emerge between patients and therapists. All published research into facial expressions in psychotherapy, however, has investigated facial expressions as they are reduced to discrete groups of basic emotions (Ekman, 1992; Ekman & Friesen, 1977). This was usually done by focusing on the most prominent expressed emotion in any given segment of the therapeutic session (also termed “leitaffect”; Merten, 2003), while putting aside other emotions that are relatively less intensely expressed. While there are advantages in focusing on discrete emotions, there is also ample empirical evidence and theory that supports representing emotions as placed on a continuous spectrum (Calder et al., 2001; Russell, 1980). Furthermore, this approach requires researchers to define the specific discrete emotions to test for, a priori. A possible everyday example from within a therapeutic session that emphasizes this point could be as follows: a patient expresses anger due to some disagreement with their therapist, who first reacts with surprise and then continues to empathically respond to the patient’s anger, which causes the patient to experience relief and express joy, or even tell a joke to lighten the mood, that makes the therapist laugh. Exploring discrete emotions would require testing for this exact emotional pattern (i.e., patient-joy, in reaction to therapist-surprise, in reaction to patient-anger). Another option, however, is exploring facial expressiveness, the mere intensity of facial movement exerted by patients and therapists, without assigning discrete emotional states to them. Investigating facial expressiveness would allow identification of the emotional ping-pong described in the above example, while not requiring any a priori assumptions regarding what discrete emotions will be expressed. As far as we know, only one study has explored facial expressiveness as an indicator of the therapeutic alliance. Yokotani et al. (2020) investigated interviews of psychology students and tested correlation between interviewer and interviewee in the movement of specific facial muscles (e.g., left eye, right eyebrow). They found that correlation in facial movements is correlated with a stronger therapeutic alliance.

The overarching goal of the present study is to explore reactivity and stability between patients’ and therapists’ facial expressions, as indicators of therapeutic alliance strength. Consistently with literature in the field, we used the Actor-Partner Interdependence Model (APIM; Kenny et al., 2006) which is designed to disentangle reactivity and stability by modeling partner effects alongside actor effects in the same statistical model. We had two aims: (1) to test whether the levels of emotional reactivity and emotional stability are associated with a stronger therapeutic alliance, and (2) to systematically explore how this association varies in different time spans. Based on the literature reviewed above, our hypotheses were: (1) therapist reactivity will indicate a stronger therapeutic alliance; (2) patient reactivity will indicate a stronger alliance; (3) the more the therapist exhibits stability the stronger the therapeutic alliance will be; (4) given the mixed results in the literature, we may expect that some effects may emerge only in specific time spans. This aspect of the analysis (i.e., analysis of different time spans) was exploratory without hypothesizing which time spans will prove more relevant than others. To test these hypotheses, we analyzed facial behavior and self-report data from 1256 sessions of 94 therapeutic dyads enrolled in a randomized control trial of short-term psychodynamic psychotherapy for major depressive disorder (MDD). The focus on MDD is important, as there is substantial literature demonstrating both the phenomenon of nonverbal interdependence between therapists and patients diagnosed with MDD (e.g., Deres-Cohen et al., 2021) and the reduced levels of facial activity in the psychopathology of depression (e.g., Gehricke & Shapiro, 2000).

Method

Design

Ninety-four patients diagnosed with major depressive disorder (MDD) were recruited through advertisements offering free-of-charge psychodynamic treatment for depression. This study is part of a larger randomized control trial (RCT), which began recruitment in March 2016. The trial compared supportive and supportive expressive therapy for MDD (Zilcha-Mano et al., 2018, 2021). Patients were randomly assigned to either supportive or supportive-expressive psychodynamic therapy (Luborsky, 1984; Luborsky et al., 1995). Patients from each treatment group received 16 sessions of therapy. All participants signed an informed consent form that included an agreement for the therapy sessions to be recorded. All procedures were approved by the institutional review board of the University of Haifa, Israel (375/19).

Patients

Inclusion criteria of the RCT were (a) meeting MDD criteria, which included an MDD diagnosis according to the MINI (International Neuropsychiatric Interview; Sheehan et al., 1998) and scores above 14 on the 17-item Hamilton Rating Scale for Depression (HRSD; Hamilton, 1967) at two evaluations 1 week apart; (b) Patients who were on medication were required to remain on a stable dosage for at least 3 months prior to beginning the therapy, as well as maintain this dosage throughout therapy; (c) patients were between 18 and 60 years old; (d) Hebrew language fluency; and (e) provision of an informed consent. Exclusion criteria were (a) current risk of suicide or self-harm (HRSD suicide item higher than 2); (b) current substance abuse disorder; (c) current or past schizophrenia or psychosis, bipolar disorder, or severe eating disorder requiring medical monitoring; (d) history of organic mental disease; and (e) currently receiving psychotherapy elsewhere.

Patients in the included sample had a mean age of 31.25 years (SD = 8.69). 60.2% of patients were women. The mean of education years was 14.5 (SD = 2.1). 80.6% of patients identified as Jewish, 8.6% as Muslim, one as Christian, one as Druze, and 7.5% as other. Regarding income status, 50.5% of patients reported their monthly family income as below the average monthly income per family, 21.5% reported their income as average, 24.7% reported it was above average, and 3.3% did not report their income. Regarding employment, 65.6% of the patients were employed, 32.3% unemployed, 2.1% reported “other”. Regarding marital status, 78.5% of patients were single, 15.1% were married, 5.4% divorced or separated, 1% widowed. In addition to MDD, 74.2% of the patients were diagnosed with one or more personality disorders using the Structured Interview for DSM-IV Personality (SIDP; Pfohl et al., 1997). The most frequent personality disorders were avoidant (41%), obsessive-compulsive (39%), and dependent (13%). The percentage of comorbidity with anxiety disorders was 67.7%. The most frequent anxiety disorders were generalized anxiety (61%), social anxiety (29%), agoraphobia (15%). Additionally, 3% were diagnosed with obsessive compulsive disorder, and 5% were diagnosed with posttraumatic stress disorder.

Therapists and Treatment

Seven therapists participated in this study, each with at least 5 years of expertise in psychodynamic treatment. All therapists had formal experience in psychodynamic treatment. The therapists attended a 20-hour training workshop in supportive and expressive techniques prior to seeing patients. Therapists received weekly private and group supervision, provided by two experienced and licensed clinical psychologists. Supervisors were themselves under supervision provided by an international supportive-expressive therapy expert. Each therapist provided both treatment conditions to act as their own controls. The mean clinical experience of the therapists was 13.36 years (SD = 5.7), mean age was 40.3 years (SD = 6.75), and only one of the therapists was male. The mean number of patients per therapist was 13.29 (SD = 8, range = 2–25). Patients received 16 weekly 50-minute sessions of either supportive expressive treatment, or supportive treatment, both of which are time-limited psychodynamic therapies adapted for depression. Supportive expressive treatment is based on the use of expressive techniques, such as interpretation, confrontation, and clarification (Luborsky et al., 1995). Supportive treatment includes the use of supportive techniques aimed at alliance strengthening, such as affirmation and empathic validation (Leibovich et al., 2018). For more details, see the study protocol (Zilcha-Mano et al., 2018);

Assignment to treatment condition was conducted via an outside institution not involved in the study. Comprehensive treatment protocols for supportive expressive treatment were used (Luborsky, 1984; Luborsky et al., 1995). The supportive treatment condition included all supportive techniques detailed in the manual, but excluded any expressive techniques (for more details, see Leibovich et al., 2020).

Measures

Alliance

The therapeutic alliance was measured using the Working Alliance Inventory—Short Form: Patient version (WAI; Horvath & Greenberg, 1989). This is a 12-item self-reported inventory based on Bordin’s (1979) conceptualization of the working alliance. The score is calculated as an average of all 12 items, and addresses the three components of the alliance: the emotional bond between patient and therapist, their agreement on the tasks, and their agreement on goals of treatment. The WAI was administered to the patients after each session. In the current study, mean WAI score was 5.62 (SD = 0.98) and the internal reliability range across measurement points of the WAI was 0.9–0.95.

Facial expressiveness

Facial expressions were automatically assessed from video-recorded sessions. All sessions were video-recorded with two cameras, one pointing at the patient, the other pointing at the therapist. Videos were recorded at 25 frames per second. Out of 118 patients who participated in the randomized control trial: one patient has requested for their data to be deleted. 11 patients received the therapy in other facilities at the beginning of the trial, with nonstationary, mostly lower-quality recording equipment; 13 patients received their entire therapy via remote video conversation after the outbreak of COVID-19 toward the end of the randomized control trial. In these remote therapy sessions, patients and therapists were not recorded simultaneously and thus analysis of simultaneous facial expressions was impossible. These 25 patients were excluded from the analysis. Still, not all sessions of the remaining 94 patients were analyzable, for the same two reasons: 10 patients have moved in the middle of their therapy into the main facility of the research trial, with the appropriate recording equipment; COVID-19 broke out when 14 patients were in the middle of their therapy, necessitating a sudden move to video therapy which disallowed further assessment of facial behavior. Eventually, 94 patients were included in the analyzed data, though some sessions of some patients could not be analyzed as mentioned above. The final sample had 7 patients who dropped out (7.4%). Overall, 1256 sessions were included in the analyzed data overall.

Facial expressions were measured using the OpenFace 2.0 software (Baltrusaitis et al., 2018) which is an open-source software applying a machine learning algorithm (convolutional neural network) aimed at identifying facial expressions. Facial expression analysis of OpenFace is based on the Facial Action Coding System (FACS; Ekman & Friesen, 1977). The automatic coding of OpenFace results in estimations of activation in single muscle innervations, termed “action units”. Essentially, OpenFace analyzes each individual frame separately. After an initial phase of identifying facial landmarks in the frame, OpenFace is capable of estimating the intensity of 17 action units separately, all of which were included in the analysis (for a list of all AUs, see Table I). Validation studies of OpenFace reported that its accuracy surpasses other methods (e.g., generative latent tree, convolutional neural networks; Baltrusaitis et al., 2018). This was also true when externally validated using unseen manually coded facial data (Namba et al., 2021). Additionally, OpenFace-based evaluations of arousal and valence were significantly correlated with therapists’ evaluations of their patients’ arousal and valence, albeit not with patients’ own reports (Terhürne et al., 2022).

Table I. List of action units assessed by OpenFace 2.0.

Id	Name
AU1	Inner brow raiser
AU2	Outer brow raiser
AU4	Brow lowerer
AU5	Upper lid raiser
AU6	Cheek raiser
AU7	Lid tighter
AU9	Nose wrinkler
AU10	Upper lip raiser
AU12	Lip corner puller
AU14	Dimpler
AU15	Lip corner depressor
AU17	Chin raiser
AU20	Lip stretched
AU23	Lip tightener
AU25	Lips part
AU26	Jaw drop
AU45	Blink

Notes: A list of all action units (AU) assessed by OpenFace 2.0 for level of activity. The entire list of AUs (N = 17) was included in the main analysis.

To calculate facial expressiveness from the extracted intensity of action units, we used a simple distance calculation between Cartesian coordinates. Each accumulated sample (representing a frame of the video recording) was placed as a point in a Cartesian system with 17 dimensions, for the 17 action units measured. We then measured the distance between each two samples representing consecutive frames. Distance was measured via the numpy.linalg.norm function in the NumPy library (Harris et al., 2020), using Python (Python 3.6.8; Van Rossum & Drake, 2009). The code has been made available on a public repository (https://osf.io/6th5q/?view_only = bea87b55c1be4c16ac9f18a3548d58ab). The result was a difference score between each two consecutive facial expression samples, representing the change in facial expressions. Higher scores represent more facial behavior. A value of zero (the lowest difference score achievable) represents a face that did not change between two consecutive frames. This distance score was named “facial expressiveness.”

To allow the exploration of different time spans, facial expressiveness was averaged with moving, non-overlapping windows. As there were no hypotheses as to why a certain time span is more relevant than another, we explored all time spans reported by studies investigating the association between emotional dynamics and the therapeutic alliance. Overall, we created 90 versions of the data (a maximum of 90 s was chosen due to this being analyzed by Tal et al., 2023). The first version included facial expressiveness with a moving window of 1 s length, the second version included facial expressiveness with a moving window of 2 s length, etc. Eventually, time spans of 1–90 s were analyzed for. However, only a subgroup of these windows will be reported (length of 1, 5, 10, 30, 60, and 90 s) in the results section. A full description of the entire 90-windows analysis can be found in the supplementary material. Additionally, in segmenting the data into windows, this analysis explores emotional dynamics in facial behavior with a constant internal temporal structure. While a case can be made for why emotional dynamics should be analyzed with dynamic temporal structure, keeping the temporal structure constant does have some advantages. One advantage is that it allows contrasting short-term reactivity with long-term reactivity. For example, keeping the temporal structure constant allows the contrasting of reactivity in shorter time spans (e.g., 1 s) with longer time spans (e.g., 10 s or 30 s). In other words, keeping the internal temporal structure constant, while it has its caveats, may allow taping into different mechanisms that are at play. See Figure 1 for an illustration and Table II for descriptive statistics.

Figure 1. Preprocessing pipeline of facial reactivity and stability.

Notes: This figure demonstrates the pipeline implemented to achieve an estimate of facial reactivity and stability for both patients and therapists. First, we obtained the raw scores of facial expressiveness for patients and therapists, as they are calculated for each frame of video recording. The left-most panel shows an example of 8 s of expressiveness, with a hypothetical dyad of patient and therapist. Second, we create 90 datasets, for 90 different possible time spans (i.e., 1, 2 s, etc.). The second panel from the left shows the result of this step. Third, an APIM model is fitted to each of these datasets. The model is made to predict expressiveness at time t with expressiveness at time t–1. The APIM model was defined with random effects of dyad and session. Finally, this results in an estimation of reactivity and stability, of both patients and therapists, in each session separately. These estimations were then used to predict WAI scores from the same session. This is demonstrated by the right-most panel, that shows the resulting datasets. Pr = patient reactivity, Ps = patient stability, Tr = therapist reactivity, Ts = therapist stability, WAI = Working Alliance Inventory.

Table II. Descriptives statistics.

Measure	Mean	SD
WAI	5.62	0.985
Patient expressiveness	0.341	0.092
Therapist expressiveness	0.242	0.057
1-second time spans
Patient Stability	−0.214	0.039
Patient Reactivity	0.146	0.026
Therapist Stability	0.009	0.022
Therapist Reactivity	−0.134	0.047
5-second time spans
Patient Stability	−0.255	0.067
Patient Reactivity	0.115	0.034
Therapist Stability	0.051	0.034
Therapist Reactivity	−0.176	0.065
10-second time spans
Patient Stability	−0.227	0.083
Patient Reactivity	0.071	0.034
Therapist Stability	0.085	0.039
Therapist Reactivity	−0.214	0.073
30-second time spans
Patient Stability	−0.168	0.11
Patient Reactivity	0.002	0.031
Therapist Stability	0.13	0.044
Therapist Reactivity	−0.275	0.088
60-second time spans
Patient Stability	−0.163	0.121
Patient Reactivity	0.002	0.029
Therapist Stability	0.134	0.044
Therapist Reactivity	−0.297	0.091
90-second time spans
Patient Stability	−0.167	0.122
Patient Reactivity	0	0.027
Therapist Stability	0.131	0.041
Therapist Reactivity	−0.278	0.087

Notes: Descriptive statistics. SD = Standard deviation.

Data Analysis

All analyses were conducted in the PROC MIXED program (Littell et al., 2006) in SAS 9.4 (SAS Institute, 2015) due to the nested structure of the data. To measure reactivity and stability in facial expressiveness, we fitted an Actor-Partner Interdependence Model (APIM; Kenny et al., 2006) unto the data. The APIM estimates the level of activity (i.e., facial expressiveness) in each participant as predicted by that participant’s previous level, and the previous level of their partner. This results in estimates of actor and partner effects. If the participant’s levels are consistent across time, the estimate of the actor effect will be greater. If a change in the participant’s levels is commonly followed by a similar change in their partner’s levels, the estimate of the partner effect will be greater. Patient stability was defined as the effect of patient expressiveness when predicting subsequent patient expressiveness (patient actor effect). Therapist stability was defined as the effect of therapist expressiveness when predicting subsequent therapist expressiveness (therapist actor effect). Patient reactivity was defined as the effect of therapist expressiveness when predicting subsequent patient expressiveness (therapist partner effect). Therapist reactivity was defined as the effect of patient expressiveness when predicting subsequent therapist expressiveness (patient partner effect). To achieve an estimation of stability and reactivity for each session separately, dyad and session were random effects in the model. These estimations of reactivity and stability were later used to predict the therapeutic alliance strength of the same sessions in which they were measured (see Table II for descriptive statistics of reactivity and stability).

An analysis of APIM is based on the premise that one partner’s attributes and behaviors can affect the other partner’s behavior (Bolger & Laurenceau, 2013; Kenny et al., 2006). This is a valid assumption in the context of facial expressions of two individuals in an interaction. Another requirement of APIM is that the two variables are placed upon the same evaluation rating scale (Bolger & Laurenceau, 2013; Kenny et al., 2006). This is also a valid assumption in the case of the current study, as the same phenomenon (i.e., facial expressiveness) was assessed in both patients and therapists, in the exact same manner.

The data were hierarchically nested on two levels: assessments nested within patients. Following Bosker and Snijders (2011; see also Falkenström et al., 2020), we adhered to the rule of thumb noting that cases in which there are less than ten groups (our sample had seven therapists) should not be estimated using random effects, thus no therapist effect was included in the model (adding a therapist effect did not change the results, this sensitivity analysis is reported in the supplementary material). To account for the resulting non-independence of assessments, and to prevent inflation of the effects, we added the patient as random effects. To measure the amount of unexplained variance in outcome due to the random effects of the patient, we used intra-class correlations (ICCs). The patient’s random variance component was estimated based on a model with only random intercept of the patient with no other covariates.

To examine alliance changes over time, we evaluated the following trend models for each: stability over time, and log of time as fixed effect. We started with a model with only a fixed intercept and random effect of log of time, and added a fixed effect of log of time. We used the Bayesian information criterion (BIC) to determine whether the inclusion of each term improved the model fit, and chose the model that reached the lowest BIC. As the aim of this model selection was to examine alliance changes over time, facial data was not included. The choice of modeling time logarithmically is due to the literature demonstrating that the psychotherapy process can follow a logarithmic pattern (Gallop & Tasca, 2009; Nissen-Lie et al., 2015; Tasca & Gallop, 2009), and therapeutic alliance literature emphasizing the importance of the early stages of treatment in building and establishing the working alliance (Horvath & Luborsky, 1993). Results of a similar analysis with time modeled linearly are presented in the supplementary material.

With the chosen model, we estimated the effects of stability and reactivity of both patients and therapists on rated alliance. The final model was as follows: $y_{\mathrm{it}}=b_{0i}+b_{1}P{s}_{\mathrm{it}}+b_{2}P{r}_{\mathrm{it}}+b_{3}T{s}_{\mathrm{it}}+b_{4}T{r}_{\mathrm{it}}+b_5\log \left(t\right)+e_{\mathrm{it}}$with $y_i$ as WAI rated by patient from dyad i at session t; $P{s}_{\mathrm{it}}$ as patient stability of patient from dyad i at session t; $P{r}_{\mathrm{it}}$ as patient reactivity of patient from dyad i at session t; $T{s}_{\mathrm{it}}$ as therapist stability of therapist from dyad i at session t; $T{s}_{\mathrm{it}}$ as therapist stability of therapist from dyad i at session t; and $e_{\mathrm{it}}$ as error of dyad i at session t.

Results

Estimated variance of the patient’s random effect for alliance was significant (${\sigma }^2$= 0.7, p < 0.001, ICC = 0.71). This means that there were significant differences in reported WAI scores between patients, regardless of the estimated effects. The model that was found to have the best fit based on the BIC for alliance was with a fixed and random effect of log of time. This model was used in all analyses. In all models analyzed, the effect of log of time was stable and significantly positive (estimates range was 0.27–0.28, mean of estimates was 0.27, SE was 0.04, p < 0.0001). This means that alliance levels tended to rise quickly and remain strong across treatment. Therapist’s reactivity in facial expressiveness, in each time window (i.e., 1, 5, 10, 30, 60, 90 s) was significantly associated with the therapeutic alliance (estimates range was 0.52–0.71 (mean = 0.6), SE was 0.17–0.27, p-value was 0.001–0.017). Therapist’s stability in facial expressiveness, in each time window was also significantly associated with the therapeutic alliance (estimates range was 1.6–2.84 (mean = 2), SE was 0.47–0.83, p-value was 0.001–0.009). Patient’s reactivity in facial expressiveness was not significantly associated with the therapeutic alliance in almost all time windows. Only in time windows of 1 s was patient reactivity in facial expressiveness significantly associated with the therapeutic alliance (B = 2.23, SE = 0.78, p = 0.004). Patient stability in facial expressiveness was significantly associated with the therapeutic alliance only in windows of 5 s (B = 0.56, SE = 0.28, p = 0.042). All reported effects regarding therapist reactivity and stability, and patient reactivity, remained significant when excluding outliers. In contrast, the significant effect of patient stability did not remain once outliers were excluded (higher than 3.3 standardized score in absolute value), and thus this effect was not further interpreted. Analyses of windows under one second (0.4, 0.6, and 0.8 s) revealed that therapist reactivity and stability, as well as patient reactivity remained significant in predicting the alliance. See Figure 2 for an illustration of the results and Table III for a complete description of linear models. Analyses of WAI subscales, and therapist-rated alliance were added in the supplementary material.

Figure 2. Types of emotional dynamics that are associated with the therapeutic alliance.

Notes: This figure presents the results of the main statistical analysis. Each line represents a dynamic that was significantly associated with the therapeutic alliance. All estimated effects were positive. The line with the square-shaped head represents therapist reactivity (significant across all analyzed time spans). The line with the arrow-shaped head represents therapist stability (significant across all analyzed time spans). The line with the circle-shaped head represents patient reactivity (significant only in time span of one second).

Table III. Results of all regression models.

Effect	Estimate	Standard error	P value
1 s time spans
Log of time	0.27	0.04	<0.0001
Patient stability	0.66	0.38	0.08
Patient reactivity	2.23	0.78	0.004
Therapist stability	1.78	0.68	0.009
Therapist reactivity	0.6	0.24	0.011
5 s time spans
Log of time	0.27	0.04	<0.0001
Patient stability	0.56	0.28	0.042
Patient reactivity	0.77	1.12	0.489
Therapist stability	1.98	0.52	0.0001
Therapist reactivity	0.52	0.19	0.005
10 s time spans
Log of time	0.27	0.04	<0.0001
Patient stability	0.43	0.25	0.089
Patient reactivity	0.94	1.76	0.593
Therapist stability	1.7	0.51	0.001
Therapist reactivity	0.55	0.18	0.002
30 s time spans
Log of time	0.27	0.04	<0.0001
Patient stability	0.44	0.24	0.062
Patient reactivity	4.72	3.41	0.166
Therapist stability	1.6	0.47	0.001
Therapist reactivity	0.57	0.17	0.001
60 s time spans
Log of time	0.28	0.04	<0.0001
Patient stability	0.26	0.27	0.337
Patient reactivity	2.18	3.59	0.544
Therapist stability	2.1	0.61	0.001
Therapist reactivity	0.71	0.19	0.001
90 s time spans
Log of time	0.28	0.04	<0.0001
Patient stability	0.44	0.35	0.208
Patient reactivity	0.66	4.58	0.886
Therapist stability	2.84	0.83	0.001
Therapist reactivity	0.65	0.27	0.017

Notes: Results of all analyzed linear models. Six models overall for six reported time spans (i.e., 1, 5, 10, 30, 60, and 90 s). All significant effects of patient stability did not remain significant when excluding outliers. All other significant effects remained when excluding outliers.

Discussion

The present study explored emotional reactivity and emotional stability between patients’ and therapists’ facial expressions as indicators of therapeutic alliance strength. Findings suggest that both therapist reactivity and therapist stability are uniquely associated with a stronger alliance. Findings also suggest that patient reactivity is associated with a stronger alliance, but only in a time span of one second. Patients’ stability, in contrast, was not consistently associated with alliance strength, no matter the specific time span analyzed. These findings contribute to the currently developing knowledge regarding the association between types of emotional dynamics between therapists and patients and the strength of the therapeutic alliance.

Therapist emotional reactivity has emerged as a significant indicator of the therapeutic alliance. As hypothesized, our findings suggest that the more the therapist reacts to their patient, via their facial expressions, the stronger is the therapeutic alliance. The effect remained significant across all analyzed time spans (i.e., 1–90 s). This finding is consistent with the literature. The notion that therapists’ emotional reactivity may be beneficial for the therapeutic relationship has already been suggested (e.g., Kimerling et al., 2000). However, most studies have focused on self-report measures, administered before or after the session (Sloan & Kring, 2007). There is only one other study, to our knowledge, that has investigated the association between the alliance and emotional reactivity within the therapeutic session. In this study, Bryan et al. (2018) explored acoustic behavior and reported that the more a therapist’s emotional arousal follows that of the patient’s, the stronger is the therapeutic bond. The current study adds to the literature by reporting a similar effect to that reported by Bryan et al. (2018), albeit with facial expressions. Specifically, this may suggest that the association between therapist reactivity and the alliance is robust enough that it transcends the specific modality it is measured in (e.g., facial behavior, acoustic properties).

Constructs with some possible overlap with therapist reactivity have also been referenced in other areas of psychotherapy research, further supporting the reported findings of the current study. Elkin et al. (2014) used the term “therapist responsiveness” to describe situations when the therapist is attentive to the patient, makes eye contact, makes an effort to see the patient’s perspective, and more. The authors suggested that therapist responsiveness, manually coded within sessions by outside observers, is associated with a higher quality of the therapeutic relationship. It is important to note that therapist reactivity and therapist responsiveness are not interchangeable terms. Therapist reactivity is being suggested by this study as a possible indicator of therapeutic alliance strength. Therapist responsiveness, in contrast, has already been heavily explored and is more broadly defined as any behavior of the therapist that is being influenced by any emerging context in the therapy (Kramer & Stiles, 2015). When therapist reactivity is low, however, this may be a sign that the therapist is bored, tired, or simply emotionally distant. Dahl et al. (2017) used the term “therapist disengagement” to describe similar situations. The authors found that therapist disengagement, as indicated by a post-session self-report measure, indicates poorer therapy outcomes. Therapist disengagement was suggested in the literature both as a possible result of emotional burnout (Delgadillo et al., 2018), and as a necessary self-care measure for therapists to prevent a state of emotional burnout (Figley, 2002). The current study adds to this literature by demonstrating the importance of therapist reactivity to the therapy process while using an objective methodology, capable of automatically analyzing large amounts of data. This type of methodology potentially allows a more fine-grained analysis of within-session processes, while providing a labor-free alternative to the labor-intensive option of manual coding.

Therapist emotional stability has also emerged as a significant indicator of the therapeutic alliance. As hypothesized, our findings suggest that the more the therapist stays emotionally stable within the session, as expressed via their facial expressions, the stronger is the therapeutic alliance. The effect remained significant across all analyzed time spans (i.e., 1–90 s). This finding seemingly contradicts Bryan et al. (2018), who did not find a correlation between therapist emotional stability and the therapeutic bond. One possible post-hoc explanation of this contradiction may be rooted in the differences in stability measurement between the two studies. While our methodology entailed exploration of emotionally relevant behavior continuously, Bryan et al. (2018), by analyzing acoustic behavior, were inherently limited in their exploration, measuring emotional arousal only at times when talking took place. If replicated in other studies, it may be suggested that therapists experience different emotional arousal (e.g., higher or lower levels) when they talk, in comparison to when they listen. This hypothesis should be explored in other studies in a more direct manner. Still, this contradiction may mean that the two studies have tapped into different aspects of emotional stability; hence the contradictory findings. If replicated, this discrepancy suggests that emotional stability may be too broad of a term, and that different types of therapist emotional stability should be further considered and explored.

Therapist emotional stability is less commonly explored empirically in the literature, in comparison to therapist emotional reactivity. One area of psychotherapy research that supports these findings is countertransference management. Countertransference was originally introduced by Freud (1910) and can be defined as the thoughts, perceptions, feelings, and behavior of the therapist, as they are being influenced by the patient (Hayes et al., 2011). Oftentimes this influence, taken at face value, can be detrimental to the therapeutic process. At the same time, psychotherapeutic theories see emerging countertransference as an impasse in therapy, which, if managed by the therapist, can facilitate the therapy process instead of hindering it (Hayes et al., 2011; Levenson, 1995). An example of such an impasse can be as follows: a person is seeking therapy due to recurring painful breakups. They find themselves eventually losing patience with their partner’s behavior and acting rudely toward them, which inevitably leads to clashes and yelling. When this patient comes to treatment, they may find themselves losing patience toward their therapist as well, which may cause the patient to behave rudely in sessions. In response to this attitude, the therapist may feel hurt, belittled and even the need to “strike back”. In other words, the therapist may find themselves influenced and subtly coerced into this maladaptive dance, leading to the same fights with the patient that the therapy is meant to mitigate. Alternatively, at this impasse, the therapist may be able to remain emotionally stable, managing this countertransference (Hayes et al., 2011), and exploring the patient’s current attitude. Eventually, this stability may allow the patient a better understanding of themselves and of the therapy process (Hayes et al., 2011), which may manifest as a stronger therapeutic alliance. This rationale may hold some resemblance to the psychoanalytic rationale of therapist abstinence (Freud, 1915), according to which the therapist is advised to keep from providing the patient with the immediate emotional response they are expecting or searching for, in order to facilitate the therapy process. These post-hoc interpretations should be considered carefully, however, as this is the first study to measure patients’ and therapists’ facial expressiveness within the session. Further exploration into the stability of facial expressiveness is required to better understand the underlying subjective experience it indicates.

Turning to the patient’s facial expressions, patient emotional reactivity has emerged as a significant indicator of the therapeutic alliance. As hypothesized, our findings suggest that the more the patient reacts to the therapist, via their facial expressions, the stronger is the therapeutic alliance. The effect reached significance, however, only in a time span of 1 s. Analyses of longer time spans did not yield a significant effect for patient reactivity. This finding is consistent with the literature. Specifically, by exploring acoustic behavior, Bryan et al. (2018) reported that when a patient's emotional arousal follows that of the therapist’s, the therapeutic bond is stronger. Our study adds to this literature by suggesting that the effect of patient reactivity may only emerge in very short time spans (e.g., 1 s).

It is intriguing to speculate why patients’ reactivity was associated with the therapeutic alliance only in a short time span. This consideration should be taken carefully, as this is an exploratory analysis without prior hypothesis and the first study that systematically explores different time spans of emotional dynamics between patients and therapists. Still, while these findings require further replication in future studies, a possible post-hoc explanation can be suggested when considering other areas of research that have also focused on human emotional reaction in short time spans (i.e., around 1 s). One such area of research is that of the orienting response. The orienting response can be defined as a person’s immediate response to a novel or significant change in their environment (Bradley, 2009), as long as this change is not sudden enough to elicit a startle response (Graham, 2021). Some behavioral measures of the orienting response (e.g., skin conductance, pupil dilation) were demonstrated to also peak around the same time span as our reported effect of patient reactivity (around 1 s after the stimulus was introduced; Nieuwenhuis et al., 2011). Importantly, focusing more on the significance of a stimulus rather than its novelty, it was suggested that motivational systems are mediating the orienting response (Bradley, 2009). In other words, the orienting response may be at least partially dependent on the meaning of the stimulus for the person (Johnson, 1986) and the person’s attitude toward it (Sutton et al., 1965). For the case of the current study, if considered in terms of the orienting response, the effect of patient reactivity may potentially mean that the therapeutic alliance was stronger when the patient was more motivated to attend their therapist’s communication (via facial expression).

The effect of log of time has also reached significance in addition the interdependence effects. The positive significant effect of log of time suggested that alliance tended to increase in the early stages of therapy, and remain strong throughout it. This finding is supported by the therapeutic alliance literature, which emphasizes the importance of the early stages of treatment in building and establishing the working alliance (Horvath & Luborsky, 1993), as well as the literature suggesting that the psychotherapy process can follow a logarithmic trajectory (Gallop & Tasca, 2009; Nissen-Lie et al., 2015; Tasca & Gallop, 2009).

In interpreting the results, it is important to remember that facial expressions can serve many communicative functions, sometimes even simultaneously (Chovil, 1991; Fridlund & Russell, 2006). Some of these functions can be emotionally relevant, like raising one’s eyebrows to signal surprise. Other functions can be more neutral, like opening one’s mouth to signal their motivation to begin talking, or to signal action readiness in general (Frijda & Tcherkassof, 1997). Thus, the phenomena of reactivity and stability in facial expressions that are discussed in this study can be an equifinal manifestation of different processes of nonverbal communicative events that occur between patients and therapists. Future studies may distinguish different communicative functions of facial expressions when investigating reactivity and stability in psychotherapy.

Reported findings should be considered carefully, however, as there are some limitations to the methodology that was implemented. First, this study introduces a new data-driven approach to measuring facial behavior. While this approach allowed measurement of the amount of facial expressiveness, it is still unclear what the dynamics of facial expressiveness mean. For example, therapist stability in facial expressiveness requires further empirical investigation to achieve a better grasp of the underlying experience it indicates. Second, facial expressions were analyzed continuously, even when patients and therapists were talking, which may have influenced the results. While it can be argued that facial behavior is worth analyzing while the person is engaged in talking, it may be beneficial to separate moments of inherent facial movement (e.g., talking) from moments in which the face could remain still (e.g., listening). One possible way to do that is to fuse facial data with acoustic data within the same analysis. This may allow a more nuanced multimodal analysis which taps into different modes of reactivity and stability. Third, while the reported methodology was intentionally designed with minimal assumptions about the dynamics of facial behavior, future studies may also distinguish between different properties of the different facial muscles. For example, facial expressions can encompass larger facial areas (e.g., a smile) or smaller ones (e.g., a blink), they can also be longer or shorter. Some action units tend to activate together as well (e.g., eyebrows and eyelids in a surprised expression). One way to address these different dynamics is by adopting an event-focused approach. For example, a facial event can be a happy expression or a brief, offended expression. Facial events have onset, duration, and termination, as well as a group of facial muscles, which can all be predefined theoretically and assessed statistically, to help distinguish between signal and noise or to explore specific facial behaviors. Fourth, this study has relied upon a component conceptualization of emotions (Moors, 2009; Scherer, 2005). Furthermore, within the component conceptualization this study has focused on the motor manifestation of emotion (i.e., facial expressions). Investigating changes in other components of emotion, or relying upon a different conceptualization of emotion altogether, may yield different results. Fifth, this study focused on patients diagnosed with major depressive disorder. The literature links depression with lower emotional expression in facial behavior (Berenbaum & Oltmanns, 1992) and impaired attribution of emotion in facial behavior of others (Demenescu et al., 2010). Both findings can heavily influence nonverbal communication via facial expressions in therapy of patients with depression. Analyses of therapy with patients from different populations may lead to different results. Sixth, the reported analysis in this study adopted a non-overlapping windows approach. While this approach seems to be more common in the literature (e.g., Bar-Kalifa et al., 2019; Tschacher & Meier, 2020), it also introduces an implicit overlap between the duration of facial behavior measured and the time difference in which a reaction in the other is assessed. This is important as different methodologies were shown to tap into different aspects of nonverbal interdependence (for example with physiological data, see Tschacher & Meier, 2020). Future studies may help broaden the knowledge in this field by suggesting more approaches to data segmentation when assessing interdependence. Seventh, the analysis reported had a constant temporal structure for emotional dynamics (i.e., time spans analyzed in each version of data remained constant). Future studies can select some dynamic criteria by which to temporally segment the clinical material (e.g., by fusing acoustic analysis of talking turns; e.g., Bryan et al., 2018). This may lead to tapping into different mechanisms of communication between patients and therapists.

This study explored emotional reactivity and stability between patients and therapists, as they manifest in their facial expressions. Therapist reactivity and stability, as well as patient stability, were positively associated with the strength of the therapeutic alliance. This study demonstrates the great potential of exploring emotional dynamics between patients and therapists by empirically supporting their importance to the therapy process. Researchers may also explore a link with therapy outcome in future studies. If replicated, these findings suggest not only the potential benefit of the emotionally reactive therapist, but also the benefit of the emotionally stable therapist, who integrates their own personality, their presence, into their practice.

Acknowledgements

We thank Dr. Ilan Shimshoni (University of Haifa) for establishing the infrastructure required to use OpenFace 2.0.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Supplemental Data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/10503307.2024.2311777.

Additional information

Funding

This work was supported by the Israel Science Foundation (Grant Number 186/15 and 395/19 to S. Zilcha-Mano), and by the University of Haifa Data Science Research Center.