From face to face: the contribution of facial mimicry to cognitive and emotional empathy

ABSTRACT

Despite advances in the conceptualisation of facial mimicry, its role in the processing of social information is a matter of debate. In the present study, we investigated the relationship between mimicry and cognitive and emotional empathy. To assess mimicry, facial electromyography was recorded for 70 participants while they completed the Multifaceted Empathy Test, which presents complex context-embedded emotional expressions. As predicted, inter-individual differences in emotional and cognitive empathy were associated with the level of facial mimicry. For positive emotions, the intensity of the mimicry response scaled with the level of state emotional empathy. Mimicry was stronger for the emotional empathy task compared to the cognitive empathy task. The specific empathy condition could be successfully detected from facial muscle activity at the level of single individuals using machine learning techniques. These results support the view that mimicry occurs depending on the social context as a tool to affiliate and it is involved in cognitive as well as emotional empathy.

KEYWORDS:

• Facial mimicry
• empathy
• emotional
• cognitive
• complex emotions

Humans often react to the emotional expressions of others with spontaneous imitation. Facial expressions that match the expressed emotion of the counterpart have been labelled “facial mimicry” and are considered central for social interactions (Fischer & Hess, 2017; Hess & Fischer, 2013). Mimicry is an ubiquitous phenomenon that can be found not only in the laboratory (Dimberg, 1982) but also in the wild (Fischer, Becker, & Veenstra, 2012) and as a reaction to strangers as well as to close interaction partners (McIntosh, 2006). Importantly, recent evidence suggests that the extent of facial mimicry depends on contextual factors such as situations, motivations, and affordance (Hess & Fischer, 2014; Seibt, Mühlberger, Likowski, & Weyers, 2015). Moreover, people seem to mimic the valence of emotions rather than discrete emotions (Hess & Fischer, 2013).

One of the most pressing questions in mimicry research regards the role of mimicry in social processing. The new view of context-dependent mimicry seems promising to target this question, as social information occurs naturally within a rich context.

The extant literature emphasises two main claims regarding the function of mimicry, a role in emotion understanding (e.g. Niedenthal, Brauer, Halberstadt, & Innes-Ker, 2001), and a means of fostering affiliation (cf. Hess & Fischer, 2013). The first notion is theoretically based on the so-called Facial Feedback Hypothesis (Adelmann & Zajonc, 1989). This hypothesis posits that the automatic facial muscle activity of the perceiver of an emotion generates neural feedback. This feedback triggers a similar emotion in the perceiver. In turn, the perceiver may better understand the other person’s feelings (Barsalou, Niedenthal, Barbey, & Ruppert, 2003). The second approach regards mimicry as a reaction to an emotional signal that serves to signal affiliative intent and thereby can enhance liking and rapport in social interactions (Hess & Fischer, 2013).

Mimicry and empathy

A common conceptualisation of empathy distinguishes two facets (Blair, 2005; Shamay-Tsoory, 2011; Walter, 2012) that can be mapped to these possible functions of mimicry: The first facet, cognitive empathy, describes a person’s ability to infer the mental states of others (Baron-Cohen & Wheelwright, 2004). The second facet, emotional empathy, is defined as an observers’ emotional response to another individual’s emotional state (Eisenberg & Fabes, 1990). The conceptualisation of empathy into these two facets is supported by behavioural (Dziobek et al., 2008), neuroimaging (Dziobek et al., 2011; Fan, Duncan, de Greck, & Northoff, 2011), and neurological findings (Shamay-Tsoory, Aharon-Peretz, & Perry, 2009). A relationship with mimicry has been suggested for both facets (for an overview see, Seibt et al., 2015).

Cognitive empathy is conceptually linked with emotion recognition (Dziobek et al., 2011; Tager-Flusberg & Sullivan, 2000; Wolf, Dziobek, & Heekeren, 2010): recognising an emotion, especially a complex or subtle emotion in a rich context, requires the capacity to take the other’s perspective and to infer their mental or emotional state. Most mimicry studies focused on the recognition of so-called basic emotion expressions. There is some evidence for reduced emotion recognition when mimicry was blocked, compared to when participants were free to mimic (Ponari, Conson, D’Amico, Grossi, & Trojano, 2012; for an overview see, Hess & Fischer, 2013). This relationship has been reported mostly for positive emotions like happiness (Oberman, Winkielman, & Ramachandran, 2007) or for the detection of smiles (Niedenthal et al., 2001; Rychlowska et al., 2014) and often only in reference to detection time rather than accuracy or only for female participants (Stel & van Knippenberg, 2008). Other studies could not find an effect (Blairy, Herrera, & Hess, 1999; Fischer et al., 2012; Hess & Blairy, 2001).

Decreased cognitive empathy is a core deficit in autism-spectrum conditions (ASC; Dziobek et al., 2006, 2008). These neurodevelopmental conditions are characterised by impairments in social communication and interaction as well as restricted interests and repetitive behaviour (American Psychiatric Association, 2013). Furthermore, aberrant automatic, but intact intentional mimicry of facial expressions has been reported for people with ASC (McIntosh, Reichmann-Decker, Winkielman, & Wilbarger, 2006; Stel, van den Heuvel, & Smeets, 2008). Individuals with ASC who were instructed to imitate the facial expressions of emotions reported to not experience them – contrary to control participants in the same study and to the Facial Feedback Hypothesis (Stel, van den Heuvel, et al., 2008). As autistic traits are assumed to be continuously distributed across the population (Baron-Cohen, Wheelwright, Skinner, Martin, & Clubley, 2001), they might be confounding the relationship of empathy and mimicry, as a recent study suggests (Neufeld, Ioannou, Korb, Schilbach, & Chakrabarti, 2016).

Concerning emotional empathy, previous research points to a bidirectional relationship with mimicry. Even though people with stronger empathic traits show more consistent mimicry (Dimberg, Andréasson, & Thunberg, 2011; Sonnby-Borgstrom, 2002), it also has been observed that mimicking another person’s facial emotional expression fosters a shared emotional state in the observer, a form of emotional empathy (Stel, van Baaren, & Vonk, 2008). While many studies have focused on mimicry of anger and happiness, a recent study (Rymarczyk, Żurawski, Jankowiak-Siuda, & Szatkowska, 2016a) also showed that emotional trait empathy modulated the strength of facial mimicry of fear and disgust. However, some studies did not find any relationship between the extent of such emotional contagion and the level of mimicry (Blairy et al., 1999; Hess & Blairy, 2001).

To date, only few mimicry studies have measured cognitive and emotional empathy simultaneously. Blairy et al. (1999) conducted a series of experiments, in which they presented pictures of facial expression of basic emotions. They measured decoding accuracy as well as self-reported emotional state. Although they found that participants mimicked the emotions and that the decoding of emotions was accompanied by shared affect, they found no indication of a mediation by mimicry. In a similar study, Hess and Blairy (2001) used video clips of spontaneous emotion expressions from an emotional imagery task. This task had high ecological validity, but leads to very low decoding rates for anger and disgust. The authors reported a null result regarding the relationship of mimicry with emotional contagion and emotion recognition. In both studies, mimicry was assessed by electromyography but activity of the Zygomaticus Major was not measured. This might have skewed the assessment of mimicry of happiness, as this is often based on activity of this muscle, while the mimicry of anger is often based on the activity of Corrugator Supercilli (Rymarczyk, Żurawski, Jankowiak-Siuda, & Szatkowska, 2016b).

Hofelich and Preston (2012) used an emotional Stroop task presenting background pictures of basic emotions together with incongruent and congruent emotional adjectives. They replicated the Stroop effect in that the irrelevant emotional facial expressions slowed down the encoding of incongruent adjectives. But this effect was not associated with mimicry of the background faces. Regarding emotional empathy, their study was limited to trait empathy: they classified their participants into a high- and a low-empathic group based on three trait empathy measures. Congruent facial responses towards the emotional adjectives tended to be stronger in the high-empathic group. Sato, Fujimura, Kochiyama, and Suzuki (2013) found relationships among facial mimicry, emotional experience and emotion recognition using video clips of happiness and anger portrayed by amateur Japanese models. They asked participants to rate the valence and arousal of presented emotions. The activity of the Zygomaticus Major and Corrugator Supercilii measured with EMG correlated with the valence and arousal ratings of the recognised as well as the experienced emotions. Yet, such ratings are only a limited proxy for the assessment of discrete emotion recognition. Moreover, the results might be confounded by the intensity of the emotion.

Due to methodological characteristics and limitations, the non-findings in the reviewed studies do not necessarily represent evidence for a non-existing relationship, but point to the need for further studies. Extant research used only decontextualised facial expressions of basic emotions. However, in real life, emotions are almost always more complex, expressed subtly and within a context. Especially, to examine cognitive empathy, the use of contextually embedded and rich social stimuli is advantageous, as it requires more complex processing such as perspective taking and interpretation of an entire scene. A recent study by van der Graaff et al. (2016) targeted this issue by using empathy-inducing film clips with short happy or sad episodes and measured Zygomaticus Major and Corrugator Supercilii activity as well as emotional and cognitive empathy at state and self-reported trait level. State emotional empathy was conceptualised as experiencing the same emotion. To assess participants’ state cognitive empathy, the participants’ statements regarding the reasons why they sympathised with the individuals in the film clips, were coded regarding their cognitive attributions. Valence-congruent muscle activity was positively associated with both facets of state empathy. However, this muscle response might be an emotional reaction to the positive and negative content of the video clips, rather than an imitation of the facial expressions. Moreover, the results are limited to adolescents.

In sum, to date, the research regarding the role of mimicry for the two facets of empathy comes to divergent conclusions. To better understand the relationship of mimicry with both the cognitive and the emotional facet of empathy, studies are needed that carefully investigate both facets together, ideally based on the same stimulus material. Too little attention has been paid to the use of context-embedded naturalistic stimuli. Further, given the argument outlined above, it seems prudent to control for autistic traits as these pose a potential confound. This was the goal of the present study. Specifically, we examined the relationship between mimicry and emotional as well as cognitive empathy in reaction to complex emotions embedded in photorealistic contexts. This more naturalistic setting further allowed us to compare directly the relationships of mimicry with cognitive empathy versus mimicry with emotional empathy. We suggest here that emotional mimicry is related to the understanding of an emotion in context and functions to regulate relations with the other person. Thus, we predicted that mimicry responses should be related to both facets of empathy. Based on previous research, we expected more mimicry for the emotional empathy task pointing to the affiliative role of mimicry. Finally, we explored the possible confounding role of autistic traits.

Method

Participants

Seventy volunteers (age range 18–36 years, mean age: 25.5 years; 31 women) participated individually. The sample size of 70 exceeded the required sample size of 67 estimated by a statistical power analysis (evaluated for bivariate correlations with power = 0.80, α = 0.05 and effect size ρ = 0.30), based on a conceptually similar study (Sato, Fujimura, & Suzuki, 2008). More than half of the participants had received a high-school degree and no one scored below 20 in the WST (see below).

All participants were screened to exclude current or prior history of psychological, neurological or psychiatric disorders including substance abuse. Moreover, students of medicine or psychology were excluded as they might guess the real purpose of the EMG electrodes. Due to the EMG measurement, participants with facial sensitivity disorders, paralysis or severe acne could not take part in the study. Participants were asked to abstain from alcohol, caffeine and tobacco for 24 h prior to the experiment. To ensure that all participants understood the emotional words used in the experiment, we assessed their levels of verbal intelligence through a German vocabulary test, the Wortschatztest (WST; Schmidt & Metzler, 1992). All participants displayed sufficient language understanding. All participants provided written informed consent prior to participation and received monetary compensation after the experiment. Ethical approval was obtained from the ethics committee of the Humboldt-Universität zu Berlin.

Procedure

After completing the informed consent form, participants were seated in a sound-attenuated and electrically shielded laboratory 1 m in front of a computer screen (1024 × 768), where they completed the revised version of the Multifaceted Empathy Test (MET, Dziobek et al., 2008; see below). During the task, facial EMG was measured from the Corrugator Supercilii (brows; associated with negative affect) and Zygomaticus Major (cheek; associated with positive affect). The experimenter told the participants that the EMG electrodes measure sweat so that they would not concentrate on their facial movements. To allow checking for artefacts after completion of the testing, the face of the participant was video recorded during the EMG measurement. After the testing session, the participants completed two personality trait measures: autistic traits were measured via the German short-version (Freitag et al., 2007) of the Autism-Spectrum Quotient (AQ; Baron-Cohen et al., 2001). To assess trait empathy, the German version (Paulus, 2009) of the Interpersonal Reactivity Index (IRI; Davis, 1983) was used.

MET

The MET (Dziobek et al., 2008) is a computer-based task for the assessment of emotional and cognitive empathy with high ecological validity. There are several advantages of the MET compared to other state empathy measures. First, as the MET uses photorealistic stimuli including complex emotions and context, it requires inferring mental states beyond the simple recognition of isolated basic emotion. Second, contrary to questionnaires, it does not rely on self-report but on performance for measuring cognitive empathy. Third, the test measures both facets of empathy within the same task paradigm allowing for a direct comparison.

The MET consists of a total of 40 photographs depicting individuals of varying gender and age in emotionally charged situations. Half of the pictures show people expressing positive emotions (e.g. “relaxed”, showing a man receiving a massage) and the other half negative emotions (e.g. “painful”, showing a woman suffering acute pain in a hospital room). The emotional states are embedded in rich contexts, e.g. a hospital room, war scene, or a bicycle race, where other individuals are also seen. However, the target persons, including their face are displayed prominently in each picture to avoid ambiguity in responding. There was no evidence for a less pronounced mimicry response towards the pictures including background faces. Each picture was presented twice (once in the emotional and once in the cognitive empathy condition) and in blocks of 10 with the same valence (positive/negative) and empathy condition (cognitive/emotional). To assess cognitive empathy, the participant was asked to infer the mental state of the person in the picture by choosing one out of four possible emotional words via key press. The time between the presentation of the words and the key press was measured as reaction time. To assess emotional empathy, the participant indicated to what degree they empathise with the individual in the photo. This rating was made on a Likert-scale anchored with 1 (not at all) to 9 (very much). An example for the cognitive and emotional empathy condition is shown in Figure 1.

Figure 1. Example of a cognitive empathy condition item (left) and an emotional empathy condition item (right) of the MET.

Individual difference measures

Autism-Spectrum Quotient (AQ)

The German short-version (AQ-k; Freitag et al., 2007) of the AQ (Baron-Cohen et al., 2001) is a 33-item self-report questionnaire assessing different areas connected with ASC, such as social and communication skills, imagination and attention. Each item describes a certain behaviour or attitude. On a 4-point scale, participants indicate how strongly they agree with each statement. Every slight or strong agreement to an autistic behaviour adds one point to the total score. A score of 17 and above is seen as an indicator for an amount of autistic traits that might be clinically significant. The AQ has been shown to have good test–retest reliability and inter-rater reliability (Baron-Cohen et al., 2001) as well as good discriminative validity and good screening properties in clinical practice (Woodbury-Smith, Robinson, Wheelwright, & Baron-Cohen, 2005). While the mean AQ score was low (M = 8.8, SD = 5.4), six participants scored above the cut-off of 17, which might be indicative of a clinical diagnosis of ASC.

Interpersonal Reactivity Index (IRI)

The IRI (Davis, 1983; German version: Paulus, 2009) is a 16-item self-report questionnaire that taps different dimensions of empathy. It has four subscales: Perspective Taking, Empathic Concern, Personal Distress and Fantasy. This version has shown good reliability, factorial validity and item analysis (Paulus, 2009). We used two subscales: perspective taking, as it corresponds most closely to cognitive empathy and empathic concern, as it matches the concept of emotional empathy. An example item of perspective taking is: “before criticising someone, I try to see the issue from their point of view”. An example of an empathic concern item is: “I would describe myself as a warm-hearted person”. The two facets were not significantly correlated in our sample (rs = −0.02, p = .794).

EMG

Before attaching the electrodes, participant’s skin was cleaned with an alcohol pad. Bipolar surface electrodes were attached over the Zygomaticus Major and Corrugator Supercilii muscles on the non-dominant side of the face, according to Fridlund and Cacioppo (1986). Electrode impedance was kept below 50 kΩ. The EMG signals were amplified with EMG amplifiers (Becker Meditec, Karlsruhe, Germany; gain = 1230; band pass 19–500 Hz) and digitised by means of a USB multifunction card USB-6002 (National Instruments Inc., Ireland), connected to a laptop computer running DASYLab 10.0 (National Instruments Ireland Resources Limited). The raw EMG signals were sampled at 1000 Hz and digitised with 16-bit resolution. Within DASYLab, signals were online RMS (root mean square) integrated with a time constant of 50 ms. The integrated fEMG and the trigger signals were down sampled to 20 Hz and stored as an ASCII file.

The muscle activity was sampled for each trial consisting of a fixation cross of 1500 ms as a baseline followed by the picture for four seconds. Afterwards the possible answers were shown, the participant could react and a blank screen appeared for 100 ms (Figure 2 shows the time course of an example trial). To avoid movement artefacts, the EMG response of this second period including the behavioural reaction was not analysed.

Figure 2. Example of a trial of the MET.

Data reduction and artefact control

Following established procedure (cf. Bush, Hess, & Wolford, 1993) the data was within-subjects z-transformed. Standardised facial muscle activity was then averaged in intervals of 100 ms. A change to baseline score was calculated for each interval by subtracting the averaged activity from 500 to 1500 ms preceding stimulus onset. The 500 ms directly preceding stimulus onset could not be used to technical difficulties. The video recording of all participants was screened visually for artefacts, such as sneezing, coughing or yawning that could disrupt the EMG measure. EMG values recorded during artefacts were excluded. Trials with more than 25% of values missing due to artefacts were completely excluded. Additionally, an automatic outlier detection was performed excluding all values from a baseline-corrected trial that were three standard deviations above the mean value of all trials. Data from two participants had to be excluded as they had more than 20% missing trials.

Data analysis strategy

To assess the presence of mimicry, a repeated measures ANOVA with valence (positive, negative), muscle site (Zygomaticus Major, Corrugator Supercilii) and time (bins of 100ms: 500–4000 ms) as within-subject factors was conducted. The first 500 ms after stimulus onset were excluded to avoid confounds from orienting. A significant interaction of muscle and valence was interpreted as support for a mimicking reaction. As an index of the magnitude of mimicry, a difference value of muscle activity depending on the valence was calculated: For positive expressions Corrugator Supercilii activity was subtracted from Zygomaticus Major activity. For negative expressions, the reverse difference was calculated. This difference in muscle activity was averaged over time. A positive value was interpreted as a congruent facial expression.

To analyse the relationship between facial mimicry and emotional and cognitive empathy, linear mixed effects models using residual maximum likelihood estimation were calculated with participants as random factor to account for the dependence of the repeated measures of each individual.

Which condition a facial expression was presented in should be predictable if the task demands lead to different empathic reactions. We used machine learning to explore how accurately the empathy condition (emotional versus cognitive) can be predicted from facial muscle activity. This is a binary classification task with balanced classes (each facial expression is presented once in each condition). We built models mapping the feature vector x gained from the muscle activity to a value of a decision function to predict the empathy condition y. The basic feature vector of a single trial of one participant consists of the z-standardised and baseline-corrected activity values for each muscle (Zygomaticus Major and Corrugator Supercilii). The activity of both muscles was analysed from 500 to 4000 ms after stimulus onset and integrated with a time constant of 100 ms, so that x ∈ R⁷⁰. Across individuals all trials for each facial expression were averaged separately for each condition.

Additionally, based on previous studies analysing facial EMG (Hess, Kappas, McHugo, Kleck, & Lanzetta, 1989), a handcrafted feature vector was calculated on the muscle sequences: the mean and the minimal and maximal activity in a trial as well as the variance, the skewness and the kurtosis of the muscle activity response.

For classification, a random forest classifier and a support vector machine was used. The support vector machine classifies instances by building a separating hyperplane. The algorithm is maximising the margin of the training instances of the two classes from this hyperplane. The solution can be expressed as a linear combination of a subset of the training data called support vectors that lie next to the separating hyperplane (Boser, Guyon, & Vapnik, 1992). To account for non-linearly separable data, we used the hinge loss function that gives back zero for correct classification and a value proportional to the distance from the margin for a wrong classification.

A random forest is an ensemble classifier that builds several decision trees on sub-samples of the data and averages the results (Breiman, 2001). The use of many trees reduces the habit of decision trees to overfit to the training data. The samples were drawn with replacement (bootstrapping). To measure the quality of the splits, gini impurity was used. The depth of the trees and the minimal number of instances for a leaf node were hyperparameter.

To train and evaluate the machine learning classifiers, the data was split in train and test sets (70/30) with a randomly shuffled stratified three-fold cross-validation. The test instances were not used for training the models. For the tuning of the hyperparameter, a nested five-fold cross-validation inside the training data was used.

The aim of the machine learning analyses differs from the aim of the ANOVA approach. While the ANOVA shows that mimicry responses in the two conditions differ on average over all individuals, the machine learning analysis investigates whether or not a condition can be detected from the mimicry responses of a single individual (Bzdok, Altman, & Krzywinski, 2018). Further, while the ANOVA is based on the mean value of the mimicry response, the machine learning approach is based on a richer, high-dimensional representation of the mimicry response, using a 70-dimensional representation in the case of the basic feature vector, and information such as minimum, maximum, variance, skewness, and kurtosis in the case of the handcrafted feature vector. Compared to the other analyses, the machine learning approach thus allows to detect differences between the mimicry response in the two conditions with a much higher resolution.

Results

Behavioural measurement of cognitive and emotional empathy in the MET

On average, participants indicated a level of emotional empathy comparable to previous reported levels for healthy individuals (e.g. Hurlemann et al., 2010; Ritter et al., 2011). Similarly, their accuracy in the cognitive task (see Table 1) was comparable to previous research. One participant was excluded from all analyses of the empathy task due to a detection rate more than five standard deviations below the mean. Positive and negative expressions were not rated significantly differently – neither in the cognitive nor the emotional empathy task (see Table 1).

Table 1. Cognitive and emotional empathy values of the MET.

	Positive expressions	Negative expressions	Overall
Cognitive empathy (percentage of correct answers)	0.75 (0.43)	0.73 (0.44)	0.74 (0.10)
Emotional empathy (on a scale of 1–9)	5.31 (2.42)	5.15 (2.33)	5.23 (1.30)

Women scored significantly higher (M = 77%, SD = 9%, 95% CI [74%, 81%]) than men (M = 71%, SD = 10%, 95% CI [67%, 74%]) on the cognitive empathy task, T = 2.75, p = .008, d = 0.68. In addition, women’s reaction times (Mdn = 4539 ms), were slightly lower than the reaction times of men (Mdn = 5042 ms), U = 432.0, p = .057, r = 0.23. There were no gender differences for the emotional empathy task.

Cognitive empathy was negatively correlated with AQ values, rs = −0.31, p = .010, the association with the perspective taking subscale of the IRI was not significant (rs = 0.17, p = .181). In line with earlier studies (e.g. Dziobek et al., 2008), a positive association of emotional empathy with trait empathic concern (subscale of IRI) was found, rs = 0.33, p = .008 (see Table 2 for all correlations of MET scores with subscales of IRI).

Table 2. Correlations of MET scores with subscales of IRI.

	MET emotional empathy			MET cognitive empathy
	Overall	pos	neg	Overall	pos	neg
EC	0.33**	0.22	0.37**	0.01	−0.07	0.12
PT	0.21	0.21	0.17	0.17	0.07	0.23
FS	0.17	0.14	0.16	0.29*	0.25*	0.26*
PD	0.22	0.09	0.32*	−0.06	−0.14	0.09

Note: EC = Empathic Concern, PT = Perspective Taking, FS = Fantasy, PD = Personal Distress; pos = positive emotions, neg = negative emotions. ** < 0.01, * < 0.05.

Facial mimicry

To test the general occurrence of mimicry across both empathy conditions, we used a repeated measures ANOVA with the muscle activity score as dependent variable and with the within-subject factors muscle site (Zygomaticus Major versus Corrugator Supercilii), valence of the expression (pos. and neg.) and time (100 ms bins).

Across both empathy conditions, a significant interaction between muscle site and valence supports the assumption of a mimicry reaction, F(1, 67) = 34.24, p < .001, ${\eta }_G^2$ = 0.304. Additionally, a main effect of valence was observed, F(1, 67) = 4.035, p = .049, ${\eta }_G^2$ = 0.024, pointing towards a stronger facial reaction towards negative expressions.

Post hoc analyses revealed that, as expected for mimicry, the reaction of the Corrugator Supercilii across time was higher in response to negative facial expressions (M = 0.05, SD = 0.07, 95% CI [0.04, 0.07]) than to positive facial expressions (M = −0.06, SD = 0.07, 95% CI [−0.07, −0.04]), t = 6.27, p < .001, d = 0.77. Similarly, higher Zygomaticus Major activity was observed in response to positive emotions (M = 0.03, SD = 0.08, 95% CI [0.01, 0.05]) than to negative ones (M = −0.03, SD = 0.08, 95% CI [−0.05, −0.01]), t = 3.03, p = .004, d = 0.37. The activity of both muscles over time separated by valence is shown in Figure 3.

Figure 3. Activity of both muscles over time separated by valence.

Facial mimicry and emotional empathy

The level of facial mimicry during the emotional empathy condition correlated positively with the person’s emotional empathy score, rs = 0.24, p = .049 (Table 3). The person’s emotional empathy total score in turn was positively correlated with mimicry of both negative (r = 0.26, p = .034) and positive emotions (r = 0.29, p = .019) in the emotional empathy condition. A mixed linear model with participants as random factor supported the positive relationship of the emotional empathy response and the mimicry response, b = 0.26, SE = 0.05, z = 5.02, p < .001. However, a model separating the responses by the valence of the facial expression and including the valence by mimicry interaction, revealed that valence moderates the relationship of mimicry and emotional empathy. The model (see Table 4) performed significantly better than models without the interaction and pointed to a strong relationship for the individual’s reaction to positive facial expressions only. Post hoc correlation tests confirmed a positive relationship between mimicry and emotional empathy response for positive (r = 0.25, p = .045) but not for negative expressions (r = 0.15, p = .23). Additionally, a significant positive correlation of the average emotional empathy a photo elicited and the average mimicry the facial expression in this photo evoked was found, r = 0.57, p < .001, as can be seen in Table 3.

Table 3. Correlations of MET scores with facial mimicry.

	Facial mimicry
	Overall	neg	pos
CE (MET) – indivi.	0.06	0.13	0.10 s
EE (MET) – indivi.	0.24* s	0.25*	0.15
CE (MET) – item	0.01	−0.23	0.22 s
EE (MET) – item	0.57***	0.21	0.78***

Note: Facial mimicry measured for the specified subtask only. CE = only for cognitive empathy condition of MET, EE = only for emotional empathy condition of MET; pos = positive emotions, neg = negative emotions. *** < 0.001, ** < 0.01, * < 0.05.

_s = Spearman-correlation used in case of evidence for non-normal distribution.

Table 4. Prediction of emotional empathy in the MET with dummy coded valence (0 – positive, 1 – negative).

	Coefficient	Standard error	Z	p
Mimicry	−0.03	0.08	−0.33	.74
Valence (positive)	0.10	0.08	1.28	.20
Mimicry × valence (positive)	0.49	0.10	4.82	<.0001

To assess the association of mimicry with state empathy, which is not explained by trait empathy, we calculated a step-wise linear regression model that controlled for emotional trait empathy as measured by the empathic concern scale of the IRI. In this model, empathic concern and mimicry both contributed significantly to the prediction of emotional empathy (see Table 5). As all responses from the emotional empathy task of the MET by each individual were aggregated, the valence of each item could not be included into the model in a similar way as in the previous single-item analysis of emotional empathy measured by the MET and mimicry. The was no evidence for a direct association of any IRI subscale score with facial mimicry (see Table 6).

Table 5. Prediction of emotional empathy in the MET.

	Coefficient	Standard error	T	p
Mimicry	3.32	1.27	2.62	.01
Trait empathic concern (IRI)	0.15	0.06	2.51	.02

Table 6. Correlations of IRI subscale scores with facial mimicry.

	Facial mimicry
	Overall	cog	emo
EC	−0.05	−0.17	0.11
PT	0.12	0.08	0.12
FS	0.14	0.15	0.07
PD	−0.11	−0.17	−0.01

Note: cog = only for cognitive empathy condition of MET, emo = only for emotional empathy condition of MET; pos = positive emotions, neg = negative emotions. ** < 0.01, * < 0.05.

Autistic traits

We calculated a multiple linear regression model with the emotional empathy MET score as dependent variable and the average mimicry response and the AQ score as independent variables to control for the influence of the autistic traits. The emotional empathy score was associated positively with the mimicry response, b = 3.68, SE = 1.22, t = 3.03, p = .004, but not significantly associated with autistic traits, b = −0.04, SE = 0.03, t = −1.35, p = .182.

Facial mimicry and cognitive empathy

As for emotional empathy, we calculated a linear mixed effects model with participant as random factor and valence-condition as random slope to calculate the main effects of mimicry (mean intensity) of the facial expressions as well as valence on the correct response rates for positive and negative facial expressions (two measures). Mimicry influenced the recognition performance positively (b = 0.10, SE = 0.05, z = 2.05, p = .041). The valence of the items did not influence cognitive empathy. Further, there was no significant interaction between valence and mimicry on cognitive empathy. There was no significant point-biserial correlation between the accuracy in cognitive empathy task and the level of mimicry for each item in that task. The response time scaled with the level of mimicry, r_s = 0.36, p = .003.

A mixed linear model with the log-transformed reaction times as dependent variable and with pictures as random factor supported the positive relationship of reaction time and mimicry response, b = 0.03, SE = 0.01, z = 2.39, p = .017. An analysis with participants as random factor did not reveal an association of reaction time and mimicry (p = .39).

Autistic traits

Controlling for individual’s AQ-score by including them in the model, did not affect the main effect of mimicry (b = 0.25, SE = 0.10, z = 2.52, p = .012). Further, we replicated previous findings (e.g. Dziobek et al., 2008) of a negative association between autistic traits and cognitive empathy (b = −0.01, SE = 0.002, z = −2.18, p = .029). There was no evidence for an influence of the interaction between mimicry and autistic traits on the cognitive empathy score (b = −0.02, SE = 0.01, z = −1.74, p = .082).

Emotional versus cognitive empathy

To compare the level of mimicry in the emotional and cognitive empathy task, a repeated measures ANOVA on the accumulated mimicry activity over time was calculated with valence and condition as within-factors. The analysis revealed a significant difference between the emotional and the cognitive empathy condition, F(1, 66) = 10.43, p = .002, ${\eta }_p^2$ = 0.03. There was no evidence for a main effect of valence. The interaction between valence and condition was not significant, F(1, 66) = 3.20, p = .078, ${\eta }_p^2$ = 0.02. Table 4 shows the mean mimicry activity as a function of valence and condition.

Prediction of cognitive and emotional empathy by machine learning

It was possible to predict the condition in which a facial expression had been presented with an accuracy of 70% from the muscle activity of the Zygomaticus Major and Corrugator Supercilii (basic feature vector) with the support vector machine. Based on the handcrafted features a prediction of 72% with the random forest was possible. Figure 4 shows the learning curve of the random forest classifier.

Figure 4. Mean mimicry activity separated by valence and condition.

Discussion

The current study presents evidence for the mimicry of complex, context-embedded emotional expressions and its association with cognitive and emotional empathy. The results indicate a stronger association of mimicry with emotional than with cognitive empathy. These findings further support the view of mimicry as primarily affiliative response to an emotional signal in social interaction (Hess & Fischer, 2013).

The individuals’ average emotional empathy score (aggregated over positive and negative stimuli) was strongly associated with the degree of mimicry towards both positive and negative emotion expressions. This result goes beyond the previously reported relationship of emotional empathy and mimicry of basic emotions (e.g. Dimberg et al., 2011; Sonnby-Borgstrom, 2002), because our study focused on emotional empathy elicited by the MET’s complex emotion pictures. Our results also extent recent findings of a study showing a relationship between the degree of emotional mimicry and pain empathy (Sun, Wang, Wang, & Luo, 2015) beyond negative emotions to positive emotions.

Mimicry and emotional empathy in the MET were associated, even after we controlled for empathic concern measured by the IRI. The questionnaire serves here as a measure of trait empathy, while the emotional empathy task of the MET measures empathy at the state level. Thus, the relationship of mimicry and emotional empathy is not reducible to the influence of personal attitude or motivation to empathise. As most research on mimicry and emotional empathy so far has focused on trait measures (e.g. Dimberg et al., 2011; Sonnby-Borgstrom, 2002), this is an interesting finding that further supports the assumption of mimicry as a mechanism to regulate affiliation within a given situation. Controlling for autistic traits did also not eliminate the association between emotional empathy and mimicry.

To better understand the relationship between mimicry and state empathy, we conducted mixed linear models that included the valence of the particular item. The analyses yielded a linear relationship between mimicry intensity and emotional empathy for positive valence only. If we considered negative emotion expressions only, individuals’ mimicry did not correlate with their reported empathy towards individuals in negative emotional contexts.

Furthermore, the correlation analysis aggregating over participants for each item showed the relationship for positive emotions not only on the individual level, but also on the stimulus level: positive facial expressions that elicited more emotional empathy across participants also lead to a more pronounced mimicry response. This supports the assumption of mimicry as a reflection of the intensity with which the emotional stimulus solicits empathy instead of just a reflection of a general level of trait empathy of a person.

In contrast to previous findings (e.g. Sun et al., 2015) we found a linear relationship between mimicry intensity and emotional empathy only for positive valence. A possible explanation might be that the negative facial expressions used in this study (e.g. a painful expression in a hospital scene) led to a higher emotional involvement and triggered personal memories or associations, evident in a stronger mimicry response. Thus, the mimicry response might be confounded by a reactive response to the emotional scene, a problem that has previously been noted for mimicry (Hess & Fischer, 2014). This confounding might have led to a dissociation of the emotional empathy response and the facial muscles response and, thus, a weaker association.

Accuracy in inferring complex emotions in the MET was also positively related to mimicry. Thus, the current study extends previous findings of a positive relationship between mimicry and basic emotion recognition (e.g. Künecke, Hildebrandt, Recio, Sommer, & Wilhelm, 2014; Stel & van Knippenberg, 2008) to complex emotions. The understanding of complex emotions is considered to depend on context, culture and attribution of mental states, rather than distinctive facial expressions (Griffiths, 2004). Thus, the finding of valance-congruent mimicry of complex emotions and its relationship with recognition supports the assumption that individuals mimic the valence of an emotion in context.

It should be mentioned that not all studies have reported positive effects of mimicry on emotion recognition. A possible reason why we identified such relationship might lie in the complexity of our emotional stimuli and the resulting difficulty of the emotion recognition task. Previous studies often reported ceiling effects regarding emotion recognition that might have reduced the association with mimicry. As the level of complexity in our study equals real-life situations rather than the simplicity of full-blown basic emotions, it can be assumed that mimicry comes into play in everyday life.

The mixed results in the literature regarding the relationship between cognitive empathy and mimicry might also be explained by overlooked personality traits. However, in our study the relationship of cognitive empathy and mimicry survived controlling for autistic traits. In accordance, we did not replicate previous findings of generally reduced spontaneous mimicry in individuals with high autistic traits. This might be due to the fact that we measured a non-clinical sample instead of either individuals with an autism diagnosis (McIntosh et al., 2006; Oberman, Winkielman, & Ramachandran, 2009; Stel, van den Heuvel, et al., 2008) or extremely high or low values on the AQ (Hermans, van Wingen, Bos, Putman, & van Honk, 2009).

In line with the observation that more studies in the mimicry field found a relationship with emotional than with cognitive empathy, we also found a higher level of mimicry in the MET emotional empathy than cognitive empathy condition. It seems plausible that the emotional empathy condition triggered an affiliative intent and thus enhanced mimicry (Hess & Fischer, 2014). This result is in accordance with a study indicating that the intensity of mimicry can be modulated by the task and the resulting goal of the participant (Cannon, Hayes, & Tipper, 2009). The participants in that study only showed mimicry when they were instructed to infer the emotion of facial expressions, not when they reported on the colour of the picture frame.

In our study, the effect of the emotional versus the cognitive condition on the amount of mimicry seemed much stronger for negative valence. A possible explanation for this might be that the idea of emotional empathy triggered by the request to “feel with someone” is more closely connected to showing empathy and sympathy for negative emotions than for positive ones.

We also used machine learning to analyse the predictive value of mimicry, the specific MET empathy condition (recognition versus feeling with) could be successfully predicted via a random forest and via a support vector machine (Figure 5). This suggests a dissociable pattern of muscle activity in response to the task condition. The facial expression seems to be distinct depending on the motivation of the participant to empathise versus recognise an emotion, as the classifier where able to detect the task condition based on the muscle response. The aspect of differentiable mimicry responses has been neglected so far. Our findings encourage to further research different patterns of the mimicry response.

Figure 5. Learning curve of the random forest predicting the empathy condition (cognitive versus emotional).

Contrary to what was expected, the cognitive empathy task of the MET did not correlate with the perspective taking (PT) subscale of the IRI. There are several possible explanations for this lack of correlation, among others the different task formats, i.e. performance test versus questionnaire. Questionnaires require the ability to introspect and to abstract from single events to general trends (Dziobek et al., 2008). Further, questionnaires, are more likely than performance tests such as the MET to be biased by the tendency to give answers in a socially desirable manner (Van de Mortel, 2008). This is a special concern for topics with clear societal expectations about the “right” and morally esteemed answers like empathy.

Further, the MET measure performance in cognitive empathy and, accordingly, has been shown to differentiate between individuals struggling with cognitive empathy such as those with autism and healthy controls (e.g. Dziobek et al., 2008, 2011). In contrast, the IRI PT subscale targets mainly motivational aspects of cognitive empathy (e.g. “I try to look at everybody’s side of a disagreement before I make a decision”). Thus, it is not surprisingly that the cognitive empathy task of the MET as a measure of capacity and the PT subscale of the IRI as a measure of motivational aspects only correlate weakly.

Furthermore, the two measures differ also regarding the valence of their items: The IRI is limited mostly to negative situations, whereas the MET measures empathy for negative as well as for positive emotions to an equal extend. This is also reflected in a trend-level correlation between the PT scale of the IRI and the cognitive task of the MET for negative items. Thus, the cognitive empathy MET score measures an additional dimension (empathy for positive emotions), which is not covered by the IRI at all. This might also contribute to the week correlation of the two measures regarding cognitive empathy.

Our study was designed in line with the new view of mimicry as context-dependent and valence-based by presenting pictures of complex emotions embedded in a context. It could be argued that the static pictures used do not match the rich contexts of real social interactions. While this is true and more research in actual social interaction settings is needed, we used pictures depicting individuals in diverse contexts that show a variety of complex emotions with their faces and with their bodies and gestures. In addition, we used a balanced number of positive and negative emotions while most studies with dynamic stimuli have focused on basic emotions, which only include one positive emotion, i.e. happiness (Hyniewska & Sato, 2015; Sato et al., 2008). Most research on mimicry and emotional empathy focused on negative emotions, e.g. in the context of pain empathy paradigms or by using self-report measures of empathy biased towards negative emotion response options. To our knowledge, this is the first study that gives equal attention to negative and positive emotions. Future research should explore the relationship between mimicry and empathy in real social interactions.

In sum, we found evidence for an important role of mimicry in both emotional and cognitive empathy. The relationship between mimicry and emotional empathy seems stronger than the one between mimicry and cognitive empathy. Yet, with our design we can only report an association and not test for causality. Nonetheless, the present research provides strong evidence for a context-dependent view on mimicry and its affiliative role.

Acknowledgement

We would like to thank Katharina Bergunde and Christoph Klebl for help with data collection. We acknowledge support by the Open Access Publication Fund of Humboldt-Universität zu Berlin.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The research was supported by the Berlin School of Mind and Brain, Humboldt-Universität zu Berlin.