Person perception from changing emotional expressions: primacy, recency, or averaging effect?

ABSTRACT

Dynamic changes in emotional expressions are a valuable source of information in social interactions. As the expressive behaviour of a person changes, the inferences drawn from the behaviour may also change. Here, we test the possibility that dynamic changes in emotional expressions affect person perception in terms of stable trait attributions. Across three experiments, we examined perceivers’ inferences about others’ personality traits from changing emotional expressions. Expressions changed from one emotion (“start emotion”) to another emotion (“end emotion”), allowing us to disentangle potential primacy, recency, and averaging effects. Drawing on three influential models of person perception, we examined perceptions of dominance and affiliation (Experiment 1a), competence and warmth (Experiment 1b), and dominance and trustworthiness (Experiment 2). A strong recency effect was consistently found across all trait judgments, that is, the end emotion of dynamic expressions had a strong impact on trait ratings. Evidence for a primacy effect was also observed (i.e. the information of start emotions was integrated), but less pronounced, and only for trait ratings relating to affiliation, warmth, and trustworthiness. Taken together, these findings suggest that, when making trait judgements about others, observers weigh the most recently displayed emotion in dynamic expressions more heavily than the preceding emotion.

KEYWORDS:

• Changing emotional expressions
• dominance
• affiliation
• competence
• recency effect

When encountering other people, we immediately try to establish what they are like and how they are likely to act (Frith & Frith, 2006). Such inferences are essential for successful social communication, as they determine what kind of strategies we use to interact with other people and even whether to continue the interaction. For example, if we perceive a person as friendly, we are more likely to help them when they are in trouble; if we perceive someone as dominant and aggressive, we are less likely to choose them as a group member to work on a collaborative project; and if we perceive a person as being trustworthy, we are more inclined to cooperate with them (Cheng, Tracy, Foulsham, Kingstone, & Henrich, 2013; Clark, Pataki, & Carver, 1996; Krumhuber, Manstead, Cosker, et al., 2007; Van Kleef, De Dreu, & Manstead, 2010). In order to make these kinds of inferences about an interaction partner's personality, we make use of multiple sources of information, including facial displays (Oosterhof & Todorov, 2008, 2009), vocal expressions (McAleer, Todorov, Belin, & Larson, 2014; Scherer, 2003), and other behaviours (Fiske, 1980).

A particularly salient source of information in interactions with strangers are (facial) expressions of emotion (Fridlund, 2014; Keltner & Haidt, 1999; Van Kleef, 2009). Research on trait perceptions based on facial emotional expressions in generic, decontextualised settings has shown that observers reliably infer personality traits from certain facial emotional expressions. For example, people with happy expressions are perceived as high in dominance and affiliation; people with angry and disgusted expressions are perceived as high in dominance and low in affiliation; and people with fearful and sad expressions are perceived as low in dominance (Hess, Blairy, & Kleck, 2000; Knutson, 1996; Krumhuber, Manstead, & Kappas, 2007). Compatible effects of emotional expressions on impression formation have been observed in a range of more contextualised social interaction settings, including personal relationships, group decision making, negotiation, customer service, and leadership (e.g. Chen, Jing, & Lee, 2014; Van Kleef, 2016).

Previous research on the effects of emotional expressions on personality trait inferences has almost exclusively employed static emotional expressions or transitions between neutral and full-blown expressions (Hess et al., 2000; Knutson, 1996; Krumhuber, Manstead, Cosker, et al., 2007). Little is known about how person perception is affected by emotional expression changes, that is, expressions changing from one emotion to another. This is surprising given that facial expressions in real-life interactions are typically dynamic, changing from one state to another (Krumhuber, Kappas, & Manstead, 2013).

Dynamic changes in emotional expressions

If we accept that people's emotional expressions tend to be dynamic rather than static, the question arises how changes in emotional displays influence trait judgments. For instance, if a person first expresses anger and then fear, would she or he be perceived as high on dominance based on the anger they first expressed? Or would they be perceived as low in dominance based on the fear they expressed last? Or would they be perceived as intermediate in dominance based on a combination of the anger and fear displays? To address this question, we examine which (implicit) rules perceivers follow in making trait judgments on the basis of dynamic emotional expressions. We consider dynamic facial expressions that change from one emotion (henceforth “start emotion)” to another (henceforth “end emotion)”. Based on previous theoretical and empirical evidence, we consider three (implicit) principles that observers may rely on to form an impression about other people: the primacy effect, the recency effect, and the averaging effect (Anderson, 1971; Glanzer & Cunitz, 1966; McArthur & Baron, 1983). These principles have been widely studied in the literature on (social) information processing. We employ them here to allow for a systematic examination of the effects of dynamic emotional expressions on person perception.

A primacy effect occurs when the first information that is processed in a sequence of information units has a particularly pronounced influence on a person's subsequent judgments (e.g. Asch, 1946; Anderson, 1965; Anderson & Barrios, 1961; Forgas, 2011). In the context of person perception, if traits are presented in serial order, the traits that were presented first would thus have a greater influence on the overall impression of the target person than traits that were presented later. There is evidence to support a primacy effect in social perception. For example, a person who was first described as industrious and then as stubborn were judged more positively than one who was first described as stubborn and then as industrious (Asch, 1946). In terms of the effects of dynamic changes in emotional expressions in relation to person perception, the primacy effect would translate into perceivers’ inferences being disproportionately driven by the expresser's start emotion.

In contrast, a recency effect occurs when the information that is presented last in a series of information units has a particularly marked influence on a person's subsequent judgments. Recency effects have been found in research on persuasion, with participants’ attitudes being swayed more in the direction of the second of two sequentially presented persuasive messages (e.g. Bateman & Remmers, 1941; Crano, 1977; Miller & Campbell, 1959). If this principle applies in the context of trait inferences from changing emotional expressions, trait inferences should be driven primarily by an expresser's end emotion. Recent findings by Hareli, David, and Hess (2016) lend some support to this prediction. Hareli and colleagues asked participants to judge expressers’ personality traits (dominance and affiliation) from static emotional expressions (static facial expressions of anger and sadness) as well as from changing emotional expressions (anger-to-sadness and sadness-to-anger). The results showed that expressions ending with anger (i.e. static anger and sadness-to-anger) were perceived as more dominant and less affiliative than those ending with sadness (i.e. static sadness and anger-to-sadness). This suggests that the end emotion dominates personality trait judgments. Hareli and colleagues found inconsistent evidence for an effect of the start emotion, with the information of start emotion being integrated in some cases (e.g. anger-to-sadness in Study 1), but not in others (e.g. sadness-to-anger in Study 1). Taken together, their findings point to a recency effect in person perception from dynamic expressions.

Finally, an averaging effect occurs when information units that are presented sequentially are combined to create an integrated judgment. This effect, too, has been investigated in the context of impression formation (Anderson, 1965; Chernev & Gal, 2010; Hendrick, 1968). In Anderson’s (1965) study, for example, participants rated their liking of persons described by sets of two or four personality trait terms, including extremely or moderately favourable or unfavourable adjectives. The results showed that participants produced less extreme responses when rating a set of two moderate and two extreme adjectives than when rating a set of two extreme adjectives alone, which points to an averaging effect. If the averaging principle accounts for trait inferences from dynamic emotional expressions, the resultant perception should be an average of the expresser's start and end emotions. Filipowicz, Barsade, and Melwani (2011) provided some evidence supporting this notion: Compared to those who expressed either anger or happiness throughout a negotiation, individuals whose expressions changed between happiness and anger were rated as intermediate in positivity (also see Pietroni, Van Kleef, De Dreu, & Pagliaro, 2008). However, the emotional changes in these studies were depicted by displaying two still photographs, which may not be representative of genuine dynamic changes.

To date, only two studies have investigated personality trait inferences from changing emotional expressions. The results of one study point to a recency effect (Hareli et al., 2016), whereas the results of the other point to an averaging effect (Filipowicz et al., 2011). Two possible reasons may account for this inconsistency. Firstly, the emotional change in Filipowicz et al.'s study consisted of two still pictures which were temporally separated by a negotiation task. In Hareli et al.'s study, the emotional change consisted of two successive pictures (Study 1) or morphs between two pictures (Study 2). The long time interval between the two expressions in Filipowicz et al.'s study might have made participants rely less on visual-perceptual processing than participants in Hareli et al.'s study. Secondly, the inconsistency in results may reflect a difference in the personality traits examined in the two studies. Specifically, Hareli and colleagues measured dominance and affiliation, while Filipowicz and colleagues measured agreeableness. It is possible that effects vary across trait judgments. For example, a recency effect could occur in dominance judgments, while an averaging effect may occur in trustworthiness judgments. In the present research, we set out to test person perception from dynamic expressions that simulate the natural emotional changes we come across in daily life. We tested the primacy, averaging, and recency accounts in relation to inferences of personality traits derived from three influential models of person perception. This allowed us to determine whether perceivers use the same strategy to infer various different traits, or alternatively, whether they adopt distinct strategies to infer different traits.

Accounts of personality trait inferences

There are many traits that people can use to evaluate another person. Are they dominant, aggressive, friendly, or trustworthy? The basic features of person perception are widely believed to revolve around two overarching dimensional structures (Fiske, Cuddy, & Glick, 2007; Fiske, Cuddy, Glick, & Xu, 2002; Oosterhof & Todorov, 2008; Wiggins, 1979). We discuss three prominent models in the person perception domain, and draw on each of these models to examine the effects of changing emotional expressions on person perception.

The first model of person perception was proposed by Wiggins and colleagues (Wiggins, 1979; Wiggins, Trapnell, & Phillips, 1988). They argued that all of the personality traits relevant to social interaction can be mapped onto a two-dimensional circumplex bisected by dominance and affiliation. These two dimensions were developed from approximately 4000 relatively familiar trait-descriptive terms identified by Norman (1963). According to this model, if emotional expressions convey some fundamental personality traits of the expresser, these should include dominance and affiliation. Indeed, emotional expressions have been found to convey dominance and affiliation to perceivers (e.g. Hess et al., 2000; Knutson, 1996). In a typical paradigm, participants are presented with different facial expressions of emotion and asked to judge them in terms of the expresser's dominance and affiliation. Using this approach, it has been shown that participants reliably infer dominance and affiliation from emotional expressions, even when these occur in the absence of any contextual information that might help disambiguate the meaning of the expressions. For example, people who produce facial expressions of anger are perceived as high in dominance and low in affiliation (Hess et al., 2000; Knutson, 1996; Krumhuber, Manstead, Cosker, et al., 2007).

A second proposal regarding the elemental dimensions underlying person perception was made by Fiske and colleagues (Fiske et al., 2002, 2007). They suggested that person perception is primarily driven by inferences of the sender's competence and warmth. According to this account, evolutionary pressures are reflected in person perception, such that when people meet others, they must determine the other's intentions (relating to warmth) and their ability to act on those intentions (relating to competence). It is well established that descriptive information (e.g. a description of a person or a group) leads to consistent inferences about competence and warmth (Fiske et al., 2002, 2007; Judd, James-Hawkins, Yzerbyt, & Kashima, 2005). Research systematically examining perceptions of competence and warmth in relation to nonverbal expressions of emotion is scarce, but several studies indicate that observers do make consistent inferences of competence from structural information of faces (Chen et al., 2014; Na, Kim, Oh, Choi, & O’Toole, 2015; Tiedens, 2001; Todorov, Mandisodza, Goren, & Hall, 2005).

A third proposal for the dimensions underlying person perception is primarily based on the study of face perception (Oosterhof & Todorov, 2008; Todorov, Said, Engell, & Oosterhof, 2008). This view considers trustworthiness and dominance central to trait inferences from faces. To identify the dimensions underlying person perception, Todorov et al. (2008) selected the most frequently used trait dimensions from unconstrained person descriptions of neutral faces. The faces were then judged on these trait dimensions by a different group of participants. When these judgments were submitted to a principle components analysis (PCA), two principle components were obtained: trustworthiness and dominance. This was taken to mean that judgments of trustworthiness and dominance can be used as approximations of the underlying dimensions of face evaluations. These two dimensions have in recent years been shown to be robust in studies of face perception (e.g. Chen et al., 2014; Dotsch, Hassin, & Todorov, 2016; Stirrat & Perrett, 2010).

The current study

The current study followed the classic decontextualised person perception paradigm developed by Knutson (1996), and aimed to investigate how people infer personality traits from expressions that change from one emotion to another. We thus kept the design similar to Knutson’s study (showing faces in the absence of specific contextual information), with the exception of using dynamic rather than static emotional expressions. This allowed us to directly compare our findings to those of Knutson and others who used similar paradigms (e.g. Hess et al., 2000). We sought to establish whether trait inferences based on dynamic emotional expressions are driven primarily by a primacy effect, a recency effect, or an averaging effect. In order to examine the respective influences of start and end emotions, we utilised comparisons of different dynamic expressions. For instance, the comparison of anger-to-disgust and fear-to-disgust indicated the effect of start emotion (anger vs. fear), whereas the comparison of anger-to-disgust and anger-to-fear indicated the effect of end emotion (disgust vs. fear). By adopting this methodology, we eliminated possible confounds in comparisons between static and dynamic expressions. For example, the effect of the start emotion sadness in Hareli et al.’s study was examined by comparing a static expression of anger with a dynamic change from sadness to anger. This makes it impossible to infer whether any difference between these conditions is due to the start emotion or the difference between static and dynamic expressions, considering that emotional expressions derive part of their informational value from the fact that they change (Frijda, 1986; Krumhuber et al., 2013; Kuppens, Oravecz, & Tuerlinckx, 2010; Scherer, 2009; Van Kleef, 2016).

The dynamic expressions in the present research consisted of changes between any two of the facial expressions of anger, disgust, and fear, that is, anger-to-disgust, disgust-to-anger, anger-to-fear, fear-to-anger, disgust-to-fear, and fear-to-disgust. The use of these emotions, which are all high in arousal and negative in valence, avoided possible confounding effects of arousal and valence. Importantly, these emotions do differ reliably in the perceived traits. Individuals who show anger and disgust are perceived similarly in personality traits, as they both indicate high dominance and low affiliation, warmth, and trustworthiness. In contrast, individuals who express fear are perceived as less dominant, but more affiliative and trustworthy than those who express anger or disgust (Aviezer et al., 2008; Gutiérrez-García & Calvo, 2016; Knutson, 1996; Oosterhof & Todorov, 2009). Notably, we need at least three emotions to develop various dynamic expressions in order to prevent participants from being able to predict the end emotion from the start emotion, and to allow us to conduct different comparisons in order to separate the effects of start and end emotions.

We conducted three experiments, measuring dominance and affiliation in Experiment 1a, competence and warmth in Experiment 1b, and dominance and trustworthiness in Experiment 2. Based on the theoretical considerations and the evidence reviewed above, we developed three competing hypotheses:

Hypothesis 1: The start emotion of dynamic expressions dominates personality trait inferences, that is, a primacy effect occurs in trait inferences from changing expressions.

Hypothesis 2: The end emotion of dynamic expressions dominates personality trait inferences, that is, a recency effect occurs in trait inferences from changing expressions.

Hypothesis 3: Both start and end emotions shape personality trait inferences, such that an averaging effect occurs in trait inferences from changing expressions.

Note that we did not advance separate hypotheses about specific pairs of emotional expressions. We did not see a theoretical basis in the literature to presume that different types of effects (i.e. primacy, recency, or averaging) would occur for different pairs of emotions. Rather, we tested whether—across the board—trait inferences on the basis of dynamic emotional expressions can be best understood in terms of primacy, recency, or averaging effects, regardless of the particular emotional transition in question. Note, however, that our analytical approach enabled us to pick up on any differences between emotion pairs if they were to occur, as explained below.

Analytical approach

Since anger and disgust are rated similarly on all the target traits except competence (Aviezer et al., 2008; Gutiérrez-García & Calvo, 2016; Knutson, 1996; Oosterhof & Todorov, 2009), the six dynamic expressions were grouped into three categories (anger-to-disgust/disgust-to-anger [anger-to-disgust and disgust-to-anger], anger/disgust-to-fear [anger-to-fear and disgust-to-fear], and fear-to-anger/disgust [fear-to-anger and fear-to-disgust]) for further analysis. To test the hypotheses, we first conducted a hierarchical cluster analysis on the three categories of dynamic expressions using the ratings for each personality trait (we also conducted a hierarchical cluster analysis on the six categories of dynamic expressions, which yielded similar conclusions; see supplementary Figure S1 for details). We opted for a hierarchical cluster analysis instead of a mixed-design ANOVA or a linear mixed model analysis due to the nature of the hypotheses and stimuli. Specifically, in order to examine the respective effects of start and end emotion, the factors in an ANOVA would have been Start Emotion (anger, disgust, fear) and End Emotion (anger, disgust, fear). This would require nine conditions: anger-to-anger, anger-to-disgust, anger-to-fear, disgust-to-anger, disgust-to-disgust, disgust-to-fear, fear-to-anger, fear-to-disgust, and fear-to-fear. However, anger-to-anger, disgust-to-disgust, and fear-to-fear were not included in the present study because they would have been static expressions. We could therefore could not employ an ANOVA or linear mixed model analysis.

If H1 (a primacy effect) were true, then dynamic expressions with the same start emotion should be classified into one cluster. Specifically, this would mean that expressions changing from fear (fear-to-anger/disgust) would form one cluster, while expressions changing from anger/disgust (anger-to-disgust/disgust-to-anger and anger/disgust-to-fear) would form another cluster. If H2 (a recency effect) were true, then dynamic expressions with the same end emotion should be classified into one cluster, that is, the expressions changing to fear (anger/disgust-to-fear) would be in one cluster, and the expressions changing to anger/disgust (anger-to-disgust/disgust-to-anger and fear-to-anger/disgust) would be in another cluster. If H3 (an averaging effect) were true, then dynamic expressions with the same emotions, irrespective of the direction of the change, would be grouped together. That is, the expressions changing between anger and disgust (anger-to-disgust/disgust-to-anger) would fall into one cluster, and the expressions changing between fear and anger/disgust (anger/disgust-to-fear and fear-to-anger/disgust) would form another cluster.

We used a hierarchical cluster analysis that produces a set of nested clusters organised in a hierarchical tree. Different from other clustering procedures (e.g. K-means), the number of clusters is determined by the structure of the data, rather than being pre-specified. Thus, the current analytical approach let the data indicate whether and how the various dynamic emotional expressions clustered together. This means that, even though we did not predict differential effects of start versus end emotions across emotion pairs, our analytical procedure allowed us to detect such effects in case they occurred. The cluster analysis was followed by two planned contrasts (paired t-tests with a Bonferroni-corrected alpha level of .025) for each dependent measurement, with Contrast 1 comparing the two clusters, and Contrast 2 comparing the categories of dynamic expressions within the same cluster.

Experiment 1a

The goal of this experiment was to examine the inferences individuals draw from changing emotional expressions in terms of the personality dimensions proposed by Wiggins and colleagues (Wiggins, 1979; Wiggins et al., 1988), that is, inferences of dominance and affiliation.

Method

Participants

One hundred students (M_age = 20.72, SD = 2.20; 78 female) from the University of Amsterdam volunteered to participate in return for course credits. All participants provided written informed consent, and the ethics committee of the University of Amsterdam approved the experiment.

Stimuli and apparatus

The tests were programmed using a custom-written Psychopy programme (Psychophysics software in Python; Peirce, 2007), and implemented on a Windows 7 computer. The monitor was 24-inch, with a screen resolution of 1920 × 1080 pixels, and a screen refresh rate of 60 Hz. Twelve images of Dutch actors (six female, six male) posing facial expressions of anger, disgust, and fear were selected from the Radboud Faces Database (Langner et al., 2010). We used Fantamorph5 (http://www.fantamorph.com/) to produce dynamic expressions, which changed from a start emotion to an end emotion for the same individual. The 26-frame morphs were presented at the speed of 30 frames per second, which has been found to adequately reflect natural changes in dynamic facial expressions (Ambadar, Schooler, & Cohn, 2005; Sato & Yoshikawa, 2004). Based on the unfolding time of natural dynamic emotional expressions (Hoffmann, Traue, Bachmayr, & Kessler, 2010), the exposure time of the first frame (i.e. the start emotion) and the last frame (i.e. the end emotion) were extended to 600 ms each. Thus, each clip lasted for 2000 ms. In total, 72 morphs (6 actors × 2 genders × 6 dynamic emotional expressions) were included.

Dependent measures and procedure

The dominance and affiliation items were adapted from the Revised Interpersonal Adjective Scale (Wiggins et al., 1988). Judgments were made on Likert scales ranging from 1 (extremely inaccurate) to 7 (extremely accurate). The dominance scale consisted of the four items dominant, self-assured, assertive, and self-confident (α = .71), and the affiliation scale consisted of the four items gentle, agreeable, tender, and sympathetic (α = .82).

Each trial started with a fixation cross, displayed in the centre of the screen for 500 ms, followed by a movie clip of a dynamic expression for 2000 ms. The clip disappeared immediately after it had finished playing. Participants were then asked to rate the actor on two items, with one tapping dominance and the other affiliation. Four blocks were included, and one of the four items from each scale was randomly assigned to one of the four blocks. Each block consisted of 12 trials, in which a male and a female actor appeared once with one of the six different dynamic expressions. The pairs of actor-expressions were randomly designated, consistent across blocks for each participant, but different across participants. In this way, each participant saw only one dynamic expression of each actor, ensuring that trait judgments would not be biased by prior judgments of the same actor with different expressions. Participants received four practice trials (with different actors) to familiarise themselves with the task before encountering the formal trials. Each participant thus completed a total of 48 experimental trials.

Results and discussion

The results of the hierarchical cluster analyses are presented in Figure 1a and b, and the means and standard deviations of the ratings of the three combined categories of dynamic expressions are presented in Table 1 (the means and standard deviations of the ratings of the six original categories of dynamic expressions are presented in supplementary Table S1).

Figure 1. Dendrogram of hierarchical cluster analyses (Euclidean distance matrix using Ward's linkage method) of the three categories of dynamic expressions for trait ratings in Experiments 1a, 1b, and 2. The height denotes the dissimilarities between clusters, with greater height indicating greater differences between clusters. Dynamic expressions with the same end emotion were perceived to be more similar to each other than dynamic expressions with different end emotions.

Table 1. Means and standard deviations for dynamic emotional expressions in Experiments 1a, 1b, and 2.

Experiment	Dependent measure	Contrast 1 (between clusters): Effect of end emotion		Contrast 2 (within Cluster 1): Effect of start emotion
		Cluster 1 (Expressions changing to anger/disgust)	Cluster 2 (Expressions changing to fear)	Anger-to-disgust/disgust-to-anger	Fear-to-anger/disgust
Experiment 1a	Dominance	3.99(0.67)a	3.50(0.74)b	3.96(0.77)a	4.03(0.72)a
	Affiliation	3.00(0.83)a	3.97(0.88)b	2.91(0.84)a	3.09(0.87)b
Experiment 1b	Warmth	3.23(0.82)a	4.03(0.80)b	3.39(0.88)a	3.61(0.85)b
Experiment 2	Dominance	4.15(0.59)a	3.32(0.76)b	4.23(0.66)a	4.08(0.68)a
	Trustworthiness	3.60(0.69)a	4.24(0.77)b	3.45(0.67)a	3.76(0.79)b
		Cluster 1 (Expressions changing to disgust/fear)	Cluster 2 (Expressions changing to anger)	Disgust-to-fear/fear-to-disgust	Anger-to-disgust/fear
Experiment 1b	Competence	3.36(0.81)a	3.74(0.92)b	3.32(0.83)a	3.41(0.87)a

Note: Based on the results of the cluster analysis, two contrasts (paired t-tests with Bonferroni-corrected alpha level at .025) were conducted for each dependent measurement. Numbers with different subscripts differ at p < .025.

Dominance

The cluster analysis for dominance yielded two clusters, with expressions changing to anger/disgust in Cluster 1 (anger-to-disgust/disgust-to-anger and fear-to-anger/disgust) and expressions changing to fear in Cluster 2 (anger/disgust-to-fear). This result supports a recency effect in dominance judgments of dynamic emotional expressions. A paired t-test showed that expressions changing to anger/disgust were judged as higher in dominance than expressions changing to fear, t(99) = 7.29, p < .001, Cohen's d = 0.73. To test for an effect of start emotion, a paired t-test was conducted within Cluster 1 (no paired t-test was conducted within Cluster 2 because it only contained one emotion category). The t test within Cluster 1 revealed that the dominance ratings of anger-to-disgust/disgust-to-anger did not differ from those of fear-to-anger/disgust, t(99) = 1.32, p = .382, Cohen's d = 0.13. There was thus no evidence suggesting that the start emotion (anger/disgust vs. fear) influenced the dominance ratings.

Affiliation

The result of the cluster analysis for affiliation also supported a recency effect, with expressions changing to fear (Cluster 1: anger/disgust-to-fear) being judged as more affiliative than expressions changing to anger/disgust (Cluster 2: anger-to-disgust/disgust-to-anger and fear-to-anger/disgust), t(99) = 12.64, p < .001, Cohen's d = 1.27. Within Cluster 2, fear-to-anger/disgust was judged as more affiliative than anger-to-disgust/disgust-to-anger, t(99) = 3.75, p < .001, Cohen's d = 0.38, suggesting that the start emotion (fear vs. anger/disgust) did exert an influence on the affiliation ratings. However, the effect of start emotion was weaker than that of end emotion, as evidenced by the fact that the initial cluster analysis yielded two clusters grouped by end emotions rather than start emotions.

In sum, the results of Experiment 1a indicate a recency effect in both dominance and affiliation ratings of dynamic emotional expressions. However, while start emotional expressions were to some degree taken into account in the affiliation ratings, they did not affect dominance ratings. There was no evidence for an averaging effect in perceived dominance or affiliation from dynamic expressions (as the cluster analyses showed clusters grouped by end emotions rather than clusters grouped by dynamic expressions with the same emotions).

Experiment 1b

Experiment 1b aimed to examine the inferences perceivers draw from changing emotional facial expressions in terms of the personality dimensions proposed by Fiske and colleagues (Fiske et al., 2002, 2007), namely competence and warmth.

Method

Participants

The participants who participated in Experiment 1a also participated in Experiment 1b. The two tasks were separated by a 5-min filler task. One participant was excluded from the analysis for failing to complete the experiment. This left 99 participants (M_age = 20.68, SD = 2.17; 77 female) for this experiment.

Dependent measures and procedure

The competence and warmth items were derived from the scale used by Fiske et al. (2002), with the competence scale consisting of competence, intelligence, skilfulness, and ability (α = .90), and the warmth scale consisting of warmth, friendliness, sincerity, and good-naturedness (α = .85). Judgments ranged from 1 (extremely inaccurate) to 7 (extremely accurate). Otherwise the procedure was identical to Experiment 1a.

Results and discussion

Since there is no evidence showing that the facial expressions of anger and disgust are perceived similarly in terms of competence, we did not collapse anger and disgust for the analysis of the competence ratings. Instead, we used the six original categories of dynamic expressions. The results of the hierarchical cluster analyses are presented in Figure 1c and d, and the means and standard deviations of the ratings of the three combined categories of dynamic expressions are presented in Table 1 (the means and standard deviations of the ratings of the six original categories of dynamic expressions are presented in supplementary Table S1).

Competence

The analysis yielded two clusters, with expressions changing to disgust/fear in Cluster 1 (anger-to-disgust, fear-to-disgust, anger-to-fear, and disgust-to-fear), and expressions changing to anger in Cluster 2 (disgust-to-anger and fear-to-anger). This result is consistent with H2, which predicted a recency effect. Furthermore, a paired t-test showed that expressions changing to anger were judged as more competent than expressions changing to disgust/fear, t(98) = 5.09, p < .001, Cohen's d = 0.51. Together this suggests that disgust and fear were perceived similarly in terms of competence, but that both were perceived differently from expressions of anger. We therefore created three new emotion categories, collapsing disgust and fear: disgust/fear-to-anger (disgust-to-anger and fear-to-anger), anger-to-disgust/fear (anger-to-disgust and anger-to-fear), and disgust-to-fear/fear-to-disgust (disgust-to-fear and fear-to-disgust). To test for an effect of start emotion, we conducted a paired t-test within Cluster 1 (no paired t-test was conducted within Cluster 2 because it only contained one emotion category of disgust/fear-to-anger). The results showed no difference between anger-to-disgust/fear and disgust-to-fear/fear-to-disgust, t(98) = 1.71, p = .182, Cohen's d = 0.17. There was thus no evidence that the start emotion influenced the competence ratings. In sum, a recency effect was found, with expressions changing to anger being perceived as more competent than expressions changing to disgust and fear.

Warmth

The findings on the warmth ratings replicated the affiliation ratings in Experiment 1a. Specifically, expressions changing to fear (Cluster 1: anger/disgust-to-fear) were judged as warmer than expressions changing to anger/disgust (Cluster 2: anger-to-disgust/disgust-to-anger and fear-to-anger/disgust), t(98) = 12.38, p < .001, Cohen's d = 1.24. Furthermore, fear-to-anger/disgust was judged as warmer than anger-to-disgust/disgust-to-anger, t(98) = 3.90, p < .001, Cohen's d = 0.39. Taken together, this suggests that both start and end emotions influenced warmth ratings, though the effect of end emotions was stronger than that of start emotions.

Together, the results of Experiment 1b indicate a recency effect in both competence and warmth ratings of dynamic facial expressions. Start emotions did not affect competence ratings, but did affect warmth ratings, though less strongly than end emotions. No evidence for an averaging effect was found for perceived competence or warmth from changing expressions.

Experiment 2

In Experiment 2 we examined the inferences perceivers draw from changing emotional expressions with regard to the personality dimensions proposed by Oosterhof and Todorov (2008), that is, inferences of dominance and trustworthiness.

Method

Participants

Eighty-three participants (M_age = 24.99, SD = 9.66; 60 female) from the University of Amsterdam took part, receiving course credits or money for participation. All participants provided written informed consent, and the ethics committee of the University of Amsterdam approved the experiment.

Dependent measures and procedure

The dominance and trustworthiness items were derived from the scale used by Chen et al. (2014), with the dominance scale consisting of dominant, assertive, and forceful (α = .81), and the trustworthiness scale consisting of trustworthy, honourable, and honest (α = .83). Judgments ranged from 1 (extremely inaccurate) to 7 (extremely accurate). The procedure was identical to Experiments 1 and 2.

Results and discussion

The results of the hierarchical cluster analyses are presented in Figure 1e and f, and the means and standard deviations of the ratings of three combined categories of dynamic expressions are presented in Table 1 (the means and standard deviations of the ratings of the six original categories of dynamic expressions are presented in supplementary Table S1).

Dominance

The findings on perceived dominance replicated the dominance ratings in Experiment 1a. Specifically, expressions changing to anger/disgust (Cluster 1: anger-to-disgust/disgust-to-anger and fear-to-anger/disgust) were judged as more dominant than expressions changing to fear (Cluster 2: anger/disgust-to-fear), t(82) = 9.94, p < .001, Cohen's d = 1.09. No significant differences between anger-to-disgust/disgust-to-anger and fear-to-anger/disgust were found, t(82) = 2.14, p = .070, Cohen's d = 0.23, suggesting that the start emotion (start fear vs. anger/disgust) had no notable influence on the dominance ratings.

Trustworthiness

The findings on perceived trustworthiness mapped onto those for affiliation in Experiment 1a and warmth in Experiment 1b. Specifically, expressions changing to fear (Cluster 1: anger/disgust-to-fear) were judged as more trustworthy than expressions changing to anger/disgust (Cluster 2: anger-to-disgust/disgust-to-anger and fear-to-anger/disgust), t(82) = 7.75, p < .001, Cohen's d = 0.85. Further, fear-to-anger/disgust was judged as more trustworthy than anger-to-disgust/disgust-to-anger, t(82) = 5.26, p < .001, Cohen's d = 0.58. Thus, judgments on trustworthiness, like those on affiliation and warmth, were more strongly influenced by the end emotion than by the start emotion.

Together, the results of Experiment 2 indicate a recency effect in both dominance and trustworthiness inferences from dynamic expressions. In addition, the start emotion did not affect the dominance ratings, but was integrated to some degree in the trustworthiness ratings. Again, no evidence for an averaging effect was found.

General discussion

Across three experiments we examined observers’ inferences about others’ personality traits from changing facial expressions of emotion. In each of the experiments, we measured two dimensions of personality traits from one of three influential person perception models: dominance and affiliation in Experiment 1a, competence and warmth in Experiment 1b, and dominance and trustworthiness in Experiment 2. A strong recency effect was found across all trait judgments, that is, the end emotion of the dynamic expressions had a marked impact on all of the trait ratings. The start emotion was integrated to a lesser degree, and only in the personality trait ratings of affiliation, warmth, and trustworthiness. Taken together, these findings suggest that the most recently expressed emotion in a dynamic expression dominates person perception, while preceding emotions have weak or null effects that vary across personality dimensions.

In the current study, we used a data-driven approach (specifically a hierarchical cluster analysis) to analyse trait ratings based on the various dynamic emotional expressions, and we consistently found that dynamic expressions with the same end emotion (rather than dynamic expressions with the same start emotion or dynamic expressions involving the same emotions irrespective of the direction of the change) formed clusters. This suggests that although each dynamic expression is distinct, expressions that end with the same emotions do share similar features in terms of the information that observers use to infer personality traits.

Our finding of a main role of end emotion is noteworthy in light of classical research on impression formation, which suggests that the overall impression of a target person is often subject to a primacy effect (Asch, 1946). This result may be explained by the distinctiveness of emotional change. When confronted with an emotional change, perceivers attempt to make sense of it. We consistently found that the end emotion was more important for perceivers, presumably because the end emotion indicated the direction of the change and provided the most current information.

Based on the role of end and start emotions in trait inferences, three clusters of personality traits can be distinguished: (a) dominance; (b) affiliation, warmth, and trustworthiness; and (c) competence. The perception of dominance was examined in Experiments 1a and 2. Across both experiments, dominance ratings were only influenced by the end emotion, with expressions changing to anger or disgust being judged as more dominant than expressions changing to fear. This result is consistent with Hareli and colleagues’ finding that the end emotion in a dynamic expression has a strong impact on dominance judgments. This consistency in results across studies suggests that this finding is robust. For judgments of dominance, start emotions had no significant impact in the current study.

Previous research has found that affiliation, warmth, and trustworthiness are strongly correlated with each other, and all of them relate to the dimension of valence (Lucas, Diener, Grob, Suh, & Shao, 2000; Oosterhof & Todorov, 2008). This fits well with the results of the current study, in which the results for these three traits were highly consistent. Unlike the other judgments, ratings of affiliation, warmth, and trustworthiness were all influenced by both end and start emotions. The cluster analyses for the affiliation, warmth, and trustworthiness ratings consistently yielded two broad clusters grouped by end emotions. Expressions changing to fear (anger/disgust-to-fear) were judged as warmer, more affiliative, and more trustworthy than expressions changing to anger or disgust (anger-to-disgust/disgust-to-anger and fear-to-anger/disgust). This suggests that the end emotion dominates the start emotion in influencing trait judgments relating to affiliation, warmth, and trustworthiness. Nevertheless, further comparisons within the expressions changing to anger/disgust revealed significant differences, with fear-to-anger/disgust being judged as warmer, more affiliative, and more trustworthy than anger-to-disgust/disgust-to-anger, suggesting that the start emotion also influenced these trait ratings. It is worth noting, however, that the effects of start emotions (Cohen's ds ranging from 0.13 to 0.58) were substantially weaker than the effects of end emotions (Cohen's ds ranging from 0.73 to 1.27). Taken together, these results indicate that when making trait judgments relating to affiliation, warmth, and trustworthiness on the basis of dynamic emotional expressions, observers integrate information from both end emotion and start emotion, but that they primarily rely on the end emotion.

Why might it be that an effect of start emotions was found on judgments of affiliation, warmth, and trustworthiness, but not on judgments of dominance? One possibility is that this pattern of results reflects differences in the process of making these kinds of judgments. Judgments of dominance are more closely related to inferences about behavioural tendencies, such as approach or avoidance, than other trait judgments (Biggers & Rankis, 1983). Since the most recent expression is likely most predictive of subsequent actions, dominance inferences may thus be particularly sensitive to incorporating the most recently available information from expressers. In contrast, when inferring other's traits on dimensions such as warmth, affiliation, and trustworthiness, observers are less likely to seek to predict future actions, and may therefore also integrate information from preceding emotional expressions. Nevertheless, all trait judgments were primarily driven by the most recently displayed emotion.

We also found a recency effect for judgements of perceived competence from dynamic expressions. Given that Experiments 1a and 1b were conducted together, we sought to establish whether task order influenced the results. To this end we conducted the same analysis three times for each trait rating in Experiments 1a and 1b: Once with the participants who completed Experiment 1a first, once with the participants who completed Experiment 1b first, and once with the combined participants (for details, see supplementary Figure S2). The recency effect held for all of the trait ratings except competence. Specifically, the recency effect for perceived competence was found in the combined sample as well as among the participants who completed Experiment 1a first. However, among the participants who completed Experiment 1b first, no recency effect for perceived competence was found. These results suggest that the recency effect for perceived competence was not as robust as that for perceived dominance, affiliation, and warmth. This may be due to the fact that the relations between the current emotional expressions (anger, disgust, and fear) and perceived competence were not as robust as the relations between the emotional expressions and perceived dominance, affiliation, and warmth (e.g. Aviezer et al., 2008; Hess et al., 2000; Knutson, 1996; Todorov et al., 2005).

In addition, given the conceptual overlap between the measure of affiliation in Experiment 1a and the measure of warmth in Experiment 1b, we exploratively combined these two measures into a single index of affiliation/warmth and reran the analysis using this combined measure. The results were similar to the results for affiliation in Experiment 1a and those for warmth in Experiment 1b (for details, see supplementary Figure S3; note that we did not perform this exercise for the measures of dominance and competence in Experiments 1a and 1b as these are not conceptually related; see e.g. Chen et al., 2014).

Limitations and future directions

A number of limitations of the current study are worth mentioning. Firstly, the dynamic emotional expression morphs we employed in the current research may not perfectly reflect the natural emotional changes that we come across in real life. For example, research has shown that non-linear movements are generally perceived as more natural than linear ones (which were used in the present research; Cosker, Krumhuber, & Hilton, 2010). Despite this potential shortcoming, morphs have clear merits, such as the easily adjustable rate of change and the lack of noise, and accordingly they are commonly used in dynamic emotion research (e.g. Krumhuber & Scherer, 2016; Sato & Yoshikawa, 2004). Nevertheless, this methodology could be improved in future research by using videos of changing emotional expressions as they happen in real time. Even greater ecological validity could be obtained by examining emotional expressions occurring in ongoing interactions. Future studies should establish whether the current results generalise to such genuine dynamic expressions.

Secondly, in order to establish a fundamental and generic effect of emotional changes on perceived traits, the current study was conducted in the absence of specific contextual information. Previous research suggests that context may influence how people perceive emotions (for a review, see Wieser & Brosch, 2012). Research on trait inferences from emotional facial expressions, however, has documented consistent patterns of trait inferences from emotional expressions across contexts (e.g. Brescoll & Uhlmann, 2008; Van Kleef, 2016), that is, similar trait inferences from discrete emotional expressions have been observed across situations. Moreover, the only study we are aware of that has examined trait inferences from emotional changes in both contextualised and decontextualised paradigms found highly similar results in both cases (Hareli et al., 2016). Therefore contextual information may not necessarily play a central role in trait inferences from dynamic expressions. Nevertheless, greater ecological validity could be obtained by examining the effects of emotional changes in real-life situations that include contextual information.

Thirdly, we only examined three emotions (anger, disgust, and fear), all of which are high in arousal and negative in valence. This allowed us to eliminate potential confounds, but it cannot be ruled out that our findings are limited to negative, high arousal emotions.

Finally, although we demonstrated that observers’ inferences about the expresser's personality traits were heavily influenced by the end emotion in dynamic expressions, observers’ inferences in real life are of course much more complex. Beliefs about the causes of emotional changes, and knowledge of the situation and/or the expresser, are likely to have an effect not only on judgments of the expresser's personality traits, but also on how observers use the trait information that they infer.

Conclusions

In sum, the current findings suggest a recency effect in trait judgments from dynamic emotional expressions, that is, the end emotion of dynamic expressions has a particularly pronounced impact on inferences of the expresser's personality. Preceding emotional expressions were integrated, but to a lesser degree, and only for some traits (affiliation, warmth, and trustworthiness). Taken together, these results indicate that observers weigh the end emotion of dynamic expressions more heavily than the start emotion when making trait judgements about others.

There is a growing trend of using more dynamic expressions to study emotion perception, and this research has revealed differences between static and dynamic expressions (Hareli et al., 2016; Jellema, Pecchinenda, Palumbo, & Tan, 2011; Krumhuber et al., 2013; Marian & Shimamura, 2013). However, little attention has been paid to social perceptions based on dynamic signals, that is, how observers use changes in others’ emotional facial expressions to arrive at personality judgments. The current study contributes to this line of research by investigating personality trait inferences from changing emotional expressions, demonstrating a dominant role for the most recent expression across a range of trait inferences.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was facilitated by a scholarship from the China Scholarship Council awarded to the first author, and grants from the Netherlands Organisation for Scientific Research (NWO) awarded to the second (452-09-010) and third (275-70-033) authors.