Looking beyond the static face in emotion recognition: The informative case of interest

ABSTRACT

A new wave of studies on emotion recognition encourages researchers to look beyond the face as the sole source of pertinent information. One study has proposed that, while there is overwhelming evidence that negative emotions may be differentiated in static facial expressions, postural information is needed to differentiate positive emotions such as pride, love and amusement. We review the most influential of these recent studies before providing some supporting evidence in an exploratory study of a fourth positive emotion – interest – that adds to the recent calls to move beyond static facial expressions of emotion when investigating how we determine how others are feeling and how they are likely to act next.

KEYWORDS:

• Body emotion recognition
• dynamic emotion recognition
• positive emotion
• facial emotion recognition
• interest

Acknowledgements

The authors would like to thank the participants and students who volunteered for this study and the replication study, the precious help of Nadège Foudon, the two actresses (Pauline Schneider and Sarah-Lise Salomon Maufroy), the director (Paul Walther) and the FACS coders who coded the facial expressions used in this study, Cyrielle Chapuis, Benoît Cano Peuker and Susanne Kaiser.

Disclosure Statement

No potential conflict of interest was reported by the authors.

ORCID

Catherine Audrin http://orcid.org/0000-0003-2905-6000

Notes

1 Although we focus on facial and bodily expressions of interest, it is intriguing to note that in a vocal emotion recognition test of 14 emotions, Banse and Scherer (1996) found interest to be the third most recognized emotion, scoring an impressive 75% behind just happiness (78%) and boredom (76%).

2 A replication study was carried out using a between-subjects design and 27 participants for each of the four conditions. The results of that study are strikingly similar to the results reported here and therefore no further comment will be made about the replication study in this text. The results of the replication study are available on request from the corresponding author.

3 While the actresses were free to express the emotions as they wished, it is clear that the expressions are still not “authentic”. As such, it may be that real-life expressions of interest would bear different results.

4 This was borne out by statistical analysis. We recoded the wrong answers by attributing a “1” to the trials where participants answered “surprise” and a “0” for any other answers. A logistic mixed effect model was performed on this variable in order to test how the probability to incorrectly answer “surprise” rather any other emotion was modulated by the factors of our design; the fixed effects we entered were MOTION and MODALITY in the model and the random effect we entered was the participants. Results showed that the participants were significantly more likely to answer surprise when they made a mistake in the dynamic face condition rather than in the face condition (z = 3.487, p < .001) and in the dynamic body condition rather than in the static body condition (z = 1.471, p < .05).

5 It could be that a dynamic presentation of the expressions did not affect the recognition of all other emotions the same way. For example, from a qualitative analysis of Table 3, it seems that the facial expressions of anger, fear, surprise and sadness benefit from being presented dynamically, while the facial expressions of disgust and joy do not. Meanwhile, in the body conditions, only fear and joy seem to change particularly when represented dynamically – joy is less well recognized while fear is recognized more frequently. Clearly it would be interesting to speculate about the reasons for the specific patterns uncovered here, but testing of these speculations and hypotheses is beyond the scope of the current manuscript. Furthermore, to carry out such analysis would entail the creation of many more stimuli and require testing that would be statistically inappropriate in our experimental design.

6 Surprise has sometimes been argued to be a second positive emotion, but see Noordewier and Breugelmans (2013) for a convincing explanation of why surprise is initially more likely to be negative than positive.

Additional information

Funding

The data collection was carried out at the University of Neuchâtel, Neuchâtel, Switzerland. This work was supported by the National Centre of Competence in Research (NCCR) “Affective Sciences: Emotion in individual Behaviour and Social Processes”, financed by the Swiss National Science Foundation [grant number: SNSF, 51NF40-104897], and hosted by the University of Geneva.