How do eye gaze and facial expression interact?

Abstract

Previous research has demonstrated an interaction between eye gaze and selected facial emotional expressions, whereby the perception of anger and happiness is impaired when the eyes are horizontally averted within a face, but the perception of fear and sadness is enhanced under the same conditions. The current study reexamined these claims over six experiments. In the first three experiments, the categorization of happy and sad expressions (Experiments 1 and 2) and angry and fearful expressions (Experiment 3) was impaired when eye gaze was averted, in comparison to direct gaze conditions. Experiment 4 replicated these findings in a rating task, which combined all four expressions within the same design. Experiments 5 and 6 then showed that previous findings, that the perception of selected expressions is enhanced under averted gaze, are stimulus and task-bound. The results are discussed in relation to research on facial expression processing and visual attention.

Acknowledgements

This work was supported by a Wellcome Grant (GR072308) to Mike Burton, Stefan R. Schweinberger and Stephen Langton.

Notes

¹Inverse efficiency scores were calculated for all experiments (except Experiment 4, which did not produce error data). Analysis of these scores consistently produced the same outcome as reaction time analysis and is not reported further.

²This interpretation receives some support from the statistical analysis. 3×2×2 ANOVAs of block, expression, and gaze did not reveal any significant interactions in Experiment 1 or Experiment 3. In Experiment 2, only a block×expression interaction was found, F(2, 58) = 4.03, p < .05, due to faster responses to happy faces than sad faces in Block 1, F(1, 29) = 10.65, p < .01 (all other interactions, ns). In contrast, Experiment 5 showed interactions of expression and gaze, F(1, 18) = 7.62, p<.01, and, marginally, for block and gaze, F(2, 34) = 3.04, p=.06. An advantage for direct-gaze expressions was found for Block 1, F(1, 17) = 4.92, p < .05, but not for Blocks 2 or 3 (all other interactions, ns).

³We obtained a similar result in a reaction time study with happy, sad, and neutral faces (N=19) to which observers made forced-choice happy/sad decisions. As in Experiments 1 and 2, responses to happy and sad expressions were slower in the averted than the direct gaze conditions (happy: Averted gaze, 704 ms vs. direct gaze, 667 ms; sad: Averted gaze, 899 ms vs. direct gaze, 877 ms). For neutral faces these effects varied as a function of eye gaze and response. Neutral faces classified as happy showed a cost for averted gaze trials (averted gaze, 1181 ms vs. direct gaze, 1145 ms), but this effect reversed when these faces were classified as sad (averted gaze, 900 ms vs. direct gaze, 910 ms). However, this interaction did not approach significance.

⁴We thank an anonymous reviewer for this suggestion. Our sample is drawn from Psychology undergraduates at the University of Glasgow, who are in the majority female. Consequently, we are unable to assess gender differences accurately. Future research should focus on the role of gender in expression and gaze processing.