People are a basic and fundamental focus of human perception. Our personal, social, and emotional lives are defined by the people with whom we interact and the quality and quantity of these interactions. Person perception systems in the brain are exquisitely and diversely tuned, not only to human faces, but also to bodies, actions, and voices. The information we perceive about a whole person from sensory input serves multiple adaptive functions that guide us through our social world. The perceptual system orients spontaneously to the presence of other people in the sensory environment—through visual and voice contact. What follows is a cascade of perceptual analyses that tell us about the person. Do we know them? Are they male or female? Old or young? Happy or sad? Do they want to greet us, engage with us, or avoid us? For most people, the process of person perception is spontaneous and unfolds across time and space.
Psychological investigations of person perception have barely scratched the surface of the complex and interconnected system that we use to perceive and represent other people. The vast majority of what we know about person perception comes from studies that focus exclusively on the face—often as presented in a single static image. Although studies of face perception lay the foundation for the broader study of person perception, there is much more to know. This special issue takes the first steps in this next frontier of research. We provide the first collection of papers that bring together theoretical perspectives and empirical work aimed at the broader question of person perception.
The themes covered in this collection begin with three papers examining potential mechanisms which underlie the shaping of person representation in the context of natural experience. Collins et al., address this issue by adopting a neuropsychological, approach studying individuals with either developmental dyslexia (DD) or congenital prosopagnosia (CP). Their rational is based on the assertion that the typical right hemispheric lateralization of face representation is driven by the natural emergence of word/language representation in the left hemisphere with development. Using ERP and behavioral measures in these populations, they reveal that typical hemispheric organization for words can develop in the absence of typical hemispheric organization for faces but not vice versa, thus supporting the account that face representation is driven by developmental pressure from the language system. Two other studies examine the tuning of face representation in adults using the well-documented other-race effect which nicely demonstrates how personal experience shapes face representation. The study by Crookes and Rhodes stresses that the underlying mechanism of this effect is associated with perceptual expertise driven by experience with own-race faces and not from lack of motivation to individuate other-race faces as implied by social-cognitive accounts. The study by Butcher, Lander and Jagger provides further support for the perceptual account of the other-race effect by showing that motion acts as a robust and valuable cue for the identification of both same-race and other-race faces. We will return to the significance of motion for person perception in additional studies included in this special issue which will be reviewed below.
The second theme of this collection goes beyond the use of faces in isolation to ask the question: How do we integrate information from the face and the body to see a whole person? Balas and Pearson start from the premise that successful face perception must accomplish two things: “telling people apart” and “telling people together” (i.e., being able to see a stable identity across multiple variable images of a person). The problem of “telling people together” is also emphasized by Kramer, Jenkins, Young and Burton, who highlight the importance of studying faces from images that are widely variable. In this issue, they show that the benefit of exposure to high variability of upright faces is not generalized to inverted or contrast-reversed faces. These findings also highlight that the shaping of person representation is critically dependent on exposure to natural, ecological conditions. Balas and Pearson examine the more comprehensive problem of “telling faces, bodies, and people” together. They ask whether integrating faces and bodies into a whole makes this process less prone to error. Lecker, Shoval, Aviezer and Eitam build on this theme to explore the temporal limits of integrating emotion across body and face. Extant findings suggest that the emotional cues from the bodies intervene in processing emotional cues from the face, even when the body cues are task irrelevant. These authors show the importance of temporal immediacy in integrating emotion cues from the body and the face. Hu, Jackson, Yates, White, Phillips and O’Toole look at the role of forensic expertise with faces in the use of body information for identification. Here we find a case of the untrained novice being better able than trained facial forensic examiners to identify people, by using identity information in the body. Johnstone and Downing examine the perception of body features and the extent to which basic features such as sex and weight are perceptually dependent. They apply a Garner paradigm to body classification and find asymmetric dependence between weight and sex, such that perceptual access to weight information depends on sex, but access to sex information occurs with no dependence on weight. Turning towards automated, eye-witness descriptions of suspects in applied situations, Nixon, Guo, Stevenage, Jaha, Almudhahka, and Martinho-Corbishley demonstrated the power of comparative descriptions in increasing the accuracy of body-based person descriptions, such as “he was taller”. Although these authors re-assert the primacy of the face as an identifying cue, their work shows that accuracy can be increased by the combination of cues from faces, bodies, and peripheral person features, such as clothing.
A third theme focuses on the role of motion in person perception and includes several papers on —the face and body, as well as the face and voice in motion. Pilz and Thornton used recent motion capture and computer animation techniques to quantitatively explore the impact of body motion on person recognition. With an elegant method, they demonstrate the influence of learned body motions in biasing the identification of people when information in the face is ambiguous. Simhi and Yovel demonstrate the importance of familiarity in the effective use of idiosyncratic body motion as a cue for identity. People familiarized with whole people in motion, therefore, are able to exploit identity cues that fail for identification of less familiarized people. This dove-tails well with stark differences in the robust ability for recognition of familiar versus unfamiliar faces documented in the face perception literature. Nelson and Mondloch test both children and adults and found that recognition of facial expression was impaired when the body posture was incongruent with face emotion, especially when the displays were presented with naturalistic motion. Dukes, Clément, Audrin, and Mortillaro add to a fast expanding literature that stresses the importance of looking beyond the face to naturally moving bodies in order to accurately perceive emotion and intent. These authors note the importance of the body, especially in conveying positive emotions. They add the important category of “interest” to the emotions we convey, not only with the face, but with the body and the body in motion. The role of motion in integrating multiple cues about person identity is also evident in a study that examine the interaction between the face and the voice. Schweinberger and Robertson examined the benefit and cost of voice and face recognition for congruent and incongruent face-voice stimuli. Congruency effects were stronger for dynamic, familiar and synchronized face-voice stimuli, further emphasizing the contribution of dynamic identity signature for familiar person recognition.
The benefit for face-voice integration in person recognition is further demonstrated by four other papers. Bülthoff and Newell show a strong integration between faces and voices in a priming study for familiarized faces. Following a face-voice familiarization session, face recognition was better for faces that were preceded by related voices, relative to faces preceded by unrelated voice or faces presented alone. The face overshadowing effect is another demonstration for the influence of the face on voice processing. Tomlin, Stevenage and Hammond found less accurate voice discrimination performance for voices that were presented with faces during study. This effect was not found when voices were presented with inverted faces, indicating a specific influence of the presence of a face in its natural, common orientation, on voice processing. In the study by Maguinness and von Kriegstein, we return to a neuropsychological approach, but here it is utilized for evaluating face-voice integration in a comprehensive review of the scarce literature on face and voice processing in prosopagnosia and phonoagnosia. Overall, the findings indicate that prosopagnosics, when presented with a face, do not show the benefit to voice recognition that is typically observed by neurotypical individuals. Phonoagnosics do show benefit for voice recognition when they are presented with bi-modal stimuli. The authors highlight the importance of these findings for developing bi-modal training regimes for prosopagnosia and phonoagnosia. The strong association between faces and voices demonstrated in these different reports, using variety of paradigms, is likely to be mediated by their similar neural and cognitive mechanisms. As discussed by Belin, faces and voices both show norm-based coding, a similar profile of trait inferences that are similarity processed by category selective brain regions.
An additional central question in the study of face-voice integration is whether the two sensory modalities converge at an early sensory stage or at a later semantic stage. Two papers that examined the relationship between face, voice, and semantic information report a stronger relationship between face-voice information than between face or voice information and semantic information. Bao et al. showed a recognition advantage for previously learned congruent dynamic face and voice stimuli, but no such benefit for matching face-name or voice-name stimuli. These findings indicate a stronger integration between audio-visual person related information than between sensory and semantic information about people. The relative contribution of face, voice, and semantic information for familiar person recognition was also examined by Piccininni et al., by measuring the consistency of fame level decision based on face and voice information. Results show that face-voice consistency was correlated with non-verbal fame evaluations, but not with verbal fame-evaluation, again indicating a stronger link between audio-visual person related information than semantic information.
Combined, the papers featured in this issue lay the groundwork for a field of study that is taking off in a variety of directions, but that will ultimately converge into a single direction that will lead to better understanding of how we perceive people.