The eye contact smile: The effects of sending and receiving a direct gaze

ABSTRACT

Eye contact often elicits a smiling response. We investigated whether an individual’s awareness that the recipient perceives their direct gaze during eye contact has an influence on this smiling response. Participants wore glasses with either clear or dark lenses (preventing the other person from seeing their eyes). Measurements of electromyographic activity from the zygomatic and periocular muscle regions showed that the smiling responses were greater to seeing the other’s direct versus averted gaze. The participants’ own gaze direction had also an effect, and this effect was modified by the visibility of their eyes. Zygomatic responses were greater when the participants were aware that their eyes were visible. Thus, awareness of sending a direct gaze to another plays a role in the smiling response. In addition, participants’ self-evaluated level of social anxiety was positively correlated with the magnitude of the zygomatic responses to the other person’s direct versus averted gaze.

KEYWORDS:

• Affiliation
• facial electromyography
• mutual gaze
• smiling
• social anxiety

View correction statement:

Correction

This article is related to:

Interactive Cognition: An introduction

Introduction

In various everyday social encounters, we often smile when seeing that another person is looking towards us trying to make eye contact. Another individual’s direct gaze (i.e., eyes directed towards the self) is an affiliative cue indicating social inclusion (Wirth et al., 2010) and a motivational tendency of approach (Adams & Kleck, 2003, 2005). As humans have a fundamental need for belonging (Baumeister & Leary, 1995), seeing another person directing their gaze towards the self is perceived as a positive social signal – provided, of course, that the circumstances are otherwise non-threatening. This may evoke not only an affiliative smile (Martin et al., 2017; Niedenthal et al., 2010), in the perceiver, but also other components of positive affective reactions (for a review, see Hietanen, 2018). Recently, Hietanen and colleagues reported three studies in which they measured facial electromyographic (EMG) activity, in the laboratory, in response to seeing another person’s neutral face with direct or averted gaze. Measuring facial EMG responses has been a widely used method to investigate affect-related responses, and many studies have shown that affectively positive stimuli elicit increased EMG activity of the Zygomaticus major (the muscle that elevates the corners of the mouth) (Cacioppo et al., 1986; Dimberg, 1990; Larsen et al., 2003; Tassinary & Cacioppo, 1992). The results showed that zygomatic muscle region (smiling) activity was greater in response to seeing another person with direct than with averted gaze (Hietanen et al., 2018; Hietanen et al., 2020; Kiilavuori et al., 2021).

Unlike in many other studies investigating the effects of another person’s gaze direction on a perceiver’s responses, in the aforementioned EMG study by Hietanen et al. (2018), the participants themselves were also free to either look towards or away from the other person. The participants were also led to believe that, on each trial, the other person (the model) was also choosing freely whether to look at the participant or not. The results showed that, in addition to the zygomatic responses being influenced by the other person’s gaze direction, the participant’s own gaze had an influence on the zygomatic responses, too: the zygomatic responses to the model’s direct gaze were greater when the participant was also looking towards the model (i.e., making eye contact) than when the participant’s own gaze was averted. Importantly, it was ensured by experimental arrangements that the participants were able to see whether the model was looking at them or not, even if their own gaze was averted. In proof of this, the results showed that the participants’ zygomatic responses were greater to the model’s direct than averted gaze also when their own gaze was averted. The authors concluded their results suggesting that “participants’ understanding of mutual, reciprocal reception of each other’s attention” (Hietanen et al., 2018, p. 6) played a central role in influencing the magnitude of the smiling responses.

In the fields of research focusing on social interaction and social attention, investigators have become increasingly aware that these phenomena cannot be investigated relying only on laboratory experiments where participants are passive observers of “social” stimuli, often presented as pictures on a screen (De Jaegher et al., 2010; Risko et al., 2012; Schilbach et al., 2013). Eye-tracking studies have shown, for example, that while participants frequently looked towards a person appearing on a television screen placed in a waiting room, they looked away and avoided potential eye contact when facing the same person live in the waiting-room (Laidlaw et al., 2011). The authors explained their results by the influence of social norms of avoiding eye contact in these situations. Similar results demonstrating how the understanding of being watched by another person influences participants’ own gaze behaviour were reported in a study where participants watched videos of faces of higher and lower ranked people (Gobel et al., 2015). The participants themselves were filmed and they were told that the people in the videos would later watch the films or that no one would be watching the films. The results showed that the participants looked less at the eyes of the higher-ranked people when they believed that these people would later be watching the films as compared to when they believed the films not to be watched. The effects of being watched have been shown, not only on participants’ overt gazing behaviour, but also on psychophysiological responses. In one study, skin conductance response (indexing autonomic arousal), heart rate deceleration response (attention orienting), and electroencephalographic event-related potentials (attention-related brain responses) were measured in response to seeing another person with direct or averted gaze (Myllyneva & Hietanen, 2015). In one condition, the participant and the model person were able to see each other normally, whereas, in the other condition, the participant was led to believe that a half-silvered mirror was placed between the participant and the model in such a way that the model could not see the participant. The results showed that the psychophysiological responses differentiated between direct and averted gaze when the participants believed that the model was able to see them, but not when the participants believed that the model could not see them.

In the present study, we were interested in what kind of a role an individual’s understanding of reciprocal reception of each other’s attention, that is, sending as well as receiving direct gaze, plays in the affiliative smiling responses to making eye contact. If an affiliative smile depends not only on perceiving (receiving) another person’s direct gaze, but also on sending one’s own direct gaze and knowing that the other person perceives (receives) this, it would lead to a hypothesis that the direction of a participant’s own gaze would have a smaller (or non-existent) effect on this smiling response if the participant knows that the other person cannot see whether the participant is looking towards or not. An every-day situation in which this can happen is when one is wearing sunglasses. A person wearing sunglasses can see whether another person is looking directly at them or not but, at the same time, they know that the other person cannot see the direction of their own eyes. Thus, even when they both are looking towards each other, the participant knows that the other person cannot be sure about this. Is this reflected in the participant’s affiliative smiling response?

In a recent study, Jarick and Bencic (2019) presented a similar type of a question, but instead of investigating smiling responses, they focused on autonomic arousal. They examined whether sending (one’s own) as well as receiving (the other’s) direct gaze plays a role in increased autonomic arousal during eye contact. Many previous studies had shown greater skin conductance responses when a participant is looking (i.e., with direct gaze) at another person’s direct vs. averted gaze (Helminen et al., 2011; Hietanen et al., 2008; Nichols & Champness, 1971; Pönkänen et al., 2011). Jarick and Bencic (2019) measured skin conductance simultaneously from two participants sitting side-by-side and turning their heads (looking) either away from each other (baseline), towards each other (eye contact), or looking at their partner’s profile while the partner was looking straight ahead. The results showed that autonomic arousal was greater during eye contact, that is, when the participants were both sending and receiving direct gaze than during baseline or during the participant just sending or receiving direct gaze. However, when the clarity by which the gaze signals were sent and received was degraded by having one participant wearing sunglasses, the autonomic arousal responses during eye contact were not any more greater than those in the send-only or receive-only conditions. The authors concluded that the increased autonomic arousal responses to eye contact do not reflect the “reception” of another’s direct gaze only, but also the simultaneous “sending” of one’s own direct gaze.

In the present study, we decided to use an experimental paradigm similar to that of our previous study (Hietanen et al., 2018) in which the participants were presented with another live person through an electronic shutter and were free to either look towards or away (at a pre-determined fixation spot) from the other person. The participants believed that the other person was also choosing freely whether to look at the participant or not. The novel manipulation, in the present study, was that the participants wore either glasses with clear lenses or glasses with dark lenses preventing their own gaze direction to be seen. We measured facial EMG activity from the Zygomaticus major (cheek) muscle region and from the Orbicularis oculi (periocular) muscle region. Although increased zygomatic muscle activity has been suggested to reflect an automatic positive affective reaction (Cacioppo et al., 1986; Dimberg, 1990; Larsen et al., 2003), it is also possible that zygomatic responses reflect highly automatized affiliative reactions in response to another individual’s affiliative signals (Hess et al., 2000; Knutson, 1996; Niedenthal et al., 2010). The periocular muscle activity has also been associated with smiling. According to some theories, smiles with periocular activity (so-called Duchenne smiles in which the top of the cheek is raised and the outer corner of the eye is wrinkled) are qualitatively distinct from those without periocular activity and index “true” genuinely felt positive emotions (Ekman et al., 1990). In some studies, Duchenne smiles have been associated with higher self-reported feelings of enjoyment (Ekman et al., 1990; Soussignan, 2002). However, more recently it has been suggested that the periocular activity is rather indexing the intensity of smiling (Messinger et al., 2012). In the present study, we were interested in observing whether the affiliative smile in response to received and sent direct gaze involves also periocular activity.

It is possible that the affiliative smiling response is modulated by differences in individual characteristics. One such potential factor is an individual’s level of social anxiety. As another person’s direct gaze signals that their attention is directed towards the self, it may be a potential threat for people with social anxiety. And indeed, avoidance of eye contact in social situations is a prominent clinical symptom of social anxiety disorders (Greist, 1995). Previous research has shown that social anxiety is associated with shortened viewing times of the eye region and reduced eye contact (Daly, 1978; Farabee et al., 1993; Garner et al., 2006; Moukheiber et al., 2010). Compatibly with these results, previous studies have reported enhanced autonomic reactivity and attenuated relative left-sided frontal cortical electroencephalographic activity (associated with decreased positivity and approach motivation) to direct vs. averted gaze in participants scoring high in social anxiety (Myllyneva et al., 2015; Wieser et al., 2009). Against this background, it was interesting to study, whether participants’ level of social anxiety would modulate the zygomatic and periocular responses in the present experimental conditions. To the best of the authors’ knowledge, there are no previous studies of the effects of social anxiety on facial EMG activity in response to gaze direction. There are, however, studies investigating the effects of social anxiety on facial reactions in response to other people’s facial expressions. Considering the topic of the present study, earlier results related to facial reactions in response to happy faces are the most relevant. The results of these studies are inconclusive, however. Some studies have reported decreased zygomatic responses (Dimberg, 1997; Dimberg & Christmanson, 1991; Vrana & Gross, 2004), whereas one study reported increased zygomatic responses to happy faces in individuals with a high level of speech anxiety (Dimberg & Thunberg, 2007). Two studies based on observer ratings (Heerey & Kring, 2007) and video-based computerized coding (Dijk et al., 2018) investigated smiling during actual interaction. Heerey and Kring (2007) found no effect of social anxiety on the amount of smiling in response to displays of social (non-Duchenne) smiles, whereas the amount of smiling in response to genuine (Duchenne) smiles was greatly attenuated in individuals with high levels of social anxiety. Dijk and colleagues (2018) reported positive correlations between the intensity of the cheek area smile in response to the confederate’s smile and the level of social anxiety. For periocular area action intensity, no association with social anxiety level was found.

Based on the results from our previous studies (Hietanen et al., 2018; Hietanen et al., 2020; Kiilavuori et al., 2021), we expected first that seeing (receiving) another person’s direct gaze would result in greater zygomatic and periocular EMG activity than seeing the other person’s averted gaze. Secondly, we expected that the participant’s own gaze direction would have an effect too (cf., Hietanen et al., 2018), the responses would be greater when the participant is sending direct gaze (i.e., looking towards the other person) as compared to sending averted gaze (i.e., looking away from the other person). Third and most importantly for the present study, we expected that the participant’s own gaze direction would have a smaller effect when the participant is wearing glasses with dark lenses as compared to when wearing glasses with clear lenses.

After the physiological recordings, we asked participants to fill a questionnaire (Social Phobia Scale, SPS, Mattick & Clarke, 1998) measuring participants’ self-reported degree of social anxiety. Because of previous contradictory findings, we did not set up any hypotheses regarding the direction of the effects of social anxiety on the smiling responses. We also investigated whether the glasses would have an effect on participants’ explicit affective feelings (affective valence and arousal) during eye contact. After the physiological recordings, we asked participants to rate their feelings when looking at the model with direct gaze. If an individual’s awareness of the visibility of their own gaze has an influence on these explicit affective feelings, too, it was expected that the self-evaluated affective valence and arousal would be lower when having dark vs. clear lenses while looking at the model.

Methods

Participants

Previous studies (Hietanen et al., 2018; Hietanen et al., 2020; Kiilavuori et al., 2021) have reported highly robust main effects of gaze direction on zygomatic EMG responses (effect sizes ranging from ${\eta }_p^2$ = 0.38–0.57) with sample sizes ranging from 27 to 42. In the present study, we aimed at a sample size of 36, which had >.99 power to replicate the robust main effects and .8 power to detect 2-way interactions with effect sizes of ${\eta }_p^2$ = .19 and larger. For the correlation analyses (association between social anxiety and EMG responses, see below), this sample size had .8 power to detect correlations of r = .44 and larger. Originally data were gathered from 36 participants, but the data from 3 participants were discarded due to an insufficient number of trials with good data (see below). Thus, the final sample consisted of 33 participants (18 females and 15 males; age range = 19–45 years; mean age = 24.9, SD = 4.6) recruited via email lists of Tampere University. All participants had normal or corrected-to-normal vision and they did not report of any neurological or psychiatric diagnoses. During the recruitment, the volunteers were informed that it was not possible to wear eyeglasses during the experiment and, therefore, if vision correction was needed, only participants wearing contact lenses would be accepted. All participants gave a written, informed consent, and received either course credits or a movie ticket for their participation. Ethical statement for the experiment was obtained from the Ethics Committee of the Tampere region. The study conformed to the Code of Ethics of the World Medical Association (Declaration of Helsinki).

Stimuli and procedure

The stimulus presentation and procedure largely followed those described in our previous study (Hietanen et al., 2018). A young male and a young female, previously unknown to the participants, served as models. The model’s and the participant’s sex was matched. The models bore a neutral expression and kept their faces as motionless as possible throughout the experiment. However, when necessary, eye blinks were allowed to occur. The models were instructed to maintain a slight muscle tonus in the lower part of the face in order not to look sullen or fatigued. The models’ faces were presented through a 38 × 22 cm custom-built electronic shutter with a voltage-sensitive liquid crystal (LC) window (NSG UMU Products Co., Ltd.) attached to a black frame between the model and the participant. The participant was seated at a distance of 90 cm from the LC-shutter and the model was sitting at a distance of 40 cm from the shutter. The model’s seat was adjusted in such a way that vertically the model’s eyes were at the same level with the participant’s own eyes. The state of the LC-shutter (transparent or opaque) was operated by E-Prime 2.0 software (Psychology Software Tools, Pittsburgh, PA) running on a desktop computer and the LC-shutter switched between opaque and transparent states within an overall speed of 3 ms (Figure 1).

Figure 1. Illustration of the experimental set-up. The model (right) was presented to the participants (left) through a voltage sensitive LC window.

After arriving to the laboratory, the participants were informed that the purpose of the study was to measure physiological responses during a simple interaction situation. They were shown the LC-shutter and told that, during the experiment, they would be seeing another person through the shutter. To disguise the purpose of the facial EMG electrodes, the participants were told that skin temperature would be measured during the task with sensors attached to the face. Moreover, the participants were told that, during the experiment, they would be wearing glasses that would measure “the orientation of their eyes”. A half of the participants started with clear lenses and the other half with dark lenses. At this point, the participants were not told that there would be a second part, in the experiment, in which they would be wearing the other pair of glasses. When introducing the glasses with clear lenses, the experimenter explicitly mentioned that the lenses of the glasses are clear, like in ordinary spectacles, and that it is possible to see the eyes of the person wearing these glasses. To convince the participant of this, the experimenter put the glasses on their own head and confirmed that the participant was able to see their eyes. Likewise, when introducing the glasses with dark lenses, the experimenter explicitly mentioned that the lenses of the glasses are dark, like in sunglasses, and that it is impossible to see the eyes of the person wearing these glasses. The experimenter put the glasses on their own head and confirmed that the participant was unable to see their eyes. In pilot testing, the lighting conditions in the laboratory were set up in such a way that it really was possible and impossible to see the person’s eyes wearing the clear and dark glasses, respectively.

After attaching the facial EMG electrodes, the model person was led into the laboratory and the participants were told that they would be seeing this person on the other side of the LC-shutter. The experimenter treated the model persons as if they were participants as well (in reality, the model was a confederate). The participant and the model person were instructed that they were free to choose whether to look directly towards each other or whether to avert their gaze side-ways and look to a pre-determined fixation point. The participants were led to believe that the model also had a possibility to voluntarily choose where to look at. For the participant, the fixation marks were located on the frame of the LC-shutter. Looking at these marks resulted in a gaze deviation of 7 degrees (from the line of straight gaze). For the model, the fixation marks were laterally further away resulting in a gaze deviation of 50 degrees. This arrangement made it possible for the participants to see whether the model was looking at him/her or not, even if his/her own gaze was averted. The instructions emphasized that the participant and the model keep their heads straight ahead and move just their eyes.

The participants (and the model) were told that before the shutter becoming transparent a sound signal would be presented. The signal would indicate that the participant (and the model) should choose whether to look directly at the other person or avert their gaze to the side. They were asked to make their choice immediately after the sound signal, turn their gaze to the chosen direction, and to keep the gaze direction throughout the rest of the trial until the shutter, after becoming transparent, turned opaque again. The participants were also asked to indicate the chosen gaze direction, immediately after choosing it, with a button press on a keyboard in front of them. The sound signal was a computer-generated sound played with a relatively low level of intensity. The shutter turned transparent 5–7 s after the sound signal, stayed transparent for 3 s, and turned opaque again. The experimenter initiated the next trial after approximately 15 s had passed from the shutter turning opaque.

The trials were presented in blocks of 8 trials. E-Prime software-controlled presentation of the sound signals and the state of the LC-shutter, and it also delivered instructions to the model where to look at. The instructions appeared on a monitor located on the model’s side of the panel. On four trials, the model (M) was to look directly (d) to the participant (P) (eye contact) and, on four trials, the model was to avert his or her gaze (a) and look either to the left or right (1:1). These conditions were presented in a random order. After three 8-trial blocks, there was a short pause. The experimenter sitting 2 m behind the participant on the other side of a partition monitored on-line the model’s gaze direction (based on the trigger signals appearing on the screen of the computer recording physiological data) and the participant’s gaze directions based on their button presses. The data collection was continued until there were at least six trials in each four condition (MdPd, MdPa, MaPd, and MaPa). Thus, in minimum, there were 24 trials.

Immediately after the presentation of the experimental trials, the participants were asked to fill a brief questionnaire to evaluate their explicit affective feeling during the experiment and the stimulus presentations. The experimenter asked the participants to recall how they felt during the previous experimental blocks and in order to help the participants in this task, the shutter window was opened once more for 3000 ms to show the model person (with direct gaze) to the participant. The participants evaluated their own feelings of affective valence and arousal during on a 9-point Self-Assessment Manikin (SAM, see Bradley & Lang, 1994) scales (1 = unpleasant/calm, 9 = pleasant/arousing).

After the participants had evaluated their affective feelings, they were told that the first part of the experiment was over, but that there would be a second part. They were told that this part would be otherwise similar to the first one, but that the lenses of the glasses would be changed into dark/clear ones. As described above, the experimenter demonstrated the opacity/transparency of the lenses to the participant. After this, the experiment progressed in a similar way as in the first part.

After the participants had evaluated their affective feelings in the second part of the experiment, the glasses and the electrodes were removed. The participants were asked to fill one more questionnaire, the Social Phobia Scale (SPS, Mattick & Clarke, 1998). The Social Phobia Scale is a 20-item self-report measure. The items deal with fear of being scrutinized or observed during every-day activities in front of/in the presence of other people (i.e., “I get nervous that people are staring at me as I walk down the street”). The participants evaluated the degree to which each statement was characteristic or true for them on a 5-point scale (1 = not at all, 5 = extremely).

After filling the SPS, the participants were told that the experiment was finished. They were asked about their thoughts regarding the purpose of the experiment and whether they thought that the experimenter had not told everything or that they had been misinformed somehow. None of the participants revealed any suspicions. After this, the participants were debriefed and were told about the slight deception: that is, the true purpose of the EMG electrodes, that the model was a confederate, and that the model did not choose freely their gaze direction but followed a predetermined script.

Physiological measures

EMG was used to measure facial muscle activity over Zygomaticus major and Orbicularis oculi muscle regions. First, the skin over the recording sites was rubbed with alcohol. Electrode paste (Signa gel) was then injected to bipolar 4-mm Ag-AgCl electrodes (BioMed Electrodes) which were attached 1 cm apart with a tape over the recorded muscle sites according to the placement guidelines by Fridlund and Cacioppo (1986). A ground electrode was attached to the middle forehead, directly below the hairline. The signal was amplified by a QuickAmp amplifier and continuously recorded with the BrainVision Recorder software (Brain Products GmbH, Munich, Germany) with a sampling rate of 500 Hz. Offline, the signal was filtered with a 28–249 Hz bandpass filter and a 50-Hz notch filter using BrainVision Analyzer 2.1 software. Offline, the EMG signal around each experimental trial was visually inspected for artefacts due to excessive muscle movements and blinks. Following this procedure, the data from three participants were excluded because, at least in one experimental condition, there were less than three trials with acceptable data. For the analyses, the signal was rectified, smoothed, and segmented from a 500-ms baseline (prior stimulus onset) to 3000 ms post-stimulus in 500-ms bins. These values were then standardized within participant and within muscle to reduce the influence of extreme values. The muscle activity related to the experimental trials was calculated as change scores by subtracting the baseline muscle activity from each 500-ms average value, and these values were then averaged across the 3000-ms post-stimulus time-window within each experimental condition. These procedures are standard procedures on the field (see Tassinary & Cacioppo, 2000) and we have been following them in our previous studies (e.g., Hietanen et al., 2018; Hietanen et al., 2019; Hietanen et al., 2020; Kiilavuori et al., 2021). The mean number of accepted trials in each condition was as follows: Clear lenses MdPd: M = 6.6; MdPa: M = 7.0; MaPd: M = 7.3; MaPa: M = 6.7; Dark lenses MdPd: M = 6.7; MdPa: M = 6.8; MaPd: M = 8.7; MaPa: M = 7.6. The differences in the number of accepted trials between the conditions were not significant (Friedman’s test; Fr = 7.824, p = .348). Note that the slight differences in the number of accepted trials between conditions do not stem from differences in the number of rejected trials/condition, but from differences in the total number of collected trials/condition (to reach the criterion of a minimum of six trials/condition).

Data analysis

Statistical analyses were conducted using repeated measures 2 (Model’s gaze: direct vs. averted) × 2 (Participant’s gaze: direct vs. averted) × 2(Lense colour: clear vs. dark) ANOVA (Greenhouse–Geisser correction procedure was applied when appropriate) and calculating Pearson correlation coefficients. For ANOVAs, planned comparisons were performed if interactions were observed. To break down Model’s gaze × Participant’s gaze interaction, the effect of the model’s gaze direction was tested separately for the participant’s direct and averted gaze (i.e., two t-tests, without corrections for multiple comparisons). In the same vein, to break down Lense colour × Model’s/Participant’s gaze interaction, the effect of the model’s/participant’s gaze direction was tested separately for the clear and dark lenses (i.e., two t-tests, without corrections for multiple comparisons). For the analyses of the EMG responses, the presentation order of the experimental blocks (clear vs. dark lenses) was originally included into the analyses. However, as the results showed this factor had no main effects nor was it interacting with any of the other effects, this factor was not included into the final analyses reported in the results section below. The association between the participants’ self-evaluated level of social anxiety and the magnitude of the zygomatic and orbicular muscle responses was analyzed by correlating the social anxiety scores with the “direct gaze effect” of the EMG responses. A similar approach has been used in other previous studies investigating the association between participant-dependent factors and physiological responses to gaze direction (i.e., Prinsen et al., 2018; Prinsen & Alaerts, 2019). The “direct gaze effect” was quantified by calculating the difference between response magnitudes during the direct and averted gaze conditions (i.e., Δ direct gaze – averted gaze) separately for the model’s gaze and participant’s own gaze. These indices of the gaze effect were then correlated with the participants’ scores on the social anxiety scale, separately when participants were having clear and dark lenses. Thus, four correlations per muscle were calculated.

Results

Facial EMG responses

The EMG responses were analysed with a 2(Model’s gaze) × 2(Participant’s gaze) × 2(Lense colour) ANOVA. We had hypothesized that (1) seeing the model’s direct gaze would result in greater EMG activity than seeing the model’s averted gaze and that (2) the participant’s own gaze direction would also have an effect showing greater responses when the participant is looking towards the model as compared to when looking away from the model. For zygomatic region responses (see, Figure 2), the main effects of Model’s gaze, F(1,32)= 9.871, p = .004, ${\eta }_p^2$  = 0.236, and Participants’ gaze, F(1,32)= 27.258, p = .00001, ${\eta }_p^2$ = 0.460, were statistically significant. The zygomatic muscle region responses were greater in response to the model’s direct gaze (M = 0.400, SEM = 0.093) than averted gaze (M = 0.055, SEM = 0.080). The zygomatic responses were also overall greater when the participant’s own gaze was direct (M = 0.462, SEM = 0.086) rather than averted (M = −0.008, SEM = 0.075).

Figure 2. Individual zygomatic region electromyographic (EMG) responses and their means (and 95% CI) as a function of the model’s and participant’s gaze direction and the colour of the lenses. Abbreviations: M = model; P = participant; d = direct gaze; a = averted gaze. (The data were visualized by an R Shiny app by Postma & Goedhart, 2019).

There were also two two-way interactions. First, the interaction between Model’s gaze direction and Participant’s gaze direction, F(1, 32)= 4.623, p = .039, ${\eta }_p^2$ = 0.126, was significant. This interaction reflected the fact that the model’s gaze direction had a larger effect on the zygomatic responses when the participant was looking at the model than when the participant’s gaze was averted. The zygomatic responses were greater to the model’s direct than averted gaze when the participant’s gaze was direct (M_MdPd = 0.725, SEM = 0.133 vs. M_MaPd= 0.198, SEM = 0.097, t(32) = 3.350, p = .002, d = 0.583), whereas the model’s gaze direction did not have a significant effect on zygomatic responses when the participant’s gaze was averted (M_MdPa = 0.074, SEM = 0.087 vs. M_MaPa= −0.089, SEM = 0.103, p > .170). Most importantly, we had hypothesized that the participant’s own gaze direction would have a smaller effect when the participant is wearing glasses with dark lenses as compared to when wearing glasses with clear lenses. And indeed, the results showed an interaction between Participant’s gaze direction and Lense colour, F(1,32)= 4.401, p = .044, ${\eta }_p^2$ = 0.121. The effect of the participants’ gaze direction was significantly greater when the participants were having clear (M= 0.668) than dark (M= 0.271) lenses. It should be noted, however, that pairwise comparisons indicated that the zygomatic responses were greater for the participants’ direct than averted gaze not only when the lenses were clear (M_Pd= 0.567, SEM = 0.110 vs. M_Pa = −0.101, SEM = 0.110, t(32) = 4.861, p = .001, d = 0.846), but also when they were dark (M_Pd= 0.356, SEM = 0.104 vs. M_Pa = 0.085, SEM = 0.082, t(32) = 2.197, p = .035, d = 0.383). All the other effects were not significant (all ps > 0.333).

For periocular region responses (see, Figure 3), the main effect of Model’s gaze, F(1,32)= 10.506, p = .003, ${\eta }_p^2$ = 0.247, was statistically significant. The periocular muscle region responses were greater in response to the model’s direct gaze (M = 0.481, SEM = 0.104) than averted gaze (M = 0.102, SEM = 0.097). The main effect of Participant’s gaze was marginal, F(1,32)= 3.209, p = .083, ${\eta }_p^2$ = 0.091, the periocular responses being slightly greater when the participant’s own gaze was direct (M = 0.380, SEM = 0.119) rather than averted (M = 0.203, SEM = 0.064).

Figure 3. Individual periocular region electromyographic (EMG) responses and their means (and 95% CI) as a function of the model’s and participant’s gaze direction and the colour of the lenses. Abbreviations: M = model; P = participant; d = direct gaze; a = averted gaze. (The data were visualized by an R Shiny app by Postma & Goedhart, 2019).

The two-way interaction between Model’s gaze direction and Participant’s gaze direction, F(1, 32)= 8.108, p = .008, ${\eta }_p^2$ = 0.202, was significant. The interaction between Model’s gaze and Participant’s gaze reflected the fact that the model’s gaze direction had a larger effect on the periocular responses when the participant was looking at the model than when the participant’s gaze was averted. The periocular responses were greater to the model’s direct than averted gaze when the participant’s gaze was direct (M_MdPd = 0.682, SEM = 0.150 vs. M_MaPd= 0.077, SEM = 0.130, t(32) = 4.049, p = .001, d = 0.705), whereas the model’s gaze direction did not have a significant effect on periocular responses when the participant’s gaze was averted (M_MdPa = 0.280, SEM = 0.090 vs. M_MaPa= 0.126, SEM = 0.095, p > .250). All the other effects were not significant (all ps > 0.132).

Social anxiety

We analysed the association between the participants’ self-evaluated level of social anxiety and the magnitude of the zygomatic and orbicular muscle responses by first quantifying the “direct gaze effect.” We calculated the difference between response magnitudes during the direct and averted gaze conditions (i.e., Δ direct gaze – averted gaze) separately for the model’s gaze and participant’s own gaze. These indices of the gaze effect were correlated with the participants’ scores on the social anxiety scale, separately when participants were having clear and dark lenses.

For zygomatic responses, these analyses (see, Figure 4) revealed that the magnitude of the model’s gaze effect was positively correlated with the social anxiety scores, both for clear and dark lenses (r_clear = 0.384, p = .027; r_dark = 0.386, p = .027). Thus, the higher the participant’s score on social anxiety was, the greater was the effect of the model’s direct gaze on zygomatic responses. The magnitude of the participant’s own direct gaze effect was also positively correlated with the social anxiety scores, but only when the participants were wearing clear lenses (r_clear = 0.402, p = .020; r_dark = 0.026 p > 0.800). However, the difference between these two correlation coefficients was not statistically significant (z = 1.507, p > 0.130). The direct gaze effects on orbicular muscle region responses were not significantly correlated with the social anxiety scores (all ps > .140).

Figure 4. The association between the participants’ scores on the social anxiety scale and the gaze effect (Δ direct gaze – averted gaze) for the model’s gaze and participant’s own gaze when the participants were wearing glasses with clear and dark lenses. Dotted lines denote 95% CI.

Explicit affective feelings

After both experimental blocks (with clear and dark glasses), the participants were asked to evaluate their subjective feelings when making eye contact with the other person.

Regarding affective valence (scale range: 1–9), the ratings indicated a positive feeling, but the effect of glasses was not significant (M_clear = 6.333, SEM = 0.193 vs. M_dark = 6.303, SEM = 0.224, t(32) = 0.162, p > .800). For the arousal ratings, the effect of glasses was marginal. The participants reported feeling slightly more aroused when wearing the clear glasses (M_clear = 4.061, SEM = 0.311) as opposed to the dark glasses (M_dark = 3.606, SEM = 0.268), t(32) = 1.873, p = .07, d = 0.326.

Discussion

In the present study, we investigated participants’ smiling responses in a situation where they were facing another, live person. On each trial, the participants were free to decide whether to look towards the other person or away, and they believed that the other person (model) was similarly choosing freely whether to look at the participant or not. We measured electromyographic responses from the zygomatic and periocular muscle regions in order to investigate the effects of participant’s own as well as the model person’s gaze direction on affiliative smiling responses. Specifically, in the present study, we were interested in finding out whether an affiliative smile depends not only on receiving (perceiving) another person’s direct gaze, but also on sending one’s own direct gaze and knowing that the other person receives this. To this end, the measurements were conducted in two different conditions; in one condition, the participants wore glasses with clear lenses, whereas, in the other condition, they wore glasses with dark lenses preventing their own gaze direction to be seen by the other person. In addition to facial EMG, we investigated whether participants’ self-reported degree of social anxiety would be associated with the magnitude of the smiling responses. Finally, we also measured whether the colour of the lenses would have an effect on participants’ explicit affective feelings (affective valence and arousal) during eye contact.

First, we had expected that when facing another, live person with a neutral facial expression, receiving this person’s direct gaze would result in greater zygomatic and periocular EMG activity than seeing their averted gaze. The results fully supported these expectations, and regarding the zygomatic responses, the present study replicated our previous findings (Hietanen et al., 2018; Hietanen et al., 2020; Kiilavuori et al., 2021). There are previous studies that measured facial EMG activity in response to expressive and neutral faces looking towards the observer (eye contact) or not, but these studies did not find any effect of gaze direction on the EMG responses when the faces bore a neutral expression (Schrammel et al., 2009; Soussignan et al., 2013). The effect of gaze direction was observed only when the stimulus faces were expressive, for example, the zygomaticus responses to happy faces were greater when the gaze was direct as compared to when it was averted (Rychlowska et al., 2012; Schrammel et al., 2009; Soussignan et al., 2013). It is noteworthy, that in all these previous studies the stimuli were images (avatars or images of real people) presented on a computer monitor. The present results add support to accumulating evidence that many effects of various socially relevant signals on perceivers’ cognition, emotions, behaviour, and physiology may remain undetected when investigated with stimuli that only represent other people, stimuli that do not allow a possibility for actual interaction (De Jaegher et al., 2010; Hietanen et al., 2008; Risko et al., 2012; Schilbach et al., 2013).

Second, we had expected that the participant’s own gaze direction would also have an effect on the EMG activity: the responses were expected to be greater when the participant was looking towards the other person as compared to when looking away from the other person. The present results partially supported these hypotheses; overall the zygomatic responses were greater when the participants were sending their direct than averted gaze, whereas, for periocular region responses, this effect was only marginal. Again, regarding the zygomatic responses, the present study replicated our previous findings (Hietanen et al., 2018). The results of the present study replicated those of our previous study (Hietanen et al., 2018) also more specifically. The model’s and the participant’s gaze direction had an interaction effect on the zygomatic responses. Receiving another individual’s direct gaze elicited a greater zygomatic response than receiving another’s averted gaze, but in the present study, this was conditional to the participant’s own gaze being direct. When the participant did not send a direct gaze (i.e., was looking away), the model person’s gaze direction did not have a significant effect on the zygomatic responses. In the present study, we measured also periocular responses, and these measurements revealed a similar pattern of results as above: the periocular responses were greater to the model’s direct than averted gaze when the participant’s gaze was direct, but when the participant’s gaze was averted, the model’s gaze direction did not have a significant effect on periocular responses. In sum, the present results provide consistent evidence from the measurements of the zygomatic and periocular region muscle activity that seeing (receiving) another person’s direct gaze elicits a smiling response, but this is conditional to a situation when the perceiver is sending their own direct gaze to the other person, in other words, when the perceiver is also looking towards the other person (eye contact). Thus, the present as well as our previous study (Hietanen et al., 2018) clearly indicate that sending one’s own direct gaze to the other person’s plays a significant role in the affiliative smiling response to seeing another’s direct gaze. Importantly, like in our previous study (Hietanen et al., 2018), we confirmed with arrangements of the experimental set-up (see “Methods” section), that the participants were able to see whether the other person was looking at them or not, even when their own gaze was averted. Thus, the present results cannot be explained by arguing that the participants could not see the other person’s direct gaze when having their own gaze averted away.

The present results can also be seen to be compatible with those by Jarick and Bencic (2019) who measured autonomic arousal from two participants simultaneously and showed that autonomic arousal was elevated during eye contact, that is, when the participants were both sending and receiving direct gaze (with respect to a baseline when both participants were looking away from each other), but not when the participants were just sending or receiving direct gaze. In fact, measurements of autonomic arousal in our own previous study using the same experimental paradigm as the present study (Hietanen et al., 2018), showed exactly the same pattern of results.

Third and most importantly, the main aim of this study was to test a hypothesis that the participants’ own gaze direction would have a smaller effect when the visibility of their own gaze direction was prevented by having them to wear glasses with dark lenses. To this end, data were collected in two different blocks; when the participants were having dark lenses and when having clear lenses. The results showed that the effect of the participant’s own gaze direction was, indeed, modified by the colour of the lenses. For the zygomatic responses, the effect of the participants’ gaze direction (i.e., Δ direct gaze – averted gaze) was significantly greater when the participants were having clear than dark lenses. In other words, when the participants were aware of that the other person was able to see their gaze direction, their direct gaze increased the magnitude of their smiling response more than when the participants were aware of that the other person could not see their gaze direction.

In the aforementioned study by Jarick and Bencic (2019), the authors also manipulated the visibility of the eyes of one of the interaction partners with dark sunglasses. Their results showed that although the autonomic arousal was greater during eye contact than during the participants just sending or receiving direct gaze, the autonomic arousal responses during eye contact were not any more significantly greater than those in the send-only or receive-only conditions when one of the participants was wearing dark sunglasses. The authors concluded that the increased autonomic arousal to eye contact results from the “combined, simultaneous tasks of monitoring one’s own gaze signals and interpreting the gaze signals of others” (Jarick & Bencic, 2019, p. 10). The present results expand these findings by showing that a similar mechanism seems to apply also to affiliative smiling responses elicited by eye contact. We want to point out, however, that preventing the other person from seeing the sender’s (participant’s) gaze direction did not altogether abolish the effect of the sender’s gaze direction on zygomatic responses. The zygomatic responses were significantly greater for the participants’ own direct than averted gaze when the glasses were dark as well as when they were clear. The manipulation of the sender’s eye visibility seemed only to attenuate the effect of the sender’s gaze direction. Interestingly, the results also showed that the periocular responses were not modulated by the manipulation of the eye visibility. It would be tempting to suggest that, because the participants’ awareness of their own eyes being visible to the other person or not had an effect on their zygomatic but not on periocular responses, the modulation of the zygomatic responses reflected the controlled effects on affiliative smiling responses. Most facial expressions during face-to-face communication have both volitional (cortical) and emotional (extrapyramidal) components and the volitional control is greater for the lower than upper face muscles (Rinn, 1984).

We would like to point out that although we have been using the term “affiliative smiling” when referring to the smiling responses investigated in this study, it may be premature to adopt a strong stance towards the underlying mechanisms. The observed zygomatic and periocular EMG activity in response to eye contact may reflect an automatized affiliative reaction (Hess et al., 2000; Knutson, 1996; Niedenthal et al., 2010), but it is possible that these EMG responses reflect, in fact, a positive affective reaction (Dimberg, 1990; Larsen et al., 2003; Tassinary & Cacioppo, 1992). Previous studies employing various implicit behavioural (e.g., affective priming, Chen et al., 2017a; implicit attitude test, IAT, Lawson, 2015) and psychophysiological measures (i.e., attenuation of the eye-blink startle reflex, Chen et al., 2017b) have provided evidence that seeing another individual’s direct gaze elicits more positive affective reactions than seeing averted gaze (for a review, see Hietanen, 2018). The present results from the EMG measurements are compatible with this view and would suggest that sending one’s own as well as receiving another person’s direct gaze trigger a positively valenced affective reaction. Importantly, there is also a third possibility. It is entirely possible that the observed zygomatic and periocular EMG responses to eye contact may reflect both an automatized affiliative reaction and a positive affective reaction.

An interesting and novel finding was that the participants’ self-evaluated degree of social anxiety characteristics was associated with the magnitude of the zygomatic muscle responses. The effect of the other person’s direct gaze (i.e., Δ direct gaze – averted gaze) on zygomatic responses was the greater, the higher the participant’s score on social anxiety was. This effect was evident both when the participants were looking at the other person through the clear and dark lenses. Previous research has associated social anxiety with shortened viewing times of the eye region or reduced eye contact (Daly, 1978; Farabee et al., 1993; Garner et al., 2006; Moukheiber et al., 2010). Thus, the results of the present study appear to be strikingly incompatible with these earlier results. However, there are earlier studies investigating the effects of social anxiety on facial reactions in response to other people’s happy faces and some of these studies have also shown increased smiling responses to happy faces in individuals with high levels of social anxiety. Dimberg and Thunberg (2007) exposed people high and low in speech anxiety to pictures of angry and happy faces. The results showed that the differential zygomatic EMG activity in response to happy vs. angry faces was greater in high anxiety than low anxiety participants. However, as the differential corrugator activity in response to angry vs. happy faces was also greater in high anxiety than low anxiety participants, the authors concluded their results supporting a hypothesis that individuals with high anxiety are disposed to exhibit exaggerated facial responsiveness to positive and negative social stimuli. More recently, Dijk and colleagues investigated the facial expressions of participants with different levels of social anxiety in response others’ emotional displays (Dijk et al., 2018). The results showed that when the participants were interacting with another person live (Study 2), the intensity of the cheek area smile (in this study the analysis was based on video-based facial coding) in response to the confederate’s smile correlated positively with the level of social anxiety. For periocular area action intensity, no association with social anxiety level was found. Interestingly, when the facial expression stimuli were short video clips of dynamic expressions, no correlations were observed between social anxiety and the intensity of the participant’s facial expressions. The authors suggested that individuals with social anxiety may use a polite smile in an attempt to appease others in social situations.

The present results extend these previous findings by showing that social anxiety modulates affiliative smiles also in response to another person’s gaze direction. In the study by Dijk and colleagues (2018) described above, the association was found only between the intensity of the cheek area action and social anxiety and similarly, in the present study, the correlation was observed between the magnitude of the zygomatic EMG activity and social anxiety, but not between the periocular EMG activity and social anxiety. This finding may relate, as discussed above, to the greater degree of volitional control over the zygomatic than periocular muscle activity. The social anxiety scores were also positively correlated with the magnitude of the participant’s own gaze effect on zygomatic responses, but only when the participants were having clear lenses. However, the difference in the magnitude of correlations between the social anxiety scores and the magnitude of the participant’s own gaze effect when wearing clear vs. dark lenses was not statistically significant. Thus, the results did not provide reliable evidence that the participants’ smiling (zygomatic) responses were positively associated with their level of social anxiety only when they were aware that the other person was able to see them sending a direct gaze. We also want to emphasize that all these correlation analyses were somewhat underpowered due to the relatively small sample size and, therefore, these results should be interpreted with caution. Future studies with larger sample sizes will hopefully provide more convincing evidence for the association between social anxiety and the strength of smiling in response to another person’s direct gaze.

The participants were also asked to evaluate their subjective feelings when making eye contact with the other person. The participants reported feeling significantly more aroused when they knew that the other person was able to see their eyes as compared to when this was not the case. This finding concords with the findings from the implicit measurements of autonomic arousal in the study by Jarick and Bencic (2019). The visibility of the participants’ own eyes did not have a significant effect on the valence ratings, however. Now, as the results showed that the effect of the participants’ gaze direction (Δ direct gaze – averted gaze) on zygomatic responses was significantly greater when the participants were having clear than dark lenses while, at the same time, the visibility of the participants’ own eyes did not have a significant effect on the subjective valence ratings, one might be tempted to interpret this finding to provide preliminary evidence for the view that zygomatic responses to eye contact reflect an automatized affiliative reaction rather than a positive affective reaction, as discussed above. However, we are reluctant to interpret this pattern of results to provide strong support for the automatized affiliative reaction (social signalling) hypothesis. It should be noted that the zygomatic (physiological) responses were measured both to direct and averted gaze and the results showed a greater gaze direction effect (i.e., Δ direct gaze – averted gaze) of the participants’ own gaze when they were wearing clear vs. dark glasses, whereas the self-reported valence measurements were obtained only when the participants were looking towards the model and were wearing clear vs. dark glasses. Thus, these two measures reflect very different effects. Moreover, as self-ratings reflect the results of controlled and analytic processing they are also known to be very vulnerable to many kinds of top-down effects and biases (Evans, 2008; Hofmann et al., 2005). Because of these reasons, we are reluctant to interpret these results as providing evidence for the automatized affiliative reaction hypothesis and against the positive affective reaction hypothesis.

Throughout the text, we have been interpreting the participants’ facial reactions being responses to the model’s gaze direction. However, considering the interactive nature of the present experimental set-up, a potential factor contributing to the observed results could be that the eye contact elicited a smiling response also in the model persons and, thus, the participants’ smiles could have been responses not only to eye contact but to the model’s smiling, too. We think that this is not likely for two main reasons. First, the models were thoroughly pre-trained to keep their faces neutral during the stimulus presentation. Second, and more importantly, during the experiment, the lighting conditions in the laboratory were such that the model’s side of the room was well-lit, whereas the participant’s side of the room was relatively dark. This arrangement ensured that the participants were able to see the model person clearly, but consequently, the model person could not actually see very clearly to the participant’s side. The model was able to see the participant’s eyes, but it was very difficult if not impossible to see minor expressive facial movements on the participant’s face. Because of these two reasons, it is highly unlikely that the participants’ facial EMG responses were influenced by the model’s reciprocal facial expressions.

As noted and discussed above, an important and yet unresolved question is to which degree the observed zygomatic and periocular EMG activity in response to eye contact reflects also a positive affective reaction. One possibility to investigate this issue, in future studies, would be to present gaze stimuli with very short presentation times (<40 ms) preventing the clear conscious perception of another’s face. This would be possible even with a live interaction partner using a similar methodology (i.e., a liquid crystal shutter) as in the present study. If in a study using very short stimulus presentation times and combining EMG with measurements of autonomic arousal, a central component of the affective response (Plutchik, 1980), the results showed greater zygomatic, periocular, and physiological arousal responses to direct vs. averted gaze, this would providence evidence that positive affective reactions also play a role in the elicitation of smiling responses to eye contact.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by the Academy of Finland under grant #266187 to J.K.H. and the Academy of Finland under grant #307657 to M.J.P.