The mutual influence of an instructor’s eye gaze and facial expression in video lectures

References

• Ayres, P., & Sweller, J. (2014). The split-attention principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 206–226). Cambridge University Press.
   Google Scholar
• Baron-Cohen, S. (1995). Mindblindness: An essay on autism and theory of mind. The MIT Press.
   Google Scholar
• Bayless, S. J., Glover, M., Taylor, M. J., & Itier, R. J. (2011). Is it in the eyes? Dissociating the role of emotion and perceptual features of emotionally expressive faces in modulating orienting to eye gaze. Visual Cognition, 19(4), 483–510. https://doi.org/10.1080/13506285.2011.552895
   PubMed Web of Science ®Google Scholar
• Beege, M., Nebel, S., Schneider, S., & Rey, G. D. (2019). Social entities in educational videos: Combining the effects of addressing and professionalism. Computers in Human Behavior, 93, 40–52. https://doi.org/10.1016/j.chb.2018.11.051
   Web of Science ®Google Scholar
• Beege, M., Schneider, S., Nebel, S., & Rey, G. D. (2017). Look into my eyes! Exploring the effect of addressing in educational videos. Learning and Instruction, 49, 113–120. https://doi.org/10.1016/j.learninstruc.2017.01.004
   Web of Science ®Google Scholar
• Bindemann, M., Mike Burton, A., & Langton, S. R. (2008). How do eye gaze and facial expression interact? Visual Cognition, 16(6), 708–733. https://doi.org/10.1080/13506280701269318
   Web of Science ®Google Scholar
• Bookbinder, S. H., & Brainerd, C. J. (2017). Emotionally negative pictures enhance gist memory. Emotion, 17(1), 102–119. https://doi.org/10.1037/emo0000171
   PubMed Web of Science ®Google Scholar
• Bosch, N., Mills, C., Wammes, J. D., & Smilek, D. (2018). Quantifying classroom instructor dynamics with computer vision. In C. Penstein Rosé, R. Martínez-Maldonado, H. U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, & B. du Boulay (Eds.), et al. (Eds.), Artificial Intelligence in education (pp. 30–42). London: Springer International Publishing.
   Google Scholar
• Calvo, M. G., & Lang, P. J. (2004). Gaze patterns when looking at emotional pictures: Motivationally biased attention. Motivation and Emotion, 28(3), 221–243. https://doi.org/10.1023/B:MOEM.0000040153.26156.ed
   Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
   Google Scholar
• Conty, L., N’Diaye, K., Tijus, C., & George, N. (2007). When eye creates the contact! ERP evidence for early dissociation between direct and averted gaze motion processing. Neuropsychologia, 45(13), 3024–3037. https://doi.org/10.1016/j.neuropsychologia.2007.05.017
   PubMed Web of Science ®Google Scholar
• Earles, J. L., Kersten, A. W., Vernon, L. L., & Starkings, R. (2016). Memory for positive, negative and neutral events in younger and older adults: Does emotion influence binding in event memory? Cognition and Emotion, 30(2), 378–388. https://doi.org/10.1080/02699931.2014.996530
   PubMed Web of Science ®Google Scholar
• Farroni, T., Csibra, G., Simion, F., & Johnson, M. H. (2002). Eye contact detection in humans from birth. Proceedings of the National Academy of Sciences, 99(14), 9602–9605. https://doi.org/10.1073/pnas.152159999
   PubMed Web of Science ®Google Scholar
• Fiorella, L., Stull, A. T., Kuhlmann, S., & Mayer, R. E. (2019). Instructor presence in video lectures: The role of dynamic drawings, eye contact, and instructor visibility. Journal of Educational Psychology, 111(7), 1162–1171. https://doi.org/10.1037/edu0000325
   Web of Science ®Google Scholar
• Fiori, K. L., Brown, E. E., Cortina, K. S., & Antonucci, T. C. (2006). Locus of control as a mediator of the relationship between religiosity and life satisfaction: Age, race, and gender differences. Mental Health Religion & Culture, 9(3), 239–263. https://doi.org/10.1080/13694670600615482
   Google Scholar
• Fox, E., Mathews, A., Calder, A. J., & Yiend, J. (2007). Anxiety and sensitivity to gaze direction in emotionally expressive faces. Emotion, 7(3), 478–486. https://doi.org/10.1037/1528-3542.7.3.478
   PubMed Web of Science ®Google Scholar
• Frischen, A., Bayliss, A. P., & Tipper, S. P. (2007). Gaze cuing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133(4), 694–724. https://doi.org/10.1037/0033-2909.133.4.694
   PubMed Web of Science ®Google Scholar
• George, N., Driver, J., & Dolan, R. J. (2001). Seen gaze-direction modulates fusiform activity and its coupling with other brain areas during face processing. Neuroimage, 13(6), 1102–1112. https://doi.org/10.1006/nimg.2001.0769
   PubMed Web of Science ®Google Scholar
• Graham, R., Friesen, C. K., Fichtenholtz, H. M., & LaBar, K. S. (2010). Modulation of reflexive orienting to gaze direction by facial expressions. Visual Cognition, 18(3), 331–368. https://doi.org/10.1080/13506280802689281
   Web of Science ®Google Scholar
• Graham, R., & LaBar, K. S. (2007). Garner interference reveals dependencies between emotional expression and gaze in face perception. Emotion, 7(2), 296–313. https://doi.org/10.1037/1528-3542.7.2.296
   PubMed Web of Science ®Google Scholar
• Holmes, A., Richards, A., & Green, S. (2006). Anxiety and sensitivity to eye gaze in emotional faces. Brain and Cognition, 60(3), 282–294. https://doi.org/10.1016/j.bandc.2005.05.002
   PubMed Web of Science ®Google Scholar
• Homer, B. D., Plass, J. L., & Blake, L. (2008). The effects of video on cognitive load and social presence in multimedia-learning. Computers in Human Behavior, 24(3), 786–797. https://doi.org/10.1016/j.chb.2007.02.009
   Web of Science ®Google Scholar
• Jackson, M. C. (2018). Eye gaze influences working memory for happy but not angry faces. Cognition and Emotion, 32(4), 719–728. https://doi.org/10.1080/02699931.2017.1345720
   PubMed Web of Science ®Google Scholar
• Jung, Y., & Lee, J. (2018). Learning engagement and persistence in massive open online courses (MOOCS). Computers & Education, 122, 9–22. https://doi.org/10.1016/j.compedu.2018.02.013
   Web of Science ®Google Scholar
• Kampe, K., Frith, C., Dolan, R., & Frith, U. (2001). Psychology: Reward value of attractiveness and gaze. Nature, 413(6856), 589. https://doi.org/10.1038/35098149
   PubMed Web of Science ®Google Scholar
• Kizilcec, R. F., Bailenson, J. N., & Gomez, C. J. (2015). The instructor’s face in video instruction: Evidence from two large-scale field studies. Journal of Educational Psychology, 107(3), 724–739. https://doi.org/10.1037/edu0000013
   Web of Science ®Google Scholar
• Kizilcec, R. F., Papadopoulos, K., & Sritanyaratana, L. (2014). Showing face in video instruction: Effects on information retention, visual attention, and affect. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (pp. 2095–2102). ACM.
   Google Scholar
• Labar, K. S., & Cabeza, R. (2006). Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience, 7(1), 54–64. https://doi.org/10.1038/nrn1825
   PubMed Web of Science ®Google Scholar
• Lanthier, S. N., Jarick, M., Zhu, M. J. H., Byun, C. S. J., & Kingstone, A. (2019). Socially communicative eye contact and gender affect memory. Frontiers in Psychology, 10, 1128. https://doi.org/10.3389/fpsyg.2019.01128
   PubMed Web of Science ®Google Scholar
• Lassalle, A., & Itier, R. J. (2013). Fearful, surprised, happy, and angry facial expressions modulate gaze-oriented attention: Behavioral and ERP evidence. Social Neuroscience, 8(6), 583–600. https://doi.org/10.1080/17470919.2013.835750
   PubMed Web of Science ®Google Scholar
• Leal Rato, M., Mares, I., Aguiar de Sousa, D., Senju, A., & Pavao Martins, I. (2019). Direct gaze partially overcomes hemispatial neglect and captures spatial attention. Frontiers in Psychology, 9, 2702. https://doi.org/10.3389/fpsyg.2018.02702
   PubMed Web of Science ®Google Scholar
• Lee, D., Lee, G., H, K., & Lee, S. Y. (2015). Implicit shopping intention recognition with eye tracking data and response time. In Proceedings of the 3rd International Conference on Human-Agent Interaction (HAI ‘15). Association for Computing Machinery, pp. 295–298.
   Google Scholar
• Leong, V., Byrne, E., Clackson, K., Georgieva, S., Lam, S., & Wass, S. (2017). Speaker gaze increases information coupling between infant and adult brains. Proceedings of the National Academy of Sciences, 114(50), 13290–13295. https://doi.org/10.1073/pnas.1702493114
   PubMed Web of Science ®Google Scholar
• Mayer, R. E. (2014). Principles of multimedia learning based on social cues: Personalization, voice, image, and embodiment principles. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 345–368). Cambridge University Press.
   Google Scholar
• McCrackin, S. D., & Itier, R. J. (2018). Both fearful and happy expressions interact with gaze direction by 200 ms SOA to speed attention orienting. Visual Cognition, 26(4), 231–252. https://doi.org/10.1080/13506285.2017.1420118
   Web of Science ®Google Scholar
• McCrackin, S. D., & Itier, R. J. (2019a). Perceived gaze direction differentially affects discrimination of facial emotion, attention and gender–an ERP study. Part of the research topic special issue on typical and atypical processing of gaze. Frontiers in Neuroscience, 13, 517. https://doi.org/10.3389/fnins.2019.00517
   PubMed Web of Science ®Google Scholar
• McCrackin, S. D., & Itier, R. J. (2019b). Individual differences in the emotional modulation of gaze-cuing. Cognition and Emotion, 33(4), 768–800. https://doi.org/10.1080/02699931.2018.1495618
   PubMed Web of Science ®Google Scholar
• Montague, D. P. F., & Walker-Andrews, A. S. (2001). Peekaboo: A new look at infants’ perception of emotion expressions. Developmental Psychology, 37(6), 826–838. https://doi.org/10.1037/0012-1649.37.6.826
   PubMed Web of Science ®Google Scholar
• Neath, K., Nilsen, E. S., Gittsovich, K., & Itier, R. J. (2013). Attention orienting by gaze and facial expressions across development. Emotion, 13(3), 397–408. https://doi.org/10.1037/a0030463
   PubMed Web of Science ®Google Scholar
• Niedenthal, P. M. (2007). Embodying emotion. Science, 316(5827), 1002–1005. https://doi.org/10.1126/science.1136930
   PubMed Web of Science ®Google Scholar
• Ohue, T., Moriyama, M., & Nakaya, T. (2015). Development of the irrational belief test for nurses (IBTN): Examination of reliability and validity. Health, 7(3), 316–327. https://doi.org/10.4236/health.2015.73036
   Google Scholar
• Okruszek, L., Bala, A., Dziekan, M., Szantroch, M., Rysz, A., Marchel, A., & Hyniewska, S. (2017). Gaze matters! The effect of gaze direction on emotional enhancement of memory for faces in patients with mesial temporal lobe epilepsy. Epilepsy & Behavior, 72, 35–38. https://doi.org/10.1016/j.yebeh.2017.04.016
   PubMed Web of Science ®Google Scholar
• Ouwehand, K., van Gog, T., & Paas, F. (2015). Designing effective video-based modeling examples using gaze and gesture cues. Educational Technology & Society, 18(4), 78–88. https://www.jstor.org/stable/jeductechsoci.18.4.78
   Web of Science ®Google Scholar
• Palmer, C. J., & Clifford, C. W. G. (2018). Adaptation to other people’s eye gaze reflects habituation of high-level perceptual representations. Cognition, 180, 82–90. https://doi.org/10.1016/j.cognition.2018.07.005
   PubMed Web of Science ®Google Scholar
• Pi, Z., Chen, M., Zhu, F., Yang, J., & Hu, W. (2020). Modulation of instructor’s eye gaze by facial expression in video lectures. Innovations in Education and Teaching International, 1–9. Advance Online Publication. https://doi.org/10.1080/14703297.2020.1788410
   Google Scholar
• Pi, Z., & Hong, J. (2016). Learning process and learning outcomes of video podcasts including the instructor and PPT slides: A Chinese case. Innovations in Education and Teaching International, 53(2), 135–144. https://doi.org/10.1080/14703297.2015.1060133
   Web of Science ®Google Scholar
• Pi, Z., Xu, K., Liu, C., & Yang, J. (2020). Instructor presence in video lectures: Eye gaze matters, but not body orientation. Computers & Education, 144, 103713. https://doi.org/10.1016/j.compedu.2019.103713
   Web of Science ®Google Scholar
• Pi, Z., Zhang, Y., Zhu, F., Xu, K., Yang, J., & Hu, W. (2019). Instructors’ pointing gestures improve learning regardless of their use of directed gaze in video lectures. Computers & Education, 128, 345–352. https://doi.org/10.1016/j.compedu.2018.10.006
   Web of Science ®Google Scholar
• Pouw, W., van Gog, T., Zwaan, R. A., Agostinho, S., & Paas, F. (2018). Co-thought gestures in children’s mental problem solving: Prevalence and effects on subsequent performance. Applied Cognitive Psychology, 32(1), 66–80. https://doi.org/10.1002/acp.3380
   Web of Science ®Google Scholar
• Rösler, A., Ulrich, C., Billino, C., Sterzer, J., Weidauer, P., Bernhardt T., Steinmetz, H., Frölich, L., & Kleinschmidt, A. (2005). Effects of arousing emotional scenes on the distribution of visuospatial attention: Changes with aging and early subcortical vascular dementia. Journal of the Neurological Sciences, 229-230, 109–116. https://doi.org/10.1016/j.jns.2004.11.007
   PubMed Web of Science ®Google Scholar
• Striano, T., Kopp, F., Grossmann, T., & Reid, V. M. (2006). Eye contact influences neural processing of emotional expressions in 4-month-old infants. Social Cognitive and Affective Neuroscience, 1(2), 87–94. https://doi.org/10.1093/scan/nsl008
   PubMed Web of Science ®Google Scholar
• Stull, A. T., Fiorella, L., & Mayer, R. E. (2018). An eye-tracking analysis of instructor presence in video lectures. Computers in Human Behavior, 88, 263–272. https://doi.org/10.1016/j.chb.2018.07.019
   Web of Science ®Google Scholar
• Ulloa, J. L., Puce, A., Hugueville, L., & George, N. (2014). Sustained neural activity to gaze and emotion perception in dynamic social scenes. Social Cognitive and Affective Neuroscience, 9(3), 350–357. https://doi.org/10.1093/scan/nss141
   PubMed Web of Science ®Google Scholar
• van Wermeskerken, M., Ravensbergen, S., & van Gog, T. (2018). Effects of instructor presence in video modeling examples on attention and learning. Computers in Human Behavior, 89, 430–438. https://doi.org/10.1016/j.chb.2017.11.038
   Web of Science ®Google Scholar
• van Wermeskerken, M., & van Gog, T. (2017). Seeing the instructor’s face and gaze in demonstration video examples affects attention allocation but not learning. Computers & Education, 113, 98–107. https://doi.org/10.1016/j.compedu.2017.05.013
   Web of Science ®Google Scholar
• Vogl, E., Pekrun, R., Murayama, K., & Loderer, K. (2020). Surprised–curious–confused: Epistemic emotions and knowledge exploration. Emotion, 20(4), 625–641. https://doi.org/10.1037/emo0000578
   PubMed Web of Science ®Google Scholar
• Vuilleumier, P. (2002). Facial expression and selective attention. Current Opinion in Psychiatry, 15(3), 291–300. https://doi.org/10.1097/00001504-200205000-00011
   Web of Science ®Google Scholar
• Wang, J., & Antonenko, P. D. (2017). Instructor presence in instructional video: Effects on visual attention, recall, and perceived learning. Computers in Human Behavior, 71, 79–89. https://doi.org/10.1016/j.chb.2017.01.049
   Web of Science ®Google Scholar
• Wang, Y., Liu, Q., Chen, W., Wang, Q., & Stein, D. (2019). Effects of instructor’s facial expressions on students’ learning with video lectures. British Journal of Educational Technology, 50(3), 1381–1395. https://doi.org/10.1111/bjet.12633
   Web of Science ®Google Scholar
• Wang, H., Pi, Z., & Hu, W. (2019). The instructor’s gaze guidance in video lectures improves learning. Journal of Computer Assisted Learning, 35(1), 42–50. https://doi.org/10.1111/jcal.12309
   Web of Science ®Google Scholar
• Weidlich, J., & Bastiaens, T. J. (2017). Explaining social presence and the quality of online learning with the sips model. Computers in Human Behavior, 72, 479–487. https://doi.org/10.1016/j.chb.2017.03.016
   Web of Science ®Google Scholar
• Zhang, J., Bourguet, M. L., & Venture, G. (2018). The effects of video instructor’s body language on students’ distribution of visual attention: An eye-tracking study. In Proceedings of the 32nd International BCS human Computer Interaction Conference (p. 101). BCS Learning & Development.
   Google Scholar