The role of pointing gestures and eye gaze in second language vocabulary learning

References

• Allen, L. Q. (1995). The effects of emblematic gestures on the development and access of mental representations of French expressions. The Modern Language Journal, 79(4), 521–529. https://doi.org/10.1111/j.1540-4781.1995.tb05454.x
   Web of Science ®Google Scholar
• Allen, L. Q. (1999). Functions of nonverbal communication in teaching and learning a Foreign language. The French Review, 72, 469–480.
   Web of Science ®Google Scholar
• Atkinson, D. (2010). Extended, embodied cognition and second language acquisition. Applied Linguistics, 31(5), 599–622. https://doi.org/10.1093/applin/amq009
   Web of Science ®Google Scholar
• Atkinson, M. A., Simpson, A. A., & Cole, G. G. (2018). Visual attention and action: How cueing, direct mapping, and social interactions drive orienting. Psychonomic Bulletin & Review, 25(5), 1585–1605. https://doi.org/10.3758/s13423-017-1354-0
   PubMed Web of Science ®Google Scholar
• Baayen, R. H. (2008). Analyzing Linguistic data: A practical introduction to statistics using R. Cambridge University Press. https://doi.org/10.1017/CBO9780511801686
   Google Scholar
• Bang, J. Y., & Nadig, A. (2020). An investigation of word learning in the presence of gaze: Evidence from school-age children with typical development or autism spectrum disorder. Cognitive Development, 54, 100847. https://doi.org/10.1016/j.cogdev.2020.100847
   Web of Science ®Google Scholar
• Barry, R. A., Graf Estes, K., & Rivera, S. M. (2015). Domain general learning: Infants use social and non-social cues when learning object statistics. Frontiers in Psychology, 6, 551. https://doi.org/10.3389/fpsyg.2015.00551
   PubMed Web of Science ®Google Scholar
• Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.
   PubMed Web of Science ®Google Scholar
• Belpaeme, T., Vogt, P., van den Berghe, R., Bergmann, K., Göksun, T., de Haas, M., Kanero, J., Kennedy, J., Küntay, A. C., Oudgenoeg-Paz, O., Papadopoulos, F., Schodde, T., Verhagen, J., Wallbridge, C. D., Willemsen, B., de Wit, J., Geçkin, V., Hoffmann, L., Kopp, S., Pandey, A. K. (2018). Guidelines for designing social robots as second language tutors. International Journal of Social Robotics, 10(3), 325–341. https://doi.org/10.1007/s12369-018-0467-6
   PubMed Web of Science ®Google Scholar
• Bergmann, K., & Macedonia, M. (2013). A virtual agent as vocabulary trainer: Iconic gestures help to improve learners’ memory performance. In R. Aylett, B. Krenn, C. Pelachaud, & H. Shimodaira (Eds). Intelligent virtual agents. IVA 2013. lecture notes in computer science (Vol. 8108, pp. 139–148). Springer. https://doi.org/10.1007/978-3-642-40415-3_12
   Google Scholar
• Böckler, A., van der Wel, R. P. R. D., & Welsh, T. N. (2015). Eyes only? Perceiving eye contact is neither sufficient nor necessary for attentional capture by face direction. Acta Psychologica, 160, 134–140. https://doi.org/10.1016/j.actpsy.2015.07.009
   PubMed Web of Science ®Google Scholar
• Booth, A., McGregor, K., & Rohlfing, K. (2008). Socio-Pragmatics and attention: Contributions to gesturally guided word learning in toddlers. Language Learning and Development, 4, 179–202.
   Google Scholar
• Brodeur, M. B., Guérard, K., & Bouras, M. (2014). Bank of standardized stimuli (boss) phase II: 930 new normative photos. PLOS ONE, 9(9), e106953.
   PubMed Web of Science ®Google Scholar
• Brooks, R., & Meltzoff, A. N. (2008). Infant gaze following and pointing predict accelerated vocabulary growth through two years of age: A longitudinal, growth curve modeling study. Journal of Child Language, 35(1), 207–220. https://doi.org/10.1017/s030500090700829x
   PubMed Web of Science ®Google Scholar
• Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: a tutorial. Journal of Cognition, 1(1), 9. http://doi.org/10.5334/joc.10
   PubMedGoogle Scholar
• Caldano, M., & Coventry, K. R. (2019). Spatial demonstratives and perceptual space: To reach or not to reach? Cognition, 191, 103989. https://doi.org/10.1016/j.cognition.2019.06.001
   PubMed Web of Science ®Google Scholar
• Caruana, N., Inkley, C., & Nalepka, P. (2021). Gaze facilitates responsivity during hand coordinated joint attention. Scientific Reports, 11, 21037. https://doi.org/10.1038/s41598-021-00476-3
   PubMed Web of Science ®Google Scholar
• Çetinçelik, M., Rowland, C. F., & Snijders, T. M. (2021). Do the eyes have it? A systematic review on the role of eye gaze in infant language development. Frontiers in Psychology, 11, 589096. https://doi.org/10.3389/fpsyg.2020.589096
   PubMed Web of Science ®Google Scholar
• Comesaña, M., Soares, A. P., Sánchez-Casas, R., & Lima, C. (2012). Lexical and semantic representations of L2 cognate and noncognate words acquisition in children: Evidence from two learning methods. British Journal of Psychology, 103, 378–392.
   PubMed Web of Science ®Google Scholar
• Cooperrider, K., & Mesh, K. (2022). Pointing in gesture and sign. In A. Morgenstern & S. Goldin-Meadow (Eds.), Gesture in language: Development across the lifespan (pp. 21–46). American Psychological Association. https://doi.org/10.1037/0000269-002
   Google Scholar
• Coventry, K. R., & Garrod, S. C. (2004). Saying, seeing, and acting: The psychological semantics of spatial prepositions. Psychology Press.
   Google Scholar
• Coventry, K. R., Valdés, B., Castillo, A., & Guijarro Fuentes, P. (2008). Language within your reach: Near-far perceptual space and spatial demonstratives. Cognition, 108(3), 889–895. https://doi.org/10.1016/j.cognition.2008.06.010
   PubMed Web of Science ®Google Scholar
• Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11(6), 671. https://doi.org/10.1016/S0022-5371(72)80001-X
   Google Scholar
• Demir-Lira, Ö. E., Kanero, J., Oranç, C., Koskulu, S., Franko, I., Göksun, T., & Küntay, A. C. (2020). L2 Vocabulary teaching by social robots: The role of gestures and on-screen cues as scaffolds. Frontiers in Education, 5, 599–636. https://doi.org/10.3389/feduc.2020.599636
   Google Scholar
• Douglas Fir Group. (2016). A transdisciplinary framework for SLA in a multilingual world. Modern Language Journal, 100, 19–47.
   Web of Science ®Google Scholar
• Ellis, N. C., & Wulff, S. (2015). Second language acquisition. In E. Dabrowska & D. Divjak (Eds.), Handbook of cognitive linguistics (pp. 409–431). De Gruyter Mouton.
   Google Scholar
• García-Gámez, A. B., & Macizo, P. (2020). The way in which foreign words are learned determines their use in isolation and within sentences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 46(2), 364–379. https://doi.org/10.1037/xlm0000721
   PubMed Web of Science ®Google Scholar
• Geusebroek, J. M., Burghouts, G. J., & Smeulders, A. W. M. (2005). The Amsterdam library of object images. International Journal of Computer Vision, 61(1), 103–112. https://doi.org/10.1023/B:VISI.0000042993.50813.60
   Web of Science ®Google Scholar
• Gliga, T., Elsabbagh, M., Andravizou, A., & Johnsn, M. (2010). Faces attract infants’ attention in complex displays. Infancy, 15(5), 550–562.
   Google Scholar
• Goldin-Meadow, S. (2007). Pointing sets the stage for learning language—and creating language. Child Development, 78, 741–745. https://doi.org/10.1111/j.1467-8624.2007.01029.x
   PubMed Web of Science ®Google Scholar
• Golinkoff, R. M., & Hirsh-Pasek, K. (2006). The emergentist coalition model of word learning in children has implications for language in aging. In E. Bialystok & F. I. M. Craik (Eds.), Lifespan cognition: Mechanisms of change (pp. 207–222). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195169539.003.0014
   Google Scholar
• Grzyb, B. J., & Vigliocco, G. (2020). Beyond robotic speech: Mutual benefits to cognitive psychology and artificial intelligence from the study of multimodal communication. In S. Muggleton & N. Chater (Eds.), Human-Like Machine intelligence (pp. 274–294). Oxford Academic. https://doi.org/10.1093/oso/9780198862536.003.0014
   Google Scholar
• Haxby, J. V., Gobbini, M. I., & Nastase, S. A. (2020). Naturalistic stimuli reveal a dominant role for agentic action in visual representation. NeuroImage, 216, 116561. https://doi.org/10.1016/j.neuroimage.2020.116561
   PubMed Web of Science ®Google Scholar
• Heuven, W. J. B., Mandera, P., Keuleers, E., & Brysbaert, M. (2014). Subtlex-UK: A new and improved word frequency database for British English. Quarterly Journal of Experimental Psychology, 67(6), 1176–1190. https://doi.org/10.1080/17470218.2013.850521
   Web of Science ®Google Scholar
• Holler, J., & Levinson, S. C. (2019). Multimodal language processing in human communication. Trends in Cognitive Sciences, 23(8), 639–652. https://doi.org/10.1016/j.tics.2019.05.006
   PubMed Web of Science ®Google Scholar
• Holler, J., Schubotz, L., Kelly, S., Hagoort, P., Schuetze, M., & Özyürek, A. (2014). Social eye gaze modulates processing of speech and co-speech gesture. Cognition, 133(3), 692–697. https://doi.org/10.1016/j.cognition.2014.08.008
   PubMed Web of Science ®Google Scholar
• Jachmann, T. K., Drenhaus, H., Staudte, M., & Crocker, M. W. (2019). Influence of speakers’ gaze on situated language comprehension: Evidence from event-related potentials. Brain and Cognition, 135, 103571. https://doi.org/10.1016/j.bandc.2019.05.009
   PubMed Web of Science ®Google Scholar
• Kendon, A. (1980). Gesticulation and Speech: Two aspects of the process of utterance. In Mary R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207–228). De Gruyter Mouton.
   Google Scholar
• Keuleers, E., & Brysbaert, M. (2010). Wuggy: A multilingual pseudoword generator. Behavior Research Methods, 42(3), 627–633. https://doi.org/10.3758/BRM.42.3.627
   PubMed Web of Science ®Google Scholar
• Knoeferle, P., Kreysa, H., &Pickering, M. (2018). Effects of a speaker’s gaze on language comprehension and acquisition: Studies on the role of eye gaze in dialogue. In G. Brone & B. Oben (Eds.), Advances in interaction studies: Eye-tracking in interaction(pp. 47–66). Benjamins.
   Google Scholar
• Krashen, S. D. (1988). Second language Acquisition and second language learning. Prentice Hall.
   Google Scholar
• Kreysa, H., Nunnemann, E. M., & Knoeferle, P. (2018). Distinct effects of different visual cues on sentence comprehension and later recall: The case of speaker gaze versus depicted actions. Acta Psychologica, 188, 220–229. https://doi.org/10.1016/j.actpsy.2018.05.001
   PubMed Web of Science ®Google Scholar
• Kroll, J. F., & Stewart, E. (1994). Category interference in translation and picture naming: Evidence for asymmetric connections between bilingual memory representations. Journal of Memory and Language, 33, 149–174.
   Web of Science ®Google Scholar
• Langton, S. R. H., Watt, R. J., & Bruce, V. (2000). Do the eyes have it? Cues to the direction of social cognition. Trends in Cognitive Science S, 4(2), 50–59. https://doi.org/10.1016/S1364-6613(99)01436-9
   PubMed Web of Science ®Google Scholar
• Law, B., Houston-Price, C., & Loucas, T. (2012). Using gaze direction to learn words at 18 months: Relationships with later vocabulary. Language Studies Working Papers, 4, 3–14. https://www.reading.ac.uk/elal/our-research/LSWP
   Google Scholar
• Macdonald, R. G., & Tatler, B. W. (2013). Do as eye say: Gaze cueing and language in a real-world social interaction. Journal of Vision, 13(4), 6. https://doi.org/10.1167/13.4.6
   PubMed Web of Science ®Google Scholar
• Macedonia, M., Müller, K., & Friederici, A. D. (2011). The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping, 32(6), 982–998. https://doi.org/10.1002/hbm.21084
   PubMed Web of Science ®Google Scholar
• MacWhinney, B. (2012). The Logic of the Unified Model. In S. Gass & A. Mackey (Eds.), Handbook of Second Language Acquisition (pp. 211–227). Routledge.
   Google Scholar
• Matsumoto, Y., & Dobs, A. M. (2017). Pedagogical gestures as interactional resources for teaching and learning tense and aspect in the esl grammar classroom. Language Learning, 67, 7–42. https://doi.org/10.1111/lang.12181
   Web of Science ®Google Scholar
• McGillion, M., Herbert, J. S., Pine, J., Vihman, M., dePaolis, R., Keren-Portnoy, T., & Matthews, D. (2017). What paves the way to conventional language? The predictive value of babble, pointing, and socioeconomic status. Child Development, 88(1), 156–166. https://doi.org/10.1111/cdev.12671
   PubMed Web of Science ®Google Scholar
• McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago Press.
   Google Scholar
• Michel, C., Wronski, C., Pauen, S., Daum, M. M., & Hoehl, S. (2019). Infants’ object processing is guided specifically by social cues. Neuropsychologia, 126, 54–61. https://doi.org/10.1016/j.neuropsychologia.2017.05.022
   PubMed Web of Science ®Google Scholar
• Mitchell, R., Myles, F., & Marsden, E. (2019). Second language learning theories (4th ed.). Routledge.
   Google Scholar
• Morett, L. M. (2014). When hands speak louder than words: The role of gesture in the communication, encoding, and recall of words in a novel second language. Modern Language Journal, 98(3), 834–853. https://doi.org/10.1111/modl.12125
   Web of Science ®Google Scholar
• Murgiano, M., Motamedi, Y. S., & Vigliocco, G. (2021). Situating Language in the Real-World: The Role of multimodal iconicity and indexicality. Journal of Cognition, 4(1), 1–18. https://doi.org/10.5334/joc.113
   PubMedGoogle Scholar
• Norris, S. (2011). Three hierarchical positions of deictic gesture in relation to spoken language: A multimodal interaction analysis. Visual Communication, 10(2), 129–147. https://doi.org/10.1177/1470357211398439
   Web of Science ®Google Scholar
• Ortega, L. (2013). SLA for the 21st Century: Disciplinary progress, transdisciplinary relevance, and the Bi/multilingual Turn. Language Learning, 63, 1–24. https://doi.org/10.1111/j.1467-9922.2012.00735.x
   Web of Science ®Google Scholar
• Paivio, A. (1971). Imagery and Verbal Processes. Holt, Rinehart & Winston.
   Google Scholar
• Paulus, M., & Fikkert, P. (2014). Conflicting social Cues: Fourteen- and 24-month-old infants’ reliance on gaze and pointing cues in word learning. Journal of Cognition and Development, 15(1), 43–59. https://doi.org/10.1080/15248372.2012.698435
   Web of Science ®Google Scholar
• Piwek, P., Beun, R. J., & Cremers, A. (2008). ’Proximal’ and ‘distal’ in language and cognition: Evidence from deictic demonstratives in Dutch. Journal of Pragmatics, 40(4), 694–718. https://doi.org/10.1016/j.pragma.2007.05.001
   Web of Science ®Google Scholar
• Poarch, G. J., Van Hell, J. G., & Kroll, J. F. (2015). Accessing word meaning in beginning second language learners: Lexical or conceptual mediation? Bilingualism: Language and Cognition, 18(3), 357–371. https://doi.org/10.1017/S1366728914000558
   Web of Science ®Google Scholar
• Pulvermüller, F., Hauk, O., Nikulin, V. V., & Ilmoniemi, R. J. (2005). Functional links between motor and language systems. European Journal of Neuroscience, 21, 793–797. https://doi.org/10.1111/j.1460-9568.2005.03900.x
   PubMed Web of Science ®Google Scholar
• Ro, T., Russell, C., & Lavie, N. (2001). Changing faces: A detection advantage in the flicker paradigm. Psychological Science, 12, 94–99. https://doi.org/10.1111/1467-9280.00317
   PubMed Web of Science ®Google Scholar
• Rocca, R., Wallentin, M., Vesper, C., & Tylén, K. (2019). This is for you: Social modulations of proximal vs. distal space in collaborative interaction. Scientific Reports, 9, 14967. https://doi.org/10.1038/s41598-019-51134-8
   PubMed Web of Science ®Google Scholar
• Scott, G. G., Keitel, A., Becirspahic, M., Yao, B., & Sereno, S. C. (2019). The Glasgow norms: Ratings of 5,500 words on nine scales. Behavior Research Methods, 1258–1270. https://doi.org/10.3758/s13428-018-1099-3
   PubMed Web of Science ®Google Scholar
• Shattuck-Hufnagel, S., Ren, A., Mathew, M., Yuen, I., & Demuth, K. (2016). Non-referential gestures in adult and child speech: Are they prosodic. In Proceedings from the 8th international conference on speech prosody (pp. 836–839). Boston: Boston University.
   Google Scholar
• Stam, G., & Tellier, M. (2022). Gesture helps second and Foreign Language learning and teaching. In A. Morgenstern & S. Goldin-Meadow (Eds.), Gesture in Language: Development Across the Lifespan (pp. 335–364). De Gruyter Mouton. https://doi.org/10.1515/9783110567526-014
   Google Scholar
• Theeuwes, J., & Van Der Stigchel, S. (2006). Faces capture attention: Evidence from inhibition of return. Visual Cognition, 13, 657–665. https://doi.org/10.1080/13506280500410949
   Web of Science ®Google Scholar
• Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. (2005). Understanding and sharing intentions: The origins of cultural cognition. Behavioral and Brain Sciences, 28, 675–735. https://doi.org/10.1017/S0140525X05000129
   PubMed Web of Science ®Google Scholar
• Tomlin, R. S., & Villa, V. M. (1994). Attention in cognitive science and second language acquisition. Studies in Second Language Acquisition, 16, 183–203. https://doi.org/10.1017/S0272263100012870
   Google Scholar
• Tsuji, S., Fiévet, A., & Cristia, A. (2021). Toddler word learning from contingent screens with and without human presence. Infant Behavior & Development, 63, 101553. https://doi.org/10.1016/j.infbeh.2021.101553
   PubMed Web of Science ®Google Scholar
• Vogt, P., van den Berghe, R., de Haas, M., Hoffman, L., Kanero, J., Mamus, E., Montanier, J., Oranç, C., Oudgenoeg-Paz, O., García, D. H, Papadopoulos, P., Schodde, T., Verhagen, J., Wallbridge, C. D., Willemsen, B., de Wit, J., Belpaeme, T., Göksun, T., Kopp, S., Krahmer, E., ... Pandey, A. K. Second Language Tutoring Using Social Robots: A Large-Scale Study. In Proceedings of the 2019 14th ACM/IEEEInternational Conference on Human-Robot Interaction (HRI), Daegu, Korea, 11–14 March 2019.
   Google Scholar
• Welke, D., & Vessel, E. A. (2022). Naturalistic viewing conditions can increase task engagement and aesthetic preference but have only minimal impact on EEG quality. NeuroImage, 256, 119218. https://doi.org/10.1016/J.NEUROIMAGE.2022.119218
   PubMed Web of Science ®Google Scholar
• Zhang, Y., Frassinelli, D., Tuomainen, J., Skipper, J. I., & Vigliocco, G. (2021). More than words: The online orchestration of word predictability, prosody, gesture, and mouth movements during natural language comprehension. Proceedings of the Royal Society B, 288(1955), 20210500. https://doi.org/10.1101/2020.01.08.896712
   Google Scholar