Gazing into Cognition: Eye Behavior in Online L2 Speaking Tests

References

• Adams, R. B., Nelson, A. J., & Purring, K. (2013). Eye behavior. In J. A. Hall & M. L. Knapp (Eds.), Nonverbal communication (pp. 229–262). De Gruyter.
   Google Scholar
• American Council on the Teaching of Foreign Languages. (2012). ACTFL proficiency guidelines. ACTFL. https://www.actfl.org/sites/default/files/guidelines/ACTFLProficiencyGuidelines2012.pdf
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
   Google Scholar
• Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. https://doi.org/10.1016/j.jml.2007.12.005
   Web of Science ®Google Scholar
• Baltrušaitis, T., Zadeh, A., Lim, Y. C., & Morency, L. P. (2018). OpenFace 2.0: Facial behavior analysis toolkit. 13th IEEE International Conference on Automatic Face & Gesture Recognition. https://doi.org/10.1109/FG.2018.00019
   Google Scholar
• Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
   Web of Science ®Google Scholar
• Beattie, G. (1983). Talk: An analysis of speech and nonverbal behaviour in conversation. Open University Press.
   Google Scholar
• Block, R. A., Hancock, P. A., & Zakay, D. (2010). How cognitive load affects duration judgments: A meta-analytic review. Acta Psychologica, 134(3), 330–343. https://doi.org/10.1016/j.actpsy.2010.03.006
   PubMed Web of Science ®Google Scholar
• Brône, G., & Oben, B. Eds. (2018). Eye-tracking in interaction. Studies on the role of eye gaze in dialogue. John Benjamins.
   Google Scholar
• Brunfaut, T., & McCray, G. (2015). Looking into test-takers’ cognitive processes while completing reading tasks: A mixed-method eye-tracking and stimulated recall study (Report No. AR-G/2015/001). British Council. https://www.britishcouncil.org/sites/default/files/brunfaut_and_mccray_report_final_0.pdf
   Google Scholar
• Burton, J. D. (2021). Question complexity, nonunderstanding, and nonverbal behavior (OSF F2JXK, Version V1) [Data set]. Open Science Framework. https://doi.org/10.17605/OSF.IO/F2JXK
   Google Scholar
• Chapelle, C. A., & Voss, E. (2016). 20 years of technology and language assessment in language learning & technology. Language Learning & Technology, 20(2), 116–128. https://doi.org/10.125/44464
   Web of Science ®Google Scholar
• Council of Europe. (2001). The Common European framework of reference for languages: Learning, teaching and assessment. https://rm.coe.int/1680459f97
   Google Scholar
• Cummins, F. (2012). Gaze and blinking in dyadic conversation: A study in coordinated behaviour among individuals. Language and Cognitive Processes, 27(10), 1525–1549. https://doi.org/10.1080/01690965.2011.615220
   Web of Science ®Google Scholar
• Doherty-Sneddon, G., Bruce, V., Bonner, L., Longbotham, S., & Doyle, C. (2002). Development of gaze aversion as disengagement from visual information. Developmental psychology, 38(3), 438–445. https://doi.org/10.1037//0012-1649.38.3.438
   PubMed Web of Science ®Google Scholar
• Doherty-Sneddon, G., & Phelps, F. G. (2005). Gaze aversion: A response to cognitive or social difficulty? Memory and Cognition, 33(4), 727–733. https://doi.org/10.3758/bf03195338
   PubMed Web of Science ®Google Scholar
• Drew, G. C. (1951). Variations in reflex blink-rate during visual-motor tasks. Quarterly Journal of Experimental Psychology, 3(2), 73–88. https://doi.org/10.1080/17470215108416776
   Web of Science ®Google Scholar
• Ekman, P., & Friesen, W. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1(1), 49–98. https://doi.org/10.1515/9783110880021.57
   Google Scholar
• Elder, C., Iwashita, N., & McNamara, T. (2002). Estimating the difficulty of oral proficiency tasks: What does the test-taker have to offer? Language Testing, 19(4), 347–368. https://doi.org/10.1191/0265532202lt235oa
   Google Scholar
• Ellsworth, P. C. (1975). Direct gaze as a social stimulus: The example of aggression. In P. Pliner, L. Krames, & T. Alloway (Eds.), Nonverbal communication of aggression (pp. 53–76). Plenum Press.
   Google Scholar
• Frosina, P., Logue, M., Book, A., Huizinga, T., Amos, S., & Stark, S. (2018). The effect of cognitive load on nonverbal behavior in the cognitive interview for suspects. Personality and Individual Differences, 130, 51–58. https://doi.org/10.1016/j.paid.2018.03.012
   Web of Science ®Google Scholar
• Fukuda, K. (2001). Eye blinks: New indices for the detection of deception. International Journal of Psychophysiology, 40(3), 239–245. https://doi.org/10.1016/S0167-8760(00)00192-6
   PubMed Web of Science ®Google Scholar
• Glenberg, A. M., Schroeder, J. L., & Robertson, D. A. (1998). Averting the gaze disengages the environment and facilitates remembering. Memory & Cognition, 26(4), 651–658. https://doi.org/10.3758/BF03211385
   PubMed Web of Science ®Google Scholar
• Goldin-Meadow, S., Wein, D., & Chang, C. (1992). Assessing knowledge through gesture: Using children’s hands to read their minds. Cognition and Instruction, 9(3), 201–219. https://doi.org/10.1207/s1532690xci0903_2
   Web of Science ®Google Scholar
• Gregersen, T. S. (2005). Nonverbal cues: Clues to the detection of foreign language anxiety. Foreign Language Annals, 38(3), 388–400. https://doi.org/10.1111/j.1944-9720.2005.tb02225.x
   Web of Science ®Google Scholar
• Housen, A., Kuiken, F., Housen, A., & Kuiken, F. Eds. (2009). Special issue: Complexity, accuracy and fluency in second language acquisition research. Applied Linguistics, 30(4), 461–601. https://doi.org/10.1093/applin/amp048
   Web of Science ®Google Scholar
• Isbell, D. R., & Kremmel, B. (2020). Test review: Current options in at-home language proficiency tests for making high-stakes decisions. Language Testing, 37(4), 600–619. https://doi.org/10.1177/0265532220943483
   Web of Science ®Google Scholar
• Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
   Web of Science ®Google Scholar
• Karsen, C. N., Berman, K. F., Donnelly, E. F., Mendelson, W. B., Kleinman, J. E., & Wyatt, R. J. (1981). Speaking, thinking, and blinking. Psychiatry Research, 5(3), 243–246. https://doi.org/10.1016/0165-1781(81)90070-6
   PubMedGoogle Scholar
• Lee, J. (2019). Task complexity, cognitive load, and L1 speech. Applied Linguistics, 40(3), 506–539. https://doi.org/10.1093/applin/amx054
   Web of Science ®Google Scholar
• Levelt, W. J. M. (1983). Monitoring and self-repair in speech. Cognition, 14(1), 41–104. https://doi.org/10.1016/0010-0277(83)90026-4
   PubMed Web of Science ®Google Scholar
• Levelt, W. J. M. (1989). Speaking: From intention to articulation. MIT Press.
   Google Scholar
• Levelt, W. J. M. (1999). Language production: A blueprint of the speaker. In C. Brown & P. Hagoort (Eds.), Neurocognition of language (pp. 83–122). Oxford University Press.
   Google Scholar
• Lindberg, R., McDonough, K., & Trofimovich, P. (2021). Investigating verbal and nonverbal indicators of physiological response during second language interaction. Applied Psycholinguistics, 42(6), 1403–1425. https://doi.org/10.1017/S014271642100028X
   Google Scholar
• Ma, W., & Winke, P. (2019). Self-assessment: How reliable is it in assessing oral proficiency over time? Foreign Language Annals, 52(1), 66–86. https://doi.org/10.1111/flan.12379
   Web of Science ®Google Scholar
• McDonough, K., Lindberg, R., Trofimovich, P., & Tekin, O. (2021). The visual signature of non-understanding: A systematic replication of McDonough, Trofimovich, Lu, and Abashidze (2019). Language Teaching, Advance Online Publication 1–15. https://doi.org/10.1017/S0261444821000197
   Google Scholar
• McDonough, K., Trofimovich, P., Lu, L., & Abashidze, D. (2019). The occurrence and perception of listener visual cues during nonunderstanding episodes. Studies in Second Language Acquisition, 41(5), 1151–1165. https://doi.org/10.1017/S0272263119000238
   Web of Science ®Google Scholar
• McNeill, D. (2005). Gesture and thought. University of Chicago Press. https://doi.org/10.7208/chicago/9780226514642.001.0001
   Google Scholar
• Michel, M. C. (2011). Effects of task complexity and interaction on L2 performance. In P. Robinson (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 141–174). John Benjamins.
   Google Scholar
• Mondada, L. (2016). Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20(3), 336–366. https://doi.org/10.1111/josl.1_12177
   Web of Science ®Google Scholar
• Nakagawa, S., & Schielzeth, H. (2013). A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods in Ecology and Evolution, 4(2), 133–142. https://doi.org/10.1111/j.2041-210x.2012.00261.x
   Web of Science ®Google Scholar
• Nakano, T., & Kitazawa, S. (2010). Eyeblink entrainment at breakpoints of speech. Experimental Brain Research, 205(4), 577–581. https://doi.org/10.1007/s00221-010-2387-z
   PubMed Web of Science ®Google Scholar
• Nation, I. S. P. (2016). Making and using word lists for language learning and testing. John Benjamins.
   Google Scholar
• Neu, J. (1990). Assessing the role of nonverbal communication in the acquisition of communicative competence in L2. In R. C. Scarcella, E. S. Andersen, & S. D. Krashen (Eds.), Developing communicative competence in a second language (pp. 121–138). Newbury House.
   Google Scholar
• Norris, J., & Ortega, L. (2003). Defining and measuring L2 acquisition. In C. J. Doughty & M. H. Long (Eds.), The handbook of second language acquisition (pp. 717–761). Wiley.
   Google Scholar
• North, B., Ortega, A., & Sheehan, S. (2010). A core inventory for general English. British Council/European Association for Quality Language Services. https://www.teachingenglish.org.uk/sites/teacheng/files/pub-british-council-eaquals-core-inventoryv2.pdf
   Google Scholar
• Ockey, G. J. (2007). Construct implications of including still image or video in computer-based listening tests. Language Testing, 24(4), 517–537. https://doi.org/10.1177/0265532207080771
   Google Scholar
• Plonsky, L., & Ghanbar, H. (2018). Multiple regression in L2 research: A methodological synthesis and guide to interpreting R2 values. The Modern Language Journal, 102(4), 713–731. https://doi.org/10.1111/modl.12509
   Web of Science ®Google Scholar
• Plough, I. (2021). A case for nonverbal behavior: Implications for construct, performance, and assessment. In M. R. Salaberry & A. R. Burch (Eds.), Assessing speaking in context—Expanding the construct and its applications (pp. 50–69). Multilingual Matters.
   Google Scholar
• Plough, I. C., & Bogart, P. S. H. (2008). Perceptions of examiner behavior modulate power relations in oral performance testing. Language Assessment Quarterly, 5(3), 195–217. https://doi.org/10.1080/15434300802229375
   Web of Science ®Google Scholar
• Révész, A. (2014). Towards a fuller assessment of cognitive models of task-based learning: Investigating task-generated cognitive demands and processes. Applied Linguistics, 35(1), 87–92. https://doi.org/10.1093/applin/amt039
   Web of Science ®Google Scholar
• Révész, A., Michel, M., & Gilabert, R. (2016). Measuring cognitive task demands using dual-task methodology, subjective self-ratings, and expert judgements: A validation study. Studies in Second Language Acquisition, 38(4), 703–737. https://doi.org/10.1017/S0272263115000339
   Web of Science ®Google Scholar
• Robinson, P. (2001). Task complexity, task difficulty, and task production: Exploring inter- actions in a componential framework. Applied Linguistics, 22(1), 27–57. https://doi.org/10.1093/applin/22.1.27
   Web of Science ®Google Scholar
• Robinson, P. (2011). Second language task complexity, the cognition hypothesis, language learning, and performance. In P. Robinson (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 3–38). John Benjamins
   Google Scholar
• Rossano, F. (2012). Gaze behavior in face-to-faceinteraction. [Doctoral dissertation, Radboud University]. Radboud Repository. http://hdl.handle.net/2066/99151
   Google Scholar
• Sasayama, S. (2016). Is a ‘complex’ task really complex? Validating the assumption of cognitive task complexity. The Modern Language Journal, 100(1), 231–254. https://doi.org/10.1111/modl.12313
   Web of Science ®Google Scholar
• Segalowitz, N. (2010). Cognitive bases of second language fluency. Routledge.
   Google Scholar
• Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychological Review, 84(2), 127–190. https://doi.org/10.1037/0033-295X.84.2.127
   Web of Science ®Google Scholar
• Skehan, P. (1998). A cognitive approach to language learning. Oxford University Press.
   Google Scholar
• Suvorov, R. (2015). The use of eye tracking in research on video-based second language (L2) listening assessment: A comparison of context videos and content videos. Language Testing, 32(4), 463–483. https://doi.org/10.1177/0265532214562099
   Web of Science ®Google Scholar
• Sweller, J., Chandler, P., Tierney, P., & Cooper, M. (1990). Cognitive load as a factor in the structuring of technical material. Journal of Experimental Psychology: General, 119(2), 176–192. https://doi.org/10.1037/0096-3445.119.2.176
   Web of Science ®Google Scholar
• Tabachnick, B. G., & Fidell, L. S. (2014). Using multivariate statistics. Pearson.
   Google Scholar
• Tsunemoto, A., Lindberg, R., Trofimovich, P., & McDonough, K. (2021). Visual cues and rater perceptions of second language comphrensibility, accentedness, and fluency. Studies in Second Language Acquisition(), 759–787. Advance online publication. https://doi.org/10.1017/S0272263121000425
   Google Scholar
• Vredeveldt, A., Hitch, G. J., & Baddeley, A. D. (2011). Eye closure helps memory by reducing cognitive load and enhancing visualisation. Memory & Cognition, 39(7), 1253–1263. https://doi.org/10.3758/s13421-011-0098-8
   PubMed Web of Science ®Google Scholar
• Warton, D. I., & Hui, F. K. C. (2011). The arcsine is asinine: The analysis of proportions in ecology. Ecology, 92(1), 3–10. https://doi.org/10.1890/10-0340.1
   PubMed Web of Science ®Google Scholar
• Weir, C. J. (2005). Language testing and validation: An evidence-based approach. Palgrave Macmillan. https://doi.org/10.1057/9780230514577
   Google Scholar