Sensitivity to salience: linguistic vs. visual cues affect sentence processing and pronoun resolution

References

• Ariel, M. (1988). Referring and accessibility. Journal of Linguistics, 24(1), 65–87.
   Web of Science ®Google Scholar
• Ariel, M. (2001). Accessibility theory: An overview. In T. Sanders, J. Schilperoord, & W. Spooren (Eds.), Text representation: Linguistic and psycholinguistic aspects (pp. 29–88). Amsterdam: John Benjamins Publishing Company.
   Google Scholar
• Arnold, J. E. (1998). Reference form and discourse patterns (Dissertation). Stanford University.
   Google Scholar
• Arnold, J. E., & Griffin, Z. M. (2007). The effect of additional characters on choice of referring expression: Everyone counts. Journal of Memory and Language, 56(4), 521–536.
   PubMed Web of Science ®Google Scholar
• Arnold, J. E., & Lao, S.-Y. C. (2015). Effects of psychological attention on pronoun comprehension. Language, Cognition and Neuroscience, 30(7), 832–852. doi: 10.1080/23273798.2015.1017511
   PubMed Web of Science ®Google Scholar
• Baayen, R. H. (2008). Analysing linguistic data. Cambridge: Cambridge University Press.
   Google Scholar
• Bader, M., & Häussler, J. (2010). Word order in German: A corpus study. Lingua. International Review of General Linguistics. Revue internationale De Linguistique Generale, 120(3), 717–762.
   Web of Science ®Google Scholar
• Baldwin, D. A. (1995). Understanding the link between joint attention and language. In C. Moore, & P. J. Dunham (Eds.), Joint attention: Its origins and role in development. (pp. 131–158). New York: Psychology Press.
   Google Scholar
• Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). arXiv preprint arXiv:1506.04967.
   Google Scholar
• Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1). doi: 10.18637/jss.v067.i01
   Web of Science ®Google Scholar
• Bock, J. K., Irwin, D. E., & Davidson, D. J. (2004). Putting first things first. In J. M. Henderson, & F. Ferreira (Eds.), The interface of language, vision, and action: Eye movements and the visual world. (pp. 249–278). New York: Psychology Press.
   Google Scholar
• Bock, K. J., & Warren, R. K. (1985). Conceptual accessibility and syntactic structure in sentence formulation. Cognition, 21(1), 47–67. doi: 10.1016/0010-0277(85)90023-X
   PubMed Web of Science ®Google Scholar
• Bornkessel, I., & Schlesewsky, M. (2006). The role of contrast in the local licensing of scrambling in German: Evidence from online comprehension. Journal of Germanic Linguistics, 18(1), 1–43.
   Google Scholar
• Bosch, P., Katz, G., & Umbach, C. (2007). The Non-subject bias of German demonstrative pronouns. In M. Schwarz-Friesel, M. Consten, & M. Knees (Eds.), Anaphors in text: Cognitive, formal and applied approaches to anaphoric reference. (pp. 145–164). Amsterdam: John Benjamins Publishing Company.
   Google Scholar
• Bosch, P., & Umbach, C. (2007). Reference determination for demonstrative pronouns. ZAS Papers in Linguistics, 48, 39–51.
   Google Scholar
• Bouma, G., & Hopp, H. (2007). Coreference preferences for personal pronouns in German. ZAS Papers in Linguistics, 48, 53–74.
   Google Scholar
• Bower, G. H., & Morrow, D. G. (1990). Mental models in narrative comprehension. Science, 247(4938), 44–48.
   PubMed Web of Science ®Google Scholar
• Box, G. E. P., & Cox, D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society. Series B (Methodological), 26(2), 211–252. Retrieved from http://www.jstor.org/stable/2984418
   Web of Science ®Google Scholar
• Burkhardt, P., & Roehm, D. (2007). Differential effects of saliency: An event-related brain potential study. Neuroscience Letters, 413(2), 115–120. doi: 10.1016/j.neulet.2006.11.038
   PubMed Web of Science ®Google Scholar
• Burmester, J., Spalek, K., & Wartenburger, I. (2014). Context updating during sentence comprehension: The effect of aboutness topic. Brain & Language, 137, 62–76. doi: 10.1016/j.bandl.2014.08.001
   PubMed Web of Science ®Google Scholar
• Chafe, W. L. (1976). Givenness, contrastiveness, definiteness, subjects, topics, and point of view. In C. N. Li (Ed.), Subject and topic (pp. 26–55). New York: Academic Press.
   Google Scholar
• Chiarcos, C. (2009). Mental salience and grammatical form: Toward a framework for salience metrics in natural language generation (Dissertation). Universität Potsdam.
   Google Scholar
• Colonna, S., Schimke, S., & Hemforth, B. (2012). Information structure effects on anaphora resolution in German and French: A crosslinguistic study of pronoun resolution. Linguistics, 50(5), 901–1073.
   Web of Science ®Google Scholar
• Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorials in Quantitative Methods for Psychology, 1(1), 42–45.
   Google Scholar
• Cowles, W. (2007). The influence of “aboutness” on pronominal coreference. ZAS Papers in Linguistics, pp. 23–38.
   Google Scholar
• Cowles, H. W., & Ferreira, V. S. (2012). The influence of topic status on written and spoken sentence production. Discourse Processes, 49(1), 1–28. doi: 10.1080/0163853X.2011.635989
   Web of Science ®Google Scholar
• Cowles, H. W., Walenski, M., & Kluender, R. (2007). Linguistic and cognitive prominence in anaphor resolution: Topic, contrastive focus and pronouns. Topoi, 26(1), 3–18. doi: 10.1007/s11245-006-9004-6
   Web of Science ®Google Scholar
• Crocker, M. W., Knoeferle, P., & Mayberry, M. R. (2010). Situated sentence processing: The coordinated interplay account and a neurobehavioral model. Brain & Language, 112(3), 189–201. doi: 10.1016/j.bandl.2009.03.004
   PubMed Web of Science ®Google Scholar
• Dahan, D., Tanenhaus, M. K., & Salverda, A. P. (2007). How visual information influences phonetically-driven saccades to pictures: Effects of preview and position in display. In R. van Gompel, M. Fischer, W. Murray, & R. Hill (Eds.), Eyemovements: A window on mind and brain (pp. 471–486). Elsevier: Oxford.
   Google Scholar
• Dik, S. (1997). The theory of functional grammar. Berlin: Mouton De Gruyter.
   Google Scholar
• Flores d”Arcais, G. B. (1975). Some perceptual determinants of sentence construction. In G. B. Flores d’Arcais (Ed.), Studies in perception: Festschrift for fabio metelli (pp. 344–373). Milan: Martello-Giunti.
   Google Scholar
• Frederiksen, J. R. (1981). Understanding anaphora: Rules used by readers in assigning pronominal referents. Discourse Processes, 4(4), 323–347. doi: 10.1080/01638538109544525
   Google Scholar
• Frey, W. (2004). A medial topic position for German. Linguistische Berichte, 198, 153–196.
   Google Scholar
• Fukumura, K., van Gompel, R. P. G., & Pickering, M. J. (2010). The use of visual context during the production of referring expressions. Quarterly Journal of Experimental Psychology, 63(9), 1700–1715. doi: 10.1080/17470210903490969
   Web of Science ®Google Scholar
• Garnham, A. (2001). Mental models and the interpretation of anaphora. Hove: Psychology Press.
   Google Scholar
• Gernsbacher, M. A. (1991). Cognitive processes and mechanisms in language comprehension: The structure building framework. The Psychology of Learning and Motivation, 27, 217–263. doi: 10.1016/S0079-7421(08)60125-5
   Web of Science ®Google Scholar
• Givón, T. (Ed.). (1983). Topic continuity in discourse: A quantitative cross-language study. Amsterdam: John Benjamins Publishing Company.
   Google Scholar
• Gleitman, L. R., January, D., Nappa, R., & Trueswell, J. C. (2007). On the give and take between event apprehension and utterance formulation. Journal of Memory and Language, 57(4), 544–569. doi: 10.1016/j.jml.2007.01.007
   PubMed Web of Science ®Google Scholar
• Goodrich Smith, W., & Hudson Kam, C. L. (2015). Children’s use of gesture in ambiguous pronoun interpretation. Journal of Child Language, 42(3), 591–617. doi: 10.1017/S0305000915000045
   PubMed Web of Science ®Google Scholar
• Grewe, T., Bornkessel, I., Zysset, S., Wiese, R., Yves von Cramon, D., & Schlesewsky, M. (2006). Linguistic prominence and broca’s area: The influence of animacy as a linearization principle. NeuroImage, 32(3), 1395–1402. doi: 10.1016/j.neuroimage.2006.04.213
   PubMed Web of Science ®Google Scholar
• Grosz, B. J., Weinstein, S., & Joshi, A. K. (1995). Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2), 203–225.
   Web of Science ®Google Scholar
• Gundel, J. K. (1985). “Shared knowledge” and topicality. Journal of Pragmatics, 9, 83–107.
   Web of Science ®Google Scholar
• Gundel, J. K., Hedberg, N., & Zacharski, R. (1993). Cognitive status and the form of referring expressions in discourse. Language, 69(2), 274–307.
   Web of Science ®Google Scholar
• Halliday, M. A. K. (1967). Notes on transitivity and theme in English. Journal of Linguistics, 3(1), 177–274.
   Web of Science ®Google Scholar
• Hanna, J. E., & Brennan, S. E. (2007). Speakers’ eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57(4), 596–615. doi: 10.1016/j.jml.2007.01.008
   Web of Science ®Google Scholar
• Heister, J., Würzner, K.-M., Bubenzer, J., Pohl, E., Hanneforth, T., Geyken, A., & Kliegl, R. (2011). dlexDB – eine lexikalische Datenbank für die psychologische und linguistische Forschung. Psychologische Rundschau, 62(1), 10–20. doi: 10.1026/0033-3042/a000029
   Web of Science ®Google Scholar
• Hemforth, B. (1993). Kognitives parsing: Repräsentation und Verarbeitung sprachlichen Wissens (Dissertation). Universität Bochum.
   Google Scholar
• Holle, H., Obermeier, C., Schmidt-Kassow, M., Friederici, A. D., Ward, J., & Gunter, T. C. (2012). Gesture facilitates the syntactic analysis of speech. Frontiers in Psychology, 3, 74. doi: 10.3389/fpsyg.2012.00074
   PubMed Web of Science ®Google Scholar
• Jackendoff, R. (1990). Semantic structures. Cambridge, MA: MIT Press.
   Google Scholar
• Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59(4), 434–446.
   PubMed Web of Science ®Google Scholar
• Järvikivi, J., van Gompel, R. P. G., Hyönä, J., & Bertram, R. (2005). Ambiguous pronoun resolution: Contrasting the first-mention and subject-preference accounts. Psychological Science, 16(4), 260–264.
   PubMed Web of Science ®Google Scholar
• Johnson-Laird, P. N. (1980). Mental models in cognitive science. Cognitive Science, 4, 71–115. doi: 10.1207/s15516709cog0401_4
   Web of Science ®Google Scholar
• Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354.
   PubMed Web of Science ®Google Scholar
• Kaiser, E. (2006). Effects of topic and focus on salience. In C. Ebert, & C. Endriss (Eds.), ZAS working papers in linguistics. Proceedings of Sinn und Bedeutung 10 (pp. 139–154). Berlin: ZAS.
   Google Scholar
• Kaiser, E. (2011). Focusing on pronouns: Consequences of subjecthood, pronominalisation, and contrastive focus. Language and Cognitive Processes, 26(10), 1625–1666. doi: 10.1080/01690965.2010.523082
   Web of Science ®Google Scholar
• Kaiser, E., & Trueswell, J. C. (2004). The role of discourse context in the processing of a flexible word-order language. Cognition, 94(2), 113–147. doi: 10.1016/j.cognition.2004.01.002
   PubMed Web of Science ®Google Scholar
• Kamide, Y., Altmann, G. T., & Haywood, S. L. (2003). The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language, 49(1), 133–156. doi: 10.1016/S0749-596X(03)00023-8
   Web of Science ®Google Scholar
• Knoeferle, P., Crocker, M. W., Scheepers, C., & Pickering, M. J. (2005). The influence of the immediate visual context on incremental thematic role-assignment: Evidence from eye-movements in depicted events. Cognition, 95(1), 95–127. doi: 10.1016/j.cognition.2004.03.002
   PubMed Web of Science ®Google Scholar
• Knoeferle, P., Habets, B., Crocker, M. W., & Münte, T. F. (2007). Visual scenes trigger immediate syntactic reanalysis: Evidence from ERPs during situated spoken comprehension. Cerebral Cortex, 18(4), 789–795. doi: 10.1093/cercor/bhm121
   PubMed Web of Science ®Google Scholar
• Knoeferle, P., & Kreysa, H. (2012). Can speaker gaze modulate syntactic structuring and thematic role assignment during spoken sentence comprehension? Frontiers in Psychology, 3, 538. doi: 10.3389/fpsyg.2012.00538
   PubMed Web of Science ®Google Scholar
• Krifka, M. (2008). Basic notions of information structure. Acta Linguistica Hungarica, 55(3), 243–276. doi: 10.1556/ALing.55.2008.3-4.2
   Web of Science ®Google Scholar
• Lambrecht, K. (1994). Information structure and sentence form: Topic, focus and the mental representations of discourse referents. Cambridge: University Press.
   Google Scholar
• Lenth, R. V. (2016). Least-Squares means: The R package Lsmeans. Journal of Statistical Software, 69(1), 1–33. doi: 10.18637/jss.v069.i01
   Web of Science ®Google Scholar
• Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
   Google Scholar
• MacWhinney, B. (1977). Starting points. Language, 53, 152–168. doi: 10.1145/249094.249103
   Web of Science ®Google Scholar
• Matzke, M., Mai, H., Nager, W., Rüsseler, J., & Münte, T. F. (2002). The costs of freedom: An ERP – study of non-canonical sentences. Clinical Neurophysiology, 113, 844–852.
   PubMed Web of Science ®Google Scholar
• Mitchell, D. C. (1984). An evaluation of subject-paced reading tasks and other methods of investigating immediate processes in reading. In D. E. Kieras, & M. A. Just (Eds.), New methods in reading comprehension research (pp. 69–89). Hillsdale, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Morey, R. D. (2008). Confidence intervals from normalized data: A correction to Cousineau (2005). Tutorials in Quantitative Methods for Psychology, 4(2), 61–64.
   Google Scholar
• Myachykov, A., Thompson, D., Garrod, S., & Scheepers, C. (2012). Referential and visual cues to structural choice in visually situated sentence production. Frontiers in Psychology, 2, 396. doi: 10.3389/fpsyg.2011.00396
   PubMed Web of Science ®Google Scholar
• Nappa, R., & Arnold, J. E. (2014). The road to understanding is paved with the speaker’s intentions: Cues to the speaker’s attention and intentions affect pronoun comprehension. Cognitive Psychology, 70, 58–81. doi: 10.1016/j.cogpsych.2013.12.003
   PubMed Web of Science ®Google Scholar
• Nappa, R., Wessel, A., McEldoon, K. L., Gleitman, L. R., & Trueswell, J. C. (2009). Use of speaker’s gaze and syntax in verb learning. Language Learning and Development, 5(4), 203–234. doi: 10.1080/15475440903167528
   PubMedGoogle Scholar
• Osgood, C. E., & Bock, J. K. (1977). Salience and sentencing. Some production principles. In S. Rosenberg (Ed.), Sentence production: Developments in research and theory (pp. 89–140). Hillsdale, NJ: John Wiley & Sons.
   Google Scholar
• Portner, P. (2007). Instructions for interpretation as separate performatives. In K. Schwabe, & S. Winkler (Eds.), On information structure, meaning and form: Generalizations across languages (pp. 407–425). Amsterdam: John Benjamins Publishing Company.
   Google Scholar
• Prat-Sala, M. (2000). Discourse constraints on syntactic processing in language production: A cross-linguistic study in English and Spanish. Journal of Memory and Language, 42(2), 168–182. doi: 10.1006/jmla.1999.2668
   Web of Science ®Google Scholar
• R Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org
   Google Scholar
• Reinhart, T. (1981). Pragmatics and linguistics: An analysis of sentence topics. Philosophical Transactions of the Royal Society B: Biological Sciences, 1, 53–94.
   Google Scholar
• Rohde, H., & Kehler, A. (2014). Grammatical and information-structural influences on pronoun production. Language, Cognition and Neuroscience, 29(8), 912–927. doi: 10.1080/01690965.2013.854918
   Web of Science ®Google Scholar
• Rosengren, I. (1993). Wahlfreiheit mit Konsequenzen -Scrambling, Topikalisierung und FHG im Dienste der Informationsstrukturierung. In M. Reis (Ed.), Wortstellung und informationsstruktur (pp. 251–312). Tübingen: Niemeyer.
   Google Scholar
• Schumacher, P. B. (2014). Content and context in incremental processing: “the ham sandwich” revisited. Philosophical Studies, 168(1), 151–165. doi: 10.1007/s11098-013-0179-6
   Web of Science ®Google Scholar
• Sgall, P., Hajičová, E., & Panevová, J. (1986). The meaning of the sentence in its semantic and pragmatic aspects. Dordrecht: Reidel Publishing, Academia.
   Google Scholar
• Speyer, A. (2008). German vorfeld-filling as constraint interaction. In A. Benz, & P. Kühnlein (Eds.), Constraints in discourse (pp. 267–290). Amsterdam: John Benjamins Publishing Co.
   Google Scholar
• Sridhar, S. N. (1988). Cognition and sentence production: A cross-linguistic study. New York: Springer-Verlag.
   Google Scholar
• Staudte, M., & Crocker, M. W. (2011). Investigating joint attention mechanisms through spoken human–robot interaction. Cognition, 120(2), 268–291. doi: 10.1016/j.cognition.2011.05.005
   PubMed Web of Science ®Google Scholar
• Staudte, M., Crocker, M. W., Heloir, A., & Kipp, M. (2014). The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts. Cognition, 133(1), 317–328. doi: 10.1016/j.cognition.2014.06.003
   PubMed Web of Science ®Google Scholar
• Stevenson, R. J., Crawley, R. A., & Kleinman, D. (1994). Thematic roles, focus and the representation of events. Language and Cognitive Processes, 9(4), 519–548. doi: 10.1080/01690969408402130
   Web of Science ®Google Scholar
• Stevenson, R., Nelson, A., & Stenning, K. (1995). The role of parallelism in strategies of pronoun comprehension. Language and Speech, 38(4), 393–418. doi: 10.1177/002383099503800404
   Google Scholar
• Tannenbaum, P. H., & Williams, F. (1968). Generation of active and passive sentences as a function of subject and object focus. Journal of Verbal Learning and Verbal Behavior, 7, 246–250. doi: 10.1016/S0022-5371(68)80197-5
   Google Scholar
• Tomlin, R. S. (1997). Mapping conceptual representations into linguistic representations: The role of attention in grammar. In J. Nuyts, & E. Pederson (Eds.), Language, culture & cognition: Vol. 1. Language and conceptualisation (pp. 162–189). Cambridge: Cambridge University Press.
   Google Scholar
• van den Noort, M., Bosch, P., Haverkort, M., & Hugdahl, K. (2008). A standard computerized version of the reading span test in different languages. European Journal of Psychological Assessment, 24(1), 35–42. doi: 10.1027/1015-5759.24.1.35
   Web of Science ®Google Scholar
• Venables, W. N., & Ripley, B. D. (2002). Modern applied statistics with S–plus. New York: Springer.
   Google Scholar
• Vogels, J., Krahmer, E., & Maes, A. (2013). Who is where referred to how, and why? The influence of visual saliency on referent accessibility in spoken language production. Language and Cognitive Processes, 28(9), 1323–1349. doi: 10.1080/01690965.2012.682072
   Web of Science ®Google Scholar
• Weskott, T., Hoernig, R., Fanselow, G., & Kliegl, R. (2011). Contextual licensing of marked OVS word order in German. Linguistische Berichte, 225(16), 3–18.
   Google Scholar
• Zhang, L., & Knoeferle, P. (2012). Visual context effects on thematic role assignment in children versus adults: Evidence from Eye tracking in German. In N. Miyake (Ed.), Proceedings of the 34th annual conference of the cognitive science society (pp. 2593–2598). Japan: Cognitive Science Society.
   Google Scholar