Speakers prioritise affordance-based object semantics in scene descriptions

ABSTRACT

This work investigates the linearisation strategies used by speakers when describing real-world scenes to better understand production plans for multi-utterance sequences. In this study, 30 participants described real-world scenes aloud. To investigate which semantic features of scenes predict order of mention, we quantified three features (meaning, graspability, and interactability) using two techniques (whole-object ratings and feature map values). We found that object-level semantic features, namely those affordance-based, predicted order of mention in a scene description task. Our findings provide the first evidence for an object-related semantic feature that guides linguistic ordering decisions and offers theoretical support for the role of object semantics in scene viewing and description.

KEYWORDS:

• Language production
• linearisation
• discourse production
• scene semantics
• eye movements

Acknowledgements

We thank our research assistants, Lauren Alimento, Casey Felton, and Susannah Rudd, for their excellent work.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

All study materials including data, stimuli, and analysis code are available at: https://osf.io/ey3wd/. This study was not preregistered.

Notes

1 Refer to Henderson et al. (2018) for analyses pertaining to visual attention.

Additional information

Funding

This work was funded by the National Institute of Health [grant number R01 HD100516]; and by the National Science Foundation [grant number BCS1650888] awarded to Fernanda Ferreira.