Semantic encoding in working memory: Is there a (multi)modality effect?

Abstract

In spite of a large body of empirical research demonstrating the importance of multisensory integration in cognition, there is still little research about multimodal encoding and maintenance effects in working memory. In this study we investigated multimodal encoding in working memory by means of an immediate serial recall task with different modality and format conditions. In a first non-verbal condition participants were presented with sequences of non-verbal inputs representing familiar (concrete) objects, either in visual, auditory or audio-visual formats. In a second verbal condition participants were presented with written, spoken, or bimodally presented words denoting the same objects represented by pictures or sounds in the non-verbal condition. The effects of articulatory suppression were assessed in both conditions. We found a bimodal superiority effect on memory span with non-verbal material, and a larger span with auditory (or bimodal) versus visual presentation with verbal material, with a significant effect of articulatory suppression in the two conditions.

Keywords:

• Working memory
• Encoding
• Multimodal
• Episodic Buffer
• Semantics
• Span

Acknowledgements

We are grateful to Perla Amodio and Chiara Bozzacchi for their help in the data collection, and to an anonymous reviewer for his/her constructive comments and helpful suggestions.