Investigation of Multisensory Harmonic Priming: Audiovisual Integration in Chord Perception

ABSTRACT

Although typically thought of as a unimodal phenomenon, musical experience is fundamentally multisensory. The current study examined such multisensory influences by exploring the impact of visual information on the processing of higher-order musical structure, within the context of harmonic expectancy formation. Specifically, musically trained participants experienced four different priming events, including an auditory chord prime, a visual chord prime, an auditory-visual note/chord prime, and an auditory note prime, and discriminated harmonically related versus unrelated in-tune and out-of-tune target chords. Analyses of reactions and error rates revealed typical harmonic priming effects (i.e., faster reaction times and lower error rates to related than unrelated targets for in-tune chords) for auditory chord and auditory-visual note/chord primes, but no differences for visual chord and auditory note primes. The finding that auditory-visual note/chord primes led to priming indicates that visual information can induce auditory expectancies when supported by auditory cue information. These results are discussed within the context of models of tonal relations, as well as in relation to theories of multisensory processing in general.

KEYWORDS:

• Harmonic priming
• multisensory processing
• musical expectancy

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Supplemental data

Supplemental data for this article can be accessed here.

Notes

1. For interested readers, Block Order produced only a handful of interaction effects. These effects included a significant Prime Type x Block Order interaction, F(9, 81) = 2.23, MSE = 122,122.79, p = .028, η_p² = .198, with these data showing that participants generally increased in RT over the course of the experiment, with the slowest RTs occurring in the first block for all four orders, and the fastest RTs occurring in the final block for three of the four orders. Because the different conditions were counterbalanced in a Latin Square, this practice effect produces a two-way interaction between these factors. There was also a marginally significant Target Tuning x Block Order interaction, F(3, 27) = 2.55, MSE = 118,524.87, p = .077, η_p² = .221, with in-tune targets responded to more slowly than out-of-tune targets as the experiment progressed. And finally, the four-way interaction between all factors was significant, F(9, 81) = 2.02, MSE = 20,976.28, p = .047, η_p² = .183. Truthfully, this interaction is hard to interpret, but it appears that differences between in-tune related and unrelated targets varied as a function of the Prime Type condition. Again, because Prime Type was counterbalanced using a Latin Square, such differences produce an interaction with the order variable.

Additional information

Funding

This research was funded by a Natural Sciences and Engineering Research Council of Canada grant to M. A. Schmuckler.