The Subtitle Effect: How Subtitle-Facilitated Perceptual Learning Can Make Subtitles (Seem) Less Necessary

Abstract

In television and film production, speakers whose accents are judged difficult to understand are often subtitled, but viewers may question such judgments if they later perceive the subtitles as unnecessary. Such doubts can be ill founded, however. We report two experiments showing that the presence of bimodal subtitles (subtitles in the same language as the audio) on a given video clip can make subtitles seem less necessary for that clip and can improve comprehension of a second, unsubtitled clip of the same speaker. This, we argue, is because bimodal subtitles provide alternate access to the lexical information in the audio, thus facilitating perceptual learning of the subtitled speaker’s accent. This perceptual learning, through which listeners adjust their sound–phoneme mappings to accommodate the unfamiliar phoneme realizations underlying the speaker’s accent, can quickly make subtitles less necessary, potentially creating the illusion that they were never necessary—a phenomenon we call the “subtitle effect.”

Keywords:

• Accent Learning
• Perceptual Learning
• Speaker–Listener Adaptation
• Speech Perception
• Subtitles