Modality-specific effects in bilingual language perception

ABSTRACT

Language switching typically leads to language switch costs. Previous studies demonstrated that production-based language switching often results in reduced costs when the languages are assigned to distinct modalities (i.e. bimodal language switching) compared to switching within a modality (i.e. unimodal switching). In Experiment 1–3, we compared unimodal to bimodal switching in language perception. In Experiment 4, unimodal switching was compared to bivocal switching (i.e. languages were assigned to different voices). Experiment 1–3 revealed significantly higher switch costs for bimodal than for unimodal switching, indicating fundamental differences to language production studies. While bimodal switching leads to no (additional) costs but – under specific conditions – even to advantage in production, it increases switch costs in language perception. Moreover, Experiment 4 indicates that this is more a modality-specific rather than a general perception-based effect. Different voices instead of different modalities evoked no similar increase in switch costs.

KEYWORDS:

• Language switching
• sensory modality switching
• language control

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Participants learned the signs at the beginning of the experiment in a similar way as in the production-study by Schaeffner and colleagues (2017).

2 At this point it is important to note that the output modality was constant throughout the experiment (i.e., Manual responses in the unimodal as well as in the bimodal condition). That is - in contrast to previous production-based language-switching studies (e.g., Kaufmann et al., 2018; Schaeffner et al., 2017) - “unimodal” versus “bimodal” refers only to the input modality and not to the output modality.

3 The intention behind including the single blocks was to offer participants the possibility to practice the different stimulus-response mappings of each language separately before switching between the languages and, moreover, to calculate mixing costs in addition to switch costs. While switch costs are seen as an informative value regarding the local costs that occur while switching from one task to another, mixing costs are seen as an informative value about the more global costs representing the cognitive load that is required when different task sets or stimulus-response mappings have to be maintained in parallel (see e.g. Kiesel et al., 2010).

4 In addition, we calculated mixing costs (i.e., Performance differences between single blocks and repetition trials of the experimental blocks of the two switching conditions) for German. That is, we conducted two individual paired t-tests in which we compared the performance in the single blocks German (which were actually discarded for the switch costs analysis) once to the performance in German repetition trials in the unimodal switching condition and once to the performance in German repetition trials in the bimodal switching condition. Even though the RT in the single blocks (707 ms) were shorter than the RT in repetition trials of the unimodal switching condition (724 ms) as well as in the bimodal switching condition (724 ms), performance differences were neither significant for unimodal switching (t(23) = 1.9; p = .075) nor for bimodal switching (t(23) = 1.8; p = .081). That is, there were neither significant RT mixing costs for the unimodal switching condition nor for the bimodal switching condition. Furthermore, there was a speed-accuracy tradeoff with regard to mixing costs. That is, the error data revealed a significant mixing advantage (i.e., Higher error rates in the single blocks German than in repetition trials of the switching condition) for unimodal switching (t(23) = 2.1; p = .046) as well as for bimodal switching (t(23) = 3.2; p = .004). Thus, we think that the informative value of the analysis of mixing costs is severely limited and allows no reasonable interpretation.

5 In line with Experiment 1, we additionally calculated mixing costs for German for unimodal and bimodal switching in Experiment 2. However, the analysis of RT data revealed neither significant mixing costs for unimodal nor for bimodal switching (ts < 1). Similarly, the analysis of the error rates revealed neither significant mixing costs for unimodal switching (t(23) = 1.4; p = .191) nor for bimodal switching (t(23) = 2.0; p = .06).

6 In line with Experiment 1 and 2, we additionally calculated mixing costs for German for unimodal and bimodal switching in Experiment 3. The analysis of RT data revealed significant mixing costs for unimodal (t(23) = 2.6; p = .02) but not for bimodal switching (t < 1). The analysis of the error rates revealed neither significant mixing costs for unimodal (t(23) = 1.1; p = .297) nor for bimodal switching (t < 1).

7 In line with Experiment 1-3, we additionally calculated mixing costs for German for both conditions. However, neither the analysis of RT data nor the analysis of the error rates revealed significant mixing costs in any condition (ts < 1).

8 Switch costs were examined by post-hoc paired t-tests comparing language switch and language repetition trials for each language in each experiment. Experiment 1: t(23) = 3.4; p = .003 for German and t < 1 for English. Experiment 2: t(23) = 1; p = .325 for German and t(23) = 2.2; p = .039 for English. Experiment 3: ts < 1 for German and for English. Experiment 4: t(23) = 2.3; p = .029 for German and t < 1 for English.

9 A similar calculation for the bimodal condition would be not informative regarding language switch costs asymmetry due to the presentation of the languages in different modalities.

Additional information

Funding

This research was funded by DFG grant PH 156/3-2.