Prediction and integration of semantics during L2 and L1 listening

ABSTRACT

Using the visual world paradigm, we tested whether Dutch-English bilinguals predict upcoming semantic information in auditory sentence comprehension to the same extent in their native (L1) and second language (L2). Participants listened to sentences in L1 and L2 while their eye-movements were measured. A display containing a picture of either a target word or a semantic competitor, and three unrelated objects was shown before the onset of the auditory target word in the sentence. There were more fixations on the target and competitor pictures relative to the unrelated pictures in both languages, before hearing the target word could affect fixations. Also, semantically stronger related competitors attracted more fixations. This relatedness effect was stronger, and it started earlier in the L1 than in the L2. These results suggest that bilinguals predict semantics in the L2, but the spread of semantic activation during prediction is slower and weaker than in the L1.

KEYWORDS:

• Visual world paradigm
• prediction
• comprehension
• bilingualism
• semantic competitor

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Aster Dijkgraaf http://orcid.org/0000-0002-2420-4150

Notes

1 Out of the 871 sentences, 54 were from the Block and Baldwin (2010) sentence set, and 31 from Hamberger, Friedman, and Rosen (1996). Another 39 were adapted from Block and Boldwin, and 31 were adapted from Hamberger, Friedman and Rosen. These sentences were adapted so that they could be translated to Dutch without changing the sentence final word.

2 0 = no overlap, 1 = identical (Schepens, Dijkstra, Grootjen, & van Heuven, 2013).

3 We applied the optimal models to the prediction time frame data excluding trials in which the experimental image was a cognate (phonological levenshtein distance >.5; following Schepens et al., 2013). For the target, the language by image type interaction remained significant (β = 35, SE = .08, t = 4.19, p < .001). For the competitor data, the three-way interaction between language, image type and semantic distance also remained significant (β = −.21, SE = .08, t = −2.54, p = .01).

4 The target/competitor words sometimes had false friends in the other language (e.g. map, meaning folder in Dutch). We applied the optimal models to the prediction time frame data excluding trials in which the experimental image (target or competitor) had (identical) false friends in the other language. Both words with identical orthographic false friends (85 out of 724 words) and words with identical phonological false friends (25 out of 724 words) were excluded (106 in total). For the target, the language by image type interaction remained significant (β = .24, SE = .09, t = 2.77, p = .006). As for the competitor, competitor semantic distance still interacted with image type (β = .28, SE = .08, t = 3.49, p <.001), but the three-way interaction with language was no longer significant (β = −.13, SE = .09, t = −1.54, p = .12). To investigate whether the three-way interaction disappeared because of loss of power or because false friend status actually affected looking behaviour we compared the final model with the final model plus the factor false friend status (false friend in the other language yes or no) and the interaction between false friend status and image type. False friend status did not contribute to the model fit ((2) = 1.73, p = .42).

5 Competitors were sometimes ungrammatical as sentence ending (e.g. because of a gender mismatch with the preceding determiner) and/or they could violate a phonotactic rule (due to a mismatch with preceding indefinite article a or an). To test whether competitor grammaticality affected our results we applied the optimal models to the prediction frame data excluding trials in which the competitor was ungrammatical or violated a phonotactic rule. Fifty (out of 362) English sentences and 43 (out of 362) Dutch sentences were excluded. For the target, the language by image type interaction remained significant (β = .25, SE = .09, t = 2.89, p = .004). For the competitor data, the two-way language by image type interaction remained significant (β = .22, SE = .08, t = 2.68, p = .007), as did the interaction between image type and semantic distance (β = .27, SE = .08, t = 3.45, p < .001). The three-way interaction between language, image type and semantic distance approached significance (β = −.15, SE = .08, t = −1.87, p = .06). In addition, adding competitor grammaticality and the interaction between grammaticality and image type to the optimal model for the prediction time frame (competitor data set) did not improve the model fit (χ(2) = 1.63, p = .44).

6 The English corpora used were UKWAC (Ferraresi, Zanchetta, Baroni, & Bernardini, 2008) (containing texts from the .uk internet domain) and a subtitle corpus (Mandera et al., 2017) (downloaded from http://opensubtitles.org). For Dutch Sonar-500 text corpus (Oostdijk, Reynaert, Hoste, & van den Heuvel, 2013) (texts from conventional and new media) and another subtitle corpus (Mandera et al., 2017) were used.

7 In 8 sentences (out of 362 Dutch and 362 English sentences) either the target word or the competitor word was present in the sentence, either with the same meaning or a slightly different meaning (e.g. She locked her bicycle to a fence with a lock, Ivory is derived from an elephant or a rhino-> competitor: elephant). A picture of the target or competitor word also present in the sentence was likely to attract more fixations in these sentences than in other sentences. The random slope for item in the analyses ensured that this possible confound did not affect the results. In addition, an analysis of the target and competitor data of the full prediction time frame without these 7 sentences did not change the results.

8 Due to an error in the test plausibility ratings for three (out of 724 sentences) were missing.

9 Due to technical problems the scores for fluency (Dutch and English) is missing for two participants, and the score for digit span is missing for one participant.

10 It is possible that any of the factors excluded from the final models had a significant effect in some of the time bins. We used the final model for the time bin analyses in order to investigate whether different languages showed a different time course of effects of the relevant variables, as observed in the full prediction time frame analysis. The alternative of running a separate backfitting procedure for each time bin could not fulfill this goal, as this would lead to models with different factors in each bin, so that the results for each time bin would not have been directly comparable.

Additional information

Funding

This research was supported by a Concerted Research Action (GOA grant number BOF13/GOA/032) from the Special Research Fund (Bijzonder Onderzoeksfonds), Ghent University.