Electrophysiological correlates of the action vs. role relations congruencies in visually situated auditory sentence processing in Korean

ABSTRACT

This study investigated brain responses to verbal action mismatches (e.g. The brother catches/*bites the sister) and role relation mismatches (e.g. The sister catches the brother) in a picture-sentence verification task in Korean using event–related potentials (ERPs). EEG data collected from 30 native Korean speakers revealed a large N400 at the verb for the action mismatch, but an early negativity at the mismatched case-marker of the first NP as well as sustained negativity at the verb for the role mismatch. These results conform to the English-based data (i.e. Knoeferle et al., 2014. Different mechanisms for role relations versus verb–action congruence effects: Evidence from ERPs in picture–sentence verification. Acta Psychologica, 152, 133–148. https://doi.org/10.1016/j.actpsy.2014.08.004) in that the meaning of verb action and the thematic role relations involve different mechanisms in language comprehension. However, cross-lingustic differences were also found in terms of comprehenders’ relative sensitivity to different linguistic features in individual languages (e.g. word order in English and case-marking in Korean). Implications of the findings are discussed regrading predictive processing in language comprehension.

KEYWORDS:

• Event-related potential
• Korean case markers
• picture–sentence verification task
• predictive sentence processing
• action mismatch
• role relation mismatch
• N400

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Acknowledgements

This work has supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2016S1A5A2A01027338) for Sun-Young Lee. Also, this work was supported by the KU Research Professor Program of Konkuk University for Yunju Nam.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 The Bayesian approach assumes that our brains combine prior knowledge and contextual information to calculate the posterior probability of the presence of an object (or a word) being presented in the forthcoming input (Friston, 2005; Knill & Pouget, 2004).

2 Korean nominative and accusative case markers have phonological allomorphs; when the final syllable terminates with a consonant, it is joined to the next syllable, the case marker, and they are pronounced together. Thus, these situations show low auditory discrimination and this experiment was designed to avoid such situations.

3 LME is used in the analysis of EEG data was advocated for by Bagiella, Sloan, and Heitjan (2000) and has been used by Davidson and Indefrey (2007); Moratti et al. (2007); and Wierda et al. (2010).

4 If the absolute t-value of a fixed factor was over 2, the effect of the factor was considered to be significant at α < .05 (Gelman & Hill, 2007).

5 glmer(correct ∼ type + (1 + type | subject) + (1 | item))

6 lmer(rt_correct ∼ type + (1 + type | subject) + (1 | item))

7 Mean amplitude at 700–900 ∼ Condition * ROI + (1 + Condition | subject)

8 Mean amplitude at 700–900 ∼ Condition * Laterality * Posteriority + (1 + Condition + Laterality + Posteriority | subject)

9 Mean amplitude at 0–300 ∼ Condition * ROI + (1 + Condition | subject)

10 Mean amplitude at 0–300 ∼ Condition * Laterality * Posteriority + (1 + Condition + Laterality + Posteriority | subject)

11 Midline: Mean amplitude_300–500 ∼ Condition * ROI + (1 + Condition | subject))

Lateral: Mean amplitude_300–500 ∼ Condition * Laterality * Posteriority + (1 + Condition | subject)

12 Midline: Mean amplitude_ 500–700 ∼ Condition * ROI + (1 + Condition + ROI | subject)

Lateral: Mean amplitude_ 500–700 ∼ Condition * Laterality * Posteriority + (1 + Condition + Laterality +

Posteriority | subject))

13 One might think that the results do not seem to conform to the Constituent Comparison Model (Carpenter & Just, 1975) which has been supported by many previous behavioural studies using similar an on-line picture-sentence verification task (Chase and Clark, 1972; Gough, 1965; Just & Carpenter, 1971; Trabasso et al., 1971) as mentioned at the beginning. In those studies, the sentences that matched the picture were processed faster than mismatched ones. However, it should be reminded that the behavioural results from this study are obtained from the ERP experiment the design of which is usually different from that of a behavioural experiment. In particular, in this study, there was a 1000 ms pause after the end of the sentence until they responded in order to avoid possible noise from their movement in the recording of their brain wave, which might have kept us from catching subtle behavioural differences if there was any. In contrast, Lee et al. (2017) which used the same material as a behavioural test without any pause after the end of the sentence found that matched sentences were judged faster than mismatched sentences. In addition, the role relation mismatch was judged faster but less accurately than the action mismatch which seemed to indicate different processing mechanisms between the two types of anomalies (see Lee et al. 2017 for details).

14 Lee et al.’s (2014) finding of P600 at the sentential end verb in the case-marking mismatch condition (“brother-ka/*-lul catch”) was not born out in this study with double the number of participants. We rely on the data from this study for our discussion due to larger data than the previous one. In addition, P600 effect along with negativity at the verb which appeared at the first data set with 15 participants for this study also disappeared when we doubled the number of particpants accepting a reviewer’s comments that suggested the possibility of P600 as task-specific artifects. Still, we admit that further replication studies are needed for confirmation of the findings.

Additional information

Funding

This work has supported by the National Research Foundation of Korea (NRF) grant (No. NRF-2016S1A5A2A01027338) for Sun-Young Lee. Also, this work was supported by the KU Research Professor Program of Konkuk University forYunju Nam.