Interaction between topic marking and subject preference strategy in sign language processing

ABSTRACT

The preference of the human parser for interpreting syntactically ambiguous sentence-initial arguments as the subject of a clause (i.e. subject preference) has been documented for spoken and sign languages. Recent research (He, 2016) suggests that the subject preference can be eliminated by manipulating information structure (topicalisation). To investigate the effects of interaction between syntax and information structure on language processing, we tested the role of topic marking in sentence processing in Austrian Sign Language (ÖGS). We examined whether non-manual topic marking on the sentence-initial argument eliminates the subject preference using event-related brain potentials. We replicated the finding of the subject preference in ÖGS by identifying an N400-family response to object-first sentences. Further, topic marking in ÖGS influenced the processing of the topic argument itself and later processing stages. This suggests that interpretation of topic marking imposes additional processing costs, relative to syntactic reanalysis.

KEYWORDS:

• Non-manual markings
• subject preference
• Austrian Sign Language
• sign language processing
• topic marking

Introduction

Sign languages are expressed within the three-dimensional signing space by manual (hands and arms) as well as non-manual (face, head, upper body) articulators. Neurolinguistic processing of sign languages provides insight into universal laws in psycholinguistics, i.e. revealing how language is processed independent of language modality. One of the universal psycholinguistic phenomena that has been ascertained across multiple types of languages is the subject preference: interpretation of a sentence-initial argument NP as the subject of a clause (Bornkessel, McElree, Schlesewsky, & Friederici, 2004; Frazier & d’Arcais, 1989; Haupt, Schlesewsky, Roehm, Friederici, & Bornkessel-Schlesewsky, 2008; Schlesewsky, Fanselow, Kliegl, & Krems, 2000; Schriefers, Friederici, & Kuhn, 1995). In previous work we ascertained the subject preference phenomenon in Austrian Sign Language (ÖGS; Krebs, 2017; Krebs, Malaia, Wilbur, & Roehm, 2018; Krebs, Wilbur, Alday, & Roehm, 2018). However, a recent study on spoken Mandarin showed that the subject preference can be eliminated by topicalisation (He, 2016), suggesting that online processing of syntax is modified by requirements for information structure processing. We investigated whether this mechanism applies in the case of ÖGS using non-manual topic marking phenomenon as a testing ground. In particular, we investigated whether non-manual topic marking on the sentence-initial argument may influence the processing of otherwise locally ambiguous argument structures in Austrian Sign Language.

Subject preference effects in online processing

Incremental language processing, i.e. the immediate integration of linguistic input material within the previously established context, often causes local ambiguities which the processing system has to deal with. “Subject preference” is a strategy which has been observed for processing locally ambiguous argument structures. It is the preference for interpreting a sentence-initial argument that is ambiguous with respect to its syntactic function as the subject of the clause (Bornkessel et al., 2004; Frazier & d’Arcais, 1989; Haupt et al., 2008; Schlesewsky et al., 2000; Schriefers et al., 1995). Preference for such subject-initial (SOV) structures eventually requires reanalysis in locally ambiguous object-initial (OSV) orders, resulting in enhanced processing costs reflected in longer reading times (Schlesewsky et al., 2000), lower acceptability ratings and longer reaction times (Bornkessel et al., 2004; Haupt et al., 2008), more regressions and longer fixations during reading (Kretzschmar, Bornkessel-Schlesewsky, Staub, Roehm, & Schlesewsky, 2012) as well as different ERP effects for OSV sentences compared to their SOV counterparts (see e.g. Bornkessel-Schlesewsky & Schlesewsky, 2009a for an overview).

Although the subject preference has been attested in a number of different languages, it seems to be restricted to languages with basic word order in which the subject precedes the object. For instance, for a language with basic VOS order (Kaqchikel, a Mayan language spoken in Guatemala) the opposite effect has been shown: increased processing costs for sentences in which the subject precedes the object in contrast to sentences with object preceding the subject (Yasunaga, Yano, Yasugi, & Koizumi, 2015).

Whereas traditional accounts suggest that the preference for subject-first structures is related to specific properties of syntactic subjecthood such as dependencies (e.g. Gibson, 1998) or syntactic position (e.g. Crocker, 1994), it has been proposed more recently that this phenomenon reflects a universal ambiguity resolution processing strategy which is applied whenever a sentence-initial ambiguous argument is encountered (Bornkessel-Schlesewsky, Choudhary, Witzlack-Makarevich, & Bickel, 2008). This explanation is supported by the fact that the subject preference has been observed in typologically very different languages (Mandarin, Turkish or Hindi), which partially lack some syntactic subjecthood characteristics and subject-object asymmetries (for Mandarin, Li & Thompson, 1976; Wang, Schlesewsky, Bickel, & Bornkessel-Schlesewsky, 2009) (Bornkessel-Schlesewsky et al., 2008). Therefore, the subject preference has also been described in terms of actor preference, whereby the actor is understood as the participant primarily responsible for the state of affairs that is described by the event (cf. proto-Agens in Primus, 1999) (e.g. Alday, Schlesewsky, & Bornkessel-Schlesewsky, 2014; Bornkessel & Schlesewsky, 2006; Bornkessel-Schlesewsky & Schlesewsky, 2009b, 2013).

Interestingly, a recent ERP study by He (2016) on spoken Mandarin revealed that the subject preference can be eliminated by topicalisation of the object argument to sentence-initial position, which he considers to be a “clause external” factor (He, 2016). This finding contrasts with a previous study on Mandarin which did show a subject preference, i.e. an N400 effect, for object-initial compared to subject-initial orders in the context of topic marking (Wang, 2011). Wang et al. (2009) also observed an N400 effect for non-topic object-initial compared to non-topic subject-initial orders in Mandarin (Wang, 2011; Wang et al., 2009). As discussed by He (2016) the difference between these findings may stem from the difference in the tested stimulus material: Wang (2011) investigated the processing of discourse topics, whereby arguments were marked as topics by the preceding sentence context (e.g. “What is about the novel? The novel … ”). Thus, technically no syntactic movement operation was involved in creating these constructions. In contrast, He (2016) used isolated relative clause constructions in which sentence-initial ambiguous arguments were clearly marked as topic by a preceding “That + Classifier” phrase and a following topic marker “Ah” as well as a prosodic phrase boundary.¹ He (2016) tested sentences of the structure “That-CL – NP1 – Ah, V – relativiser-DE – NP2” which were disambiguated at the verb following the sentence-initial topic argument (e.g. That-CL Student Ah, examine DE teacher; translated in English: “The teacher that examines the student … ”).

With this data, one might conclude that it was the syntactic movement operation (present in He [2016] stimuli, but not in Wang [2011] stimuli) that cancelled out the subject preference strategy rather than the topic status of the sentence-initial argument. However, He (2016) observed a subject preference for wh-clauses, which are syntactically comparable to structures with topicalisation (i.e. both are assumed to involve a syntactic movement operation). To explain this, He (2016) suggested that syntactically-motivated processing mechanisms alone cannot fully derive the processing patterns observed for both constructions. In particular, whereas the findings for the wh-clauses can be interpreted in terms of structurally-based processing mechanisms that propose that the parser seeks to keep the structural distance between an extracted constituent (filler) and its extraction site (gap) minimal (e.g. Minimal Chain Principle or Active Filler Strategy; e.g. Clifton & Frazier, 1989; De Vincenzi, 1991; Frazier, 1987), this explanation does not seem to hold for topic constructions. Rather, he assumes that in Mandarin, which has been claimed to be a topic prominent language (e.g. Li & Thompson, 1976; but see Huang, Li, & Li, 2009), topicalisation may override local syntactic constraints and thus may directly influence sentence-level processing leading to suppression of the subject preference effect in the topicalised sentences, but not in the wh-clauses. Further, He (2016) provides an alternative explanation for the observation of a subject preference for wh-clauses but not for topicalisation, pointing to the fact that there was a phrase boundary in the sentences with topicalisation. This phrase boundary isolates the topicalised argument from the following clause and therefore may be processed differently.

Subject preference in Austrian Sign Language (ÖGS) revealed by ambiguity of argument relations

In sign languages, which are commonly assumed to lack case-marking on NPs, argument relations can be indicated by word order, semantic restrictions, or verb modification in the three-dimensional signing space in front of the signer for agreement.² Discourse referents are located in signing space by manual (index/pointing signs) and/or non-manual cues (body shift/eye gaze towards a specific location). These spatial reference points may be associated with physically present or non-present referents. Further, by pointing back to previously established locations, the signer can re-reference a discourse participant. Then agreement marking on verbs shares the space established for each referent (Lourenço & Wilbur, 2018); the net result is a path movement from the position associated with the subject to that of the object position. Marking of subject agreement is often reduced or omitted. Some verbs may show a specific hand orientation, so-called facing, whereby the palm faces the object. This facing may appear with or without movement towards the object.³ Note that not all verbs in sign languages use spatial agreement (referred to as “plain” verbs). Because in the present experiment only classical agreeing verbs are used, we will not discuss this other group of verbs.

The basic sign order in ÖGS is SOV. However, with marked agreement (on the verb or additional agreement marker, i.e. manual signs that can be inflected in space like regular verbs), OSV orders are possible without any specific prosodic marking (for more information about the two agreement markers in ÖGS glossed as AgrM-BC and AgrM-MF, see Krebs, Wilbur, & Roehm, 2017). In previous work (Krebs, 2017; Krebs, Malaia, et al., 2018; Krebs, Wilbur, et al., 2018) we tested the processing of locally ambiguous SOV compared to OSV orders, using stimuli which were constructed by keeping the spatial referencing constant among conditions (i.e. the first argument was always referenced at the left side of the signer).⁴ We observed the subject preference reflected by ERP reanalysis effects for OSV compared to SOV orders. Interestingly, these effects were revealed relatively early, that is, before the time point usually assumed to be the cue indicating the argument structure, namely, before the movement and/or the facing of the disambiguating sign is visible. We suggested that transitional movement towards the disambiguating sign and/or non-manual markings (NMMs) preceding/accompanying this transition may have resolved ambiguity. In particular, in most of the items the signer's body was shifted towards the subject position and the chin/face was directed towards the object position. Hence, these NMMs may have indicated where the movement of the disambiguating sign would start (Krebs, Malaia, et al., 2018).

This relatively early disambiguation time point was further confirmed by a gating study in which locally ambiguous SOV and OSV orders were presented in successively prolonged gates to Deaf signers who had to decide after each gate whether they thought that the argument introduced first or second is the active referent.⁵ The first gate constituted the time interval from video onset to onset of the second argument. Each subsequent gate was prolonged by four frames. In line with the ERP results, all items were disambiguated before the movement/facing of the disambiguating sign was visible; in most items the gate at which the OSV order falls below a 50% chance of being interpreted as SOV order was observed before the transition movement was visible (Krebs, 2017; Krebs, Wilbur, et al., 2018). These findings suggest that NMMs are relevant for processing locally ambiguous argument structures in ÖGS.

In the present study we examine how non-manual topic marking that is used for emphasising an argument may influence the subject preference in ÖGS (as observed for Mandarin by He, 2016).

Non-manual markers in sign languages and their processing

Non-manual cues (NMMs) serve important linguistic functions on all levels of sign language grammar. Sign languages differ in which NMMs perform a particular function (Pfau & Quer, 2010 for an overview). For instance, NMMs can be part of a specific lexical item, can add adverbial/adjectival information (Wilbur, 2000), can mark negation, can distinguish sentence types (questions, conditionals, topic constructions, imperatives) (Liddell, 1980), and can show agreement (for American Sign Language (ASL), subject agreement with head tilt, object agreement with eye gaze; Bahan, 1996; Neidle, Kegl, MacLaughlin, Bahan, & Lee, 2000). Furthermore, they function as phrasal edge and domain markers (Wilbur, 1994a), contribute important discourse information (e.g. role shift; Lillo-Martin, 1995; Quer, 2005), perform prosodic functions (stress marking; Wilbur & Patschke, 1998), and may avoid ambiguity on different grammatical levels (Quer & Steinbach, 2015 for an overview).

NMMs are produced by various facial articulators (e.g. chin/brows/cheeks/nose/tongue, eye gaze, mouth), as well as head and upper body positions/movements. Besides the place of articulation, the number of productions (single, repeated), their scope (lexical item, boundary edge, phrasal/clausal domain), and the onset and offset (abrupt, gradual) result in diverging functions (Wilbur, 2000). Interestingly, a specific non-manual cue may serve different functions, depending on what else it occurs with and/or the context of use. For example, brow raise can mark polar questions as well as topics in American Sign Language (ASL) (Liddell, 1980) or body lean (forward/backward) can convey the notion of contrast at the prosodic, lexical-semantic, syntactic and pragmatic level in ASL (Wilbur & Patschke, 1998).

Signers use the same non-manual articulators not only for linguistic devices, but also (like hearing non-signers) for producing affective facial expressions. Linguistic and non-linguistic facial expressions differ in their scope, in that linguistic markers show an abrupt onset and offset and are coordinated with the grammatical constituents they modify. In contrast, the onset/offset of affective facial markers are gradual and are not required to be coordinated with specific constituents (Liddell, 1978, 1980). For example, the negative headshake, which has an abrupt onset and offset and parallels the scope of the negative constituent, functions as a grammatical marker in ASL. In contrast, the negative headshake used by hearing English-speaking non-signers has a gradual onset and offset, and occurs in sentence positions which are not directly connected to the syntactic structure of English negation (Veinberg & Wilbur, 1990).

That the information provided by the face is crucial for sign language processing is shown by the observation that perceiving signers look into the face of the producing signer during sign language perception (Emmorey, Thompson, & Colvin, 2008; Siple, 1978). Studies suggest that signers’ unique experience with the human face due to sign language exposure leads to enhancements in face processing abilities. For example, ASL signers (hearing and Deaf) show significantly better performance than non-signers when discriminating facial expressions (Bettger, Emmorey, McCullough, & Bellugi, 1997), when discriminating local facial features (McCullough & Emmorey, 1997), or when memorising faces (Arnold & Murray, 1998).⁶

The linguistic status of grammatical facial expressions has also been supported by the observation that these are mainly processed by the left hemisphere within signers, whereas affective facial expressions are mainly processed by the right hemisphere. For example, Kegl and Poizner (1997) showed that an ASL signer with a lesion in the left hemisphere had difficulties producing linguistic NMMs, but the use of affective facial expressions was unimpaired. Persons with a lesion in the right hemisphere show the reverse pattern, i.e. their grammatical NMMs are intact, but they are not able to interpret affective NMMs adequately (Corina, Bellugi, & Reilly, 1999). Interestingly, some linguistic NMMs (those on the lexical and pragmatic level and those for non-manual negation) are maintained even with a lesion in the right hemisphere (Atkinson, Campbell, Marshall, Thacker, & Woll, 2004; Kegl & Poizner, 1997). In addition, acquisition studies showed that linguistic facial expressions are acquired later by Deaf children than affective facial expressions (Anderson & Reilly, 1998; McIntire & Reilly, 1988; Reilly, McIntire, & Bellugi, 1990).

Studies with neurally healthy signers confirmed the difference in neural representation of linguistic and affective facial expressions within signers, and in addition, showed a difference in processing of (linguistic/affective) facial expressions between signers and hearing non-signers (who do not show a different processing pattern for linguistic compared to affective facial expressions). For instance, in an fMRI study contrasting Deaf ASL signers with hearing non-signers, McCullough, Emmorey, and Sereno (2005) observed bilateral activation within the superior temporal sulcus (STS) in Deaf signers and right hemisphere dominance for hearing non-signers for the processing of affective facial expressions. For linguistic facial expressions they observed activation in left STS only for signers and only when the NMMs accompanied manually produced verb signs. Further, for both kinds of facial expressions, they observed activation in the left fusiform gyrus for Deaf signers, and bilateral activation within this area for hearing non-signers.

Most studies investigating the neural representation of NMMs by signers focus on the processing of facial expressions or information presented by the mouth (e.g. Capek et al., 2008). Pendzich, Steinbach, and Hermann (2016) conducted a study showing that lexical NMMs that are not expressed by the face are also relevant for sign language processing. In a reaction time study they investigated the time Deaf signers need to choose the correct German written word (of two available options) after they have seen a sign with or without the appropriate non-manual. Signs with facial expressions alone, with torso/head movement alone, and signs with both facial expression and head/torso movement were tested. Subjects needed longer to choose the correct German word after seeing an item without the appropriate non-manual (independent of the kind of non-manuals).

Although there are a number of studies investigating the neural representation of NMMs in signers’ brains and showing that the manipulation/absence of NMMs influences language processing (i.e. indicating that they are relevant for sign language processing), there are – to the best of our knowledge – no investigations examining how NMMs (other than mouthing) may influence online sign language processing. In a number of studies examining online sign language processing, NMMs were either excluded or used in a reduced form (e.g. Hosemann, 2015; Hosemann, Herrmann, Steinbach, Bornkessel-Schlesewsky, & Schlesewsky, 2013; Jednoróg et al., 2015). The reason for this might be that their inclusion complicates data analysis and interpretation of results. In particular, different kinds of NMMs can be and are often in parallel with hand articulation and it is not currently clear which of these cues are relevant for processing. However, NMMs are crucial in sign languages and thus have to be considered when aiming to examine natural sign language processing. That NMMs may have an impact on online sign language processing was suggested by our previous work (Krebs, 2017; Krebs, Malaia, et al., 2018; Krebs, Wilbur, et al., 2018), wherein NMMs seem to be relevant during the processing of locally ambiguous argument structures in ÖGS.

Topic marking and topicalisation in sign languages

For many sign languages a form of “topic marking” or “topicalisation” has been described. Topics in sign languages occur in sentence-initial position and are marked by specific NMMs. For ASL raised eyebrows and raised chin are topic markers (e.g. Aarons, 1994; Baker-Shenk, 1983; Liddell, 1980; Wilbur & Patschke, 1998). Further, during the signing of the latter part of the topic sign, the head may be lowered. In addition, the topic may be accompanied by widened eyes and/or by several rapid head nods. Aarons (1996) also notes that the NMMs are visible slightly before the onset of the co-occurring manual sign. Interestingly, the components involved in non-manual topic marking are sign language specific.

The sign bearing topic marking is usually followed by a slight pause, i.e. an intonational break, which may be accompanied by an eye blink and a change of NMMs (Wilbur, 1994b). Thus, topics are usually set off prosodically from the rest of the clause (Aarons, 1994, 1996; Pfau & Quer, 2010). In addition, signs with topic marking are longer in duration than non-topic signs (Liddell, 1978; Wilbur & Martínez, 2002).⁷

Aarons (1994, 1996) further noted that in ASL other forms of topic marking are possible. For instance, sideways body shifting may be used for topic marking, with the topic signed on one side and the rest of the sentence on the other side. In addition, items may appear in topic position without a specific non-manual topic marking. In this case the item in topic position is established in a specific location in space by an index-sign or a classifier and is followed by a pause which sets it off from the rest of the sentence.

As in spoken languages, a marked topic may differ in its syntactic function. Wilbur (2012) discusses different topic types (discourse-level, sentence-level), their functions (guide listener's attention, reintroduce previous topic, focus a particular argument), and their markings in English and ASL. Of relevance, discourse topics are usually backgrounded information, leading to their reduction (e.g. referenced as pronouns rather than full NPs) or complete omission (in pro-drop languages), whereas sentence-level topics are usually sentence-initial, and may be stressed or unstressed based on their function. A sentence-level topic that reintroduces a previous topic, or simply guides the receiver's attention, is often set off by an intonational break from the rest of the sentence, but never has the main/primary stress of the sentence. A sentence-level topic that “highlights” an argument (subject or object) is also set off from the rest of the sentence by an intonational break but does carry the highest/heaviest stress of the sentence. Such a topic is presumed to have moved from sentence-internal position to the front/initial position as a result of a syntactic operation that is triggered by semantic or pragmatic factors (such as contrast, disagreement, providing missing information, etc.).

Investigating possible relations of topic phrases to other sentential elements in ASL, Aarons (1994, 1996) distinguished between topics that have been topicalised, i.e. moved to topic position, and topics that are base-generated in topic position. Aarons (1994) distinguishes three forms of topic marking, differentiated by specific NMMs (Aarons used the glosses tm1, tm2 and tm3 for these topic markings). She notes that tm1 (raised eyebrows, slight back- and sideward head tilt and eyes wide open) is used with moved topics, whereas tm2 (raised eyebrows, large head movement back- and sidewards and eyes wide open) and tm3 (raised eyebrows, slight rapid head nods, eyes wide open) accompany base-generated topics, although topics marked by tm2 or tm3 differ with respect to their function and meaning.^{8, 9}

Wilbur (2012) provides the traditional labels: “topicalisation” for tm1, “topic” for tm2 and “left dislocation” for tm3.¹⁰ We will follow this terminology to differentiate between moved (topicalised) and base-generated topics. Examples for these different forms from ASL are given below (from Aarons, 1996, p. 71f).¹¹

Table

____t

(1) MARYi, JOHN LOVE ti.

Mary, John loves.

________t

(2) VEGETABLE, JOHN LIKE CORN

As for vegetables, John likes corn.

____t

(3) MARYi, JOHN LIKE IXi

As for Mary, John likes her.

Example (1) shows an example with contrastive topicalisation in which the moved item (MARY) receives the primary stress. (2) presents a plain base-generated topic (VEGETABLE) that is unstressed unless contrastively marked (note the set-membership relationship between the topic VEGETABLE and the mention in the main sentence of CORN). (3) represents left dislocation in which the nominal item in topic position (MARY) is usually unstressed unless contrastively marked. In the main clause a resumptive pronoun co-indexing the item in topic position is required, showing that the item in topic position has not been moved, but is base-generated in topic position.

By discussing the difference between focus and topic marking, Wilbur (2012) pointed out that topicalisation (e.g. example 1) is a form of focusing (i.e. contrastive focus marking), because the topicalised argument represents new information to the addressee, which is a typical characteristic of focused elements, whereas topics (examples 2 and 3) usually represent old, i.e. given information which has already been established within previous context.

In the previous EEG studies where the subject preference effect was tested in the context of topic marking in Mandarin, different kinds of topic constructions were involved. Wang (2011) used context-supported topics, that is, discourse level topics, and found a subject preference. He (2016), on the other hand, tested sentence-level moved topics, i.e. topicalisation of type (1) and did not observe the subject preference. In He’s (2016) study, stimuli involve intonational marking of the topics in sentence-initial position in the form of a slight pause before the main clause as well as additional topic-marking words preceding and following the topic itself. Prosodic marking in the form of a pause as well as topic-marking words were not part of the stimuli tested by Wang (2011).

Online processing of word order and topic marking

The present investigation examines whether non-manual topic marking on the sentence-initial argument may influence the processing of locally ambiguous argument structures in ÖGS. In particular, we tested whether the subject preference is eliminated in the presence of topic marking in ÖGS comparable to He’s (2016) study of spoken Mandarin. Like He, we used isolated sentences involving a topic followed by a pause. The stimulus material comprised SOV and OSV structures with or without topic marking on the sentence-initial argument (Table 1). The non-topic SOV and OSV orders, which functioned as a baseline condition for the present experiment, were analysed and reported in a separate paper (Krebs, Malaia, et al., 2018). That analysis revealed a reanalysis effect for OSV compared to SOV orders (negative ERP effect) bound to a time point when the transitional movement towards the disambiguating verb was visible.

Table 1. Example sentences of the four experimental conditions. Agreeing verbs were presented in SOV and OSV orders with or without topic marking on the sentence-initial argument. For an explanation of notation conventions see footnote 11.

	SOV	OSV
Non-topic	WOMAN IX3a WOMAN IX3b 3aASK3b The woman (left) asks the woman (right).	WOMAN IX3a WOMAN IX3b 3bASK3a The woman (right) asks the woman (left).
Topic	__________t WOMAN IX3a, WOMAN IX3b 3aASK3b The woman (left), asks the woman (right).	__________t WOMAN IX3a, WOMAN IX3b 3bASK3a The woman (left), the woman (right) asks.

In the material used for this study, ÖGS topic marking is expressed by NMMs, such as raised eyebrows, wide eyes, chin directed towards the chest and an enhanced mouthing (see Figure 1). Further, the index which references the argument in topic position is followed by a pause, during which the index is briefly held in space.

Figure 1. Illustration of the four experimental conditions: In all constructions (topic and non-topic SOV & OSV) the argument NPs (in this case GIRL) were signed in the same order and were referenced at the same points in space; i.e. the first argument was always referenced at the signer's left side. SOV and OSV orders were presented either with (B) or without (A) topic marking on the sentence-initial argument and its corresponding index sign. The movement of the sentence-final critical verb sign (in this case the agreeing verb HELP) unambiguously marks the argument structure.

Prior research leads to two competing hypotheses. If processing of information structure (topicalisation) eliminates subject preference bias, as it did in spoken Mandarin (He, 2016), no subject-preference-driven reanalysis effect would be expected in topicalised sentences, as compared to non-topicalised ones. In line with this hypothesis, we expected a difference in the processing of topic and non-topic orders, such that the reanalysis effect would be absent for the topic orders. If, on the other hand, information structure (topicalisation) processing and syntactico-semantic processing (that drives subject preference) interface in sign language, the previously observed reanalysis effect for OSV word order would be expected to be reflected in EEG data. This hypothesis suggests a similar processing pattern for topic and non-topic orders, with a subject preference effect expected for topic orders.

Participants

From the 25 persons who participated, 20 (9 females) were included in the final analysis, with a mean age of 39.37 years (sd = 10.19; range = 28–58 years). All participants were born Deaf or lost their hearing early in life. Three have Deaf parents, the others have hearing parents. Half acquired sign language between 4 and 7 years, five acquired sign language between 0 and 3 years, and five learned ÖGS at a later age: one between 13 and 17 years, another between 18 and 22 years and three after the age of 22. They came from different areas of Austria (Salzburg, Vienna, Upper Austria, Lower Austria, Styria). Fifteen were right-handed, four left-handed and one did not have a dominant hand preference (as tested by an adapted German version of the Edinburgh Handedness Inventory; Oldfield, 1971). None of them showed any neurological or psychological disorders. All had normal or corrected vision and were not influenced by medication or other substances which may have an impact on cognitive ability. Informed consent was obtained in written form. The participants received 30€ per session.

Materials & design

We used a 2 × 2 repeated measures ANOVA with the factors ORDER (two levels SOV and OSV) and TOPIC (two levels +Topic and -Topic). A set of 40 agreeing verbs were presented in each condition resulting in 160 critical sentences. To avoid strategic processing 120 filler sentences were additionally included for a total of 280 videos. The fillers consisted of (a) classifier constructions expressing spatial relations between arguments varying in word order (n = 80) and (b) ÖGS videos which were presented in time-reversed manner (n = 40). The time-reversed videos were created from one of the critical conditions (the OSV orders without topic marking) and were included as filler material to ensure the reliability of participants’ ratings (i.e. to be sure that participants understood the task; see section “Procedure” for a description of the task).

In the vast majority of stimuli, similar NMMs during the production of the verb sign were observed: signer's body shifted towards subject position and chest, face and eye gaze directed towards object position. Almost half of the verbs were one-handed, the other two-handed (19 two-handed verbs; 21 one-handed verbs). The sentence contexts involved non-compound, relatively frequent signs (the noun signs were MAN, WOMAN, GIRL and BOY). To avoid any semantic biases, we used the same arguments within one sentence (e.g. The man asks the man). The referencing of the arguments within the sentences were kept constant within conditions in that the sentence-initial argument was always referenced at the left side of the signer.¹² All of the material was signed by a right-handed Deaf woman who acquired ÖGS early in life, teaches ÖGS, uses ÖGS in her daily life and is a member of the Deaf community.

Unlike print stimuli, where no stimulus movement is involved, or acoustic stimuli, where synthetic speech can be used to control details that are otherwise less-controlled in natural speech, signed stimuli involve movement in the stimulus itself. In most prior studies, attempts to control for this used sequences of still frame images extracted from videos (e.g. Kutas, Neville, & Holcomb, 1987; Neville et al., 1997), and only recently has actual video of natural signing been used as stimuli (e.g. Capek et al., 2009; Hänel-Faulhaber et al., 2014; Hosemann, 2015; Hosemann et al., 2013). To understand how this affects the data analysis, we need to review measures that are taken to provide confidence in the final results. Here we provide an overview of how the stimuli are checked for dynamic effects, such as durations and timing of onsets (see also Krebs, Malaia, et al., 2018).

Checking the stimuli for dynamic effects

When using dynamic stimuli (such as natural speech or signing), is it important to control for possible timing differences within the stimuli. Studies investigating auditorily presented speech stimuli typically provide acoustic analysis, i.e. a description of the prosodic structure of their stimuli (e.g. duration, mean intensity or fundamental frequency; e.g. Haupt et al., 2008; He, 2016; Wang, 2011). As natural, dynamic sign language stimuli are used in this study, we provide a detailed account of the dynamics of the sign language stimuli used here. To check whether there were any systematic differences in timing within the stimulus material, analyses of variance comparing specific time points in the structures and durations between those time points were calculated. The specific time points and intervals considered for comparison are presented in Table 2. The ANOVAs involved the factors ORDER (levels SOV and OSV) and TOPIC (levels +Topic and -Topic). In the following only significant effects (p ≤ 0.05) will be reported.

Table 2. Time points and intervals compared. TP stands for time point; INT stands for Interval; IX refers to index-sign; NP refers to noun phrase.

Comparison of time points

TP 4: onset NP2

Comparison of mean time points revealed a significant main effect of TOPIC [F(1, 39) = 6.06; p < 0.05] only with respect to TP4 [mean time points: non-topic SOV: 2.48 (0.29); non-topic OSV: 2.43 (0.25); topic SOV: 2.55 (0.25), topic OSV: 2.54 (0.23)].¹³ Thus, the onset of the second argument was presented significantly later in the marked topic compared to the non-topic sentences (on average 70 ms later in SOV orders; on average 110 ms later in OSV orders). Due to the fact that the index referencing the first argument in topic sentences is followed by a pause, i.e. the index is slightly held in space, it is expected that the onset of the second argument would appear later in topic compared to non-topic sentences.

Comparison of durations between certain time points

INT 3: offset IX-NP1 to onset NP2

The comparison of the mean durations between certain time points revealed a significant main effect of TOPIC [F(1, 39) = 60.65; p < 0.001] for INT 3 [mean duration: non-topic SOV: 0.62 (0.07), non-topic OSV: 0.63 (0.09), topic SOV: 0.72 (0.11), topic OSV: 0.74 (0.09)]. Thus, the transition from the index referencing the first argument towards the onset of the second argument was significantly longer in topic compared to non-topic sentences (on average 100 ms longer in SOV orders; on average 110 ms longer in OSV orders). This longer INT 3 in topic sentences is in line with the observation that in topic sentences the index referencing the sentence-initial argument is held in space (before the hand starts its transitional movement towards the onset of the second argument).

INT 4: onset NP2 to offset IX-NP2

Further, a significant main effect of TOPIC [F(1, 39) = 7.19; p < 0.05] for INT 4 was observed [mean duration: non-topic SOV: 0.77 (0.13), non-topic OSV: 0.75 (0.12), topic SOV: 0.71 (0.13), topic OSV: 0.72 (0.13)]. Hence, the duration from the onset of the second argument to the time point when the index referencing the second argument was set was significantly shorter in topic sentences compared to non-topic sentences (on average 60 ms shorter in SOV orders; on average 30 ms shorter in OSV orders). The shorter INT 4 in topic sentences compared to non-topic sentences may indicate rhythmic compensation. The signer may have compensated in topic orders for the pause in the topic sentences by shortening the duration from the onset of the second argument to the time point the index referencing the second argument was set.

INT 7: obvious transition to handshape

In addition, for INT 7 a significant interaction ORDER × TOPIC was revealed [F(1, 39) = 4.17; p < 0.05; mean duration of transition: non-topic SOV: 0.28 (0.09), non-topic OSV: 0.27 (0.10), topic SOV: 0.26 (0.10), topic OSV: 0.29 (0.09)]. The resolution of the interaction ORDER × TOPIC by ORDER revealed a significant TOPIC effect for SOV orders [F(1, 39) = 5.02; p < 0.05], but not for OSV orders. The resolution of the interaction ORDER × TOPIC by TOPIC revealed a significant ORDER effect for topic sentences [F(1, 39) = 5.31; p < 0.05], but not for non-topic sentences. Thus, the transition from the time when the index referencing the second argument obviously moves away from its final position towards the time when the handshape of the verb is established was significantly shorter in topic SOV sentences compared to non-topic SOV sentences (on average 20 ms shorter). Further, this transition was significantly shorter in topic SOV structures compared to topic OSV sentences (on average 30 ms shorter).

INT 8: handshape to verb movement starts

A significant main effect of ORDER [F(1, 39) = 9.51; p < 0.01] was observed with respect to INT 8 [mean duration: non-topic SOV: 0.22 (0.10), non-topic OSV: 0.19 (0.09), topic SOV: 0.21 (0.09), topic OSV: 0.18 (0.09)]. Thus, the duration from the time when the handshape of the verb was established to the time when the verb movement starts was significantly shorter in OSV compared to SOV sentences (for both topic and non-topic sentences, the duration was on average 30 ms shorter in OSV sentences). This shorter INT 8 in OSV orders can be explained by the fact that in OSV orders the hand articulating the verb immediately starts producing its movement from the reference locus associated with the second argument as soon as the target handshape has been established. In SOV orders, however, the hand referencing the second argument has to move back to the subject position (to the argument which was referenced first in SOV orders) before the verb movement from subject to object position can start (see Figure 2). This additional trajectory back to the position of the first argument required in SOV orders may have led to the longer transitional movement in SOV orders.

Figure 2. Schematic illustration of the extra hand movement required in SOV orders (right picture, dotted line) compared to OSV structures (left picture). In both conditions the signer referenced the first argument on her left side and the second argument on the right (indicated by the numbers). In both conditions the argument structure shown by the verb is expressed by a movement from the subject (S) to the object (O) position (indicated by the continuous arrows). In OSV orders (left) this movement was produced from the argument referenced second (S) towards the argument referenced first (O). In SOV orders (right), however, to produce a movement from the subject to the object position, the signer first had to move back from the position of the second argument (O) towards the first argument (S) (indicated by the dotted arrow).

INT 9: verb movement starts to verb offset

Additionally, a significant main effect of ORDER [F(1, 39) = 5.70; p < 0.05] regarding INT 9 was revealed [mean duration: non-topic SOV: 0.83 (0.18), non-topic OSV: 0.87 (0.19), topic SOV: 0.82 (0.17), topic OSV: 0.84 (0.17)]. Thus, the duration from the time when the verb movement starts to the movement offset was significantly shorter in SOV compared to OSV orders (on average 20 ms shorter in topic SOV sentences; on average 40 ms shorter in non-topic SOV sentences). Possibly this effect also reflects compensation: Due to the fact that the transition from the time when the handshape of the verb was established to the time the verb movement starts is longer in SOV orders, the subsequent duration from the onset of the verb movement to its offset may have been shortened in SOV sentences as a prosodic adjustment.

These observed differences in timing must be taken into account when interpreting the ERP effects.

Procedure

The material was presented in 14 blocks, each consisting of 20 sentences. Every trial started with the presentation of a 2000 ms fixation cross to get the participant's attention, followed by an empty black screen for 200 ms. Then a stimulus sentence (one video) was presented in the middle of the screen. Each trial ended with a rating task, indicated by a green question mark for 3000 ms after each stimulus. Participants had to rate the videos on a scale from one to seven as to whether the stimulus was good ÖGS or not (1 for “that is not ÖGS”; 7 for “that is good ÖGS”). Ratings were given by button press on a keyboard. Instructions were given by an ÖGS video signed by one of the authors. Prior to the actual experiment, a training block was presented to familiarise participants with task requirements and permit them to ask questions in case anything was unclear. The duration of breaks after each block was determined by the participants themselves. Participants were instructed to avoid eye movements and other motions during the presentation of the video material and to view the sentences with attention.

EEG recording

The EEG was recorded from twenty-six electrodes (Fz, Cz, Pz, Oz, F3/4, F7/8, FC1/2, FC5/6, T7/8, C3/4, CP1/2, CP5/6, P3/7, P4/8, O1/2) fixed on the participant's scalp by means of an elastic cap (Easy Cap, Herrsching-Breitbrunn, Germany). Horizontal eye movements (HEOG) were registered by electrodes at the lateral ocular muscles and vertical eye movements (VEOG) were recorded by electrodes fixed above and below the left eye. All electrodes were referenced against the electrode on the left mastoid bone and offline re-referenced against the averaged electrodes at the left and right mastoid. The AFz electrode functioned as the ground electrode. The EEG signal was recorded with a sampling rate of 500 Hz. For amplifying the EEG signal we used a Brain Products amplifier (high pass: 0.01 Hz). In addition, a notch filter of 50 Hz was used. The electrode impedances were kept below 5 kΩ. Offline, the signal was filtered with a bandpass filter (Butterworth Zero Phase Filters; high pass: 0.1 Hz, 48 dB/Oct; low pass: 20 Hz, 48 dB/Oct).

Data analysis

Behavioural data

Behavioural data was analysed using linear mixed-effects models with the lme4 package (Bates, Mächler, Bolker, & Walker, 2015) in R (R Core Team, 2018). The effects of sign order and topic marking on acceptability ratings and reaction times were examined separately. We defined a model that included an interaction between the two-level factors ORDER and TOPIC as fixed effects. The random effects structure consisted of by-participant and by-item random intercepts.¹⁴ The same model was used for analysing acceptability ratings as well as reaction times.¹⁵ A t-value of 2 and above was interpreted as indicating a significant effect (Baayen, Davidson, & Bates, 2008). Additional p-values were calculated with the package “lmerTest”. Only significant effects and interactions are reported.

ERP data

Analysis of the ERP data was carried out by comparison of the mean amplitude values per time window, per condition and per participant in seven regions of interest (ROIs). The factor ROI involved the levels anterior left = F7, F3, FC5; anterior right = F8, F4, FC6; central left = FC1, CP5, CP1; central right = FC2, CP6, CP2; posterior left = P7, P3, O1; posterior right = P8, P4, O2; and midline = Fz, Cz and Pz. Time windows were determined by descriptive analysis. The signal was corrected for ocular artifacts by the Gratton and Coles method (Gratton, Coles, & Donchin, 1983) and screened for artifacts (minimal/maximal amplitude at −75/+75 µV). Data was baseline-corrected to −300–0. For each condition not more than 25% of the trials were excluded. The percentage of trials remaining after artifact rejection per condition and per Trigger are presented in Table 3. Participants were excluded from analysis if less than 60% of the critical trials were left after artifact correction. Statistical analysis was carried out in a hierarchical manner, i.e. only significant interactions (p ≤ 0.05) were included in step-down analysis. For statistical analysis of the ERP data an analysis of variance (repeated-measures ANOVA) was computed including the factors ORDER (SOV vs. OSV) and TOPIC (+Topic vs. -Topic) as well as the factor ROI. To correct for violations of sphericity, the Greenhouse and Geisser (1959) correction was applied to repeated measures with greater than one degree of freedom.

Table 3. Remaining trials after artifact rejection per condition and per trigger marker.

	Non-topic SOV	Non-topic OSV	Topic SOV	Topic OSV
Trigger 1	75%	75%	75%	77%
Trigger 2	88%	82.5%	87%	85%

Trigger marking

Because we wanted to examine whether topic marking on the first argument is reflected in the EEG signal, we measured ERPs with respect to the onset of the first argument (Trigger “Onset argument 1”; TP 2 in Table 2), which was defined as the time when the target handshape of the critical sign was established and its target location was reached. To examine whether topic marking influences the subject preference, we measured ERPs in the disambiguation area. As outlined above, the analysis of the non-topic SOV and OSV structures (presented in Krebs, Malaia, et al., 2018, and as baseline conditions here) revealed a reanalysis effect for OSV compared to SOV orders with respect to a time point when the transitional movement towards the disambiguating verb was visible. Therefore, ERPs here were also measured with respect to the obvious start of the transitional movement of the index referencing the second argument towards the verb sign (Trigger “Transition” TP 7 in Table 2).

Results

Of the 25 tested participants, 20 were included in the final analysis. Four were excluded due to artifacts (less than 60% of critical trials remaining after artifact rejection). One was excluded due to behavioural noncompliance. Only significant effects (p ≤ 0.05) are reported.

Behavioural data

All conditions were rated relatively high (mean ratings for all four conditions above 5.89 on a scale from 1 to 7). Table 4 provides an overview of the acceptability ratings and reaction times.

Table 4. Mean ratings and reaction times as well as corresponding standard deviations (sd) for the four experimental conditions.

Condition	Mean acceptability rating (sd)	Mean reaction time in ms (sd)
SOV	6.10 (0.90)	880.06 (459.81)
OSV	5.89 (1.07)	886.40 (442.82)
Topic SOV	6.16 (0.84)	879.35 (461.62)
Topic OSV	5.95 (1.14)	869.60 (452.82)

LME analysis of acceptability ratings revealed a significant ORDER effect. The results for the fixed effects revealed an intercept of all ratings (mean over all conditions) of 5.89. The analysis revealed a significant higher rating for SO orders in contrast to OS orders (t = 3.82; p < 0.001), increasing the rating intercept by 0.21 ± 0.05 (standard errors) for SO orders compared to OS orders. LME analysis of the reaction times did not reveal any significant effects (summaries of model fit for acceptability ratings and reaction times are presented as Appendix I).

ERP data

ERP effects will be reported per Trigger point and time window. Note that time windows were determined by descriptive analysis.

Analysis 1: Trigger at onset of argument 1

With regard to Trigger “Onset argument 1”, visual inspection revealed a more pronounced positivity for topic compared to non-topic sentences in the 0–150 ms time window (mean amplitude values in µV and standard deviations per condition: non-topic SOV: M = 0.36; sd = 1.25; non-topic OSV: M = 0.27; sd = 1.50; topic SOV: M = 1.20; sd = 1.44; topic OSV: M = 0.75; sd = 1.60) (Figure 3).

Figure 3. Grand average ERPs at Cz position showing the topic effect on the sentence-initial argument (Trigger “Onset argument 1”) for the topic compared to non-topic SOV orders. The vertical line represents the onset of the sentence-initial argument. Negativity is plotted upwards.

Within the 0–150 ms time window, statistical analysis revealed a significant main effect of TOPIC (ERPs were more positive for topic compared to non-topic orders) [F(1, 19) = 11.01, p < 0.01, ${\eta }_p^2$ = 0.37].

Analysis 2: Trigger at transition point

With respect to Trigger “Transition”, visual inspection revealed a pronounced negativity for OSV compared to SOV sentences in the 200–400 ms time window (mean amplitude values in µV and standard deviations per condition: non-topic SOV: M = 1.34; sd = 1.99; non-topic OSV: M = 0.28; sd = 1.71; topic SOV: M = 1.04; sd = 1.88; topic OSV: M = −0.01; sd = 1.75).

Furthermore, in the 700–950 time window a more negative going waveform for topic compared to non-topic sentences was revealed (Mean amplitude values in µV and standard deviations per condition: non-topic SOV: M = 4.57; sd = 3.50; non-topic OSV: M = 3.95; sd = 4.04; topic SOV: M = 4.58; sd = 3.30; topic OSV: M = 3.48; sd = 3.93) (Figure 4).

Figure 4. Grand average ERPs at Trigger “Transition” illustrated with respect to the −300–1000 ms time window (x-axis tick marks in 200 ms intervals). Data analysis revealed an order effect (but no topic effect) within the 200–400 ms time window for OSV compared to SOV orders as well as a topic effect within the 700–950 ms time window for the topic compared to the non-topic orders. The vertical line represents the time point when the transitional movement of the hand producing the verb movement was visible. Negativity is plotted upwards.

Within the 200–400 ms time window, the statistical analysis revealed a significant main effect of ORDER (ERPs were more negative for OSV compared to SOV orders) [F(1, 19) = 13.29, p < 0.01, ${\eta }_p^2$ = 0.41].

Within the 700–950 ms time window, the statistical analysis revealed a significant interaction ORDER × ROI [F(6, 114) = 4.59, p < 0.05, ${\eta }_p^2$ = 0.19] and a significant interaction TOPIC × ROI [F(6, 114) = 9.88, p < 0.001, ${\eta }_p^2$ = 0.34]. To better understand the interaction ORDER × ROI, we conducted step-down ANOVAs for each of the six levels of the factor ROI (anterior left = F7, F3, FC5; anterior right = F8, F4, FC6; central left = FC1, CP5, CP1; central right = FC2, CP6, CP2; posterior left = P7, P3, O1; posterior right = P8, P4, O2; and midline = Fz, Cz and Pz), such that for each level of ROI, a step-down ANOVA for the factor ORDER was calculated. For resolving the interaction ORDER × TOPIC we also divided the data into the six levels of the factor ROI (anterior left = F7, F3, FC5; anterior right = F8, F4, FC6; central left = FC1, CP5, CP1; central right = FC2, CP6, CP2; posterior left = P7, P3, O1; posterior right = P8, P4, O2; and midline = Fz, Cz and Pz). Then, for each level of ROI, a step-down ANOVA with the factor TOPIC was calculated. The resolution of the interaction TOPIC × ROI by ROI revealed a significant effect of TOPIC (ERPs were more negative for topic compared to non-topic orders) at the right anterior ROI [F(1, 19) = 6.66, p < 0.05, ${\eta }_p^2$ = 0.26]. The resolution of the interaction ORDER × ROI did not reveal any significant effects.

In sum, at Trigger “Transition” a broadly distributed negative effect was observed for OSV compared to SOV orders. In addition, effects for topic compared to non-topic orders were revealed at Trigger “Onset argument 1” (broadly distributed positivity) as well as in later time windows with respect to Trigger “Transition” (right anterior negativity).

Discussion

The present study aimed to investigate the influence of topic marking on the processing of locally ambiguous argument structures in ÖGS. In particular, we examined whether topic marking may eliminate the subject preference in ÖGS as reported for spoken Mandarin (He, 2016). The results show effects of sign order as well as effects of topic marking on sign language processing. However, no interaction between the factors ORDER and TOPIC was observed.

Subject preference effect

This experiment shows that word order variations (SOV vs. OSV) are processed in ÖGS similarly, either with or without topic marking on the sentence-initial argument. Data analysis revealed a negative ERP effect for topic OSV in contrast to topic SOV orders when disambiguating information was available – a similar effect that has been observed for non-topic OSV compared to non-topic SOV orders in ÖGS (Krebs, Malaia, et al., 2018). That is, the subject preference was present and led to processing costs in OSV compared to SOV orders, even with topic marking. Thus, the structures for which the subject preference can be observed in ÖGS can be extended to topic constructions, as topic marking did not influence the subject preference in ÖGS.

The findings on ÖGS in the context of the results reported for spoken Mandarin

The present study showed a different pattern of results compared to He’s (2016) report for Mandarin. What might be the reason for the different findings observed for ÖGS in contrast to Mandarin? Why can the subject preference be eliminated by topic marking in Mandarin, but not in ÖGS? Does the present study shed light on interpretation of the inconsistent results reported for Mandarin? By discussing our findings on ÖGS in relation to the results described for Mandarin, in the following subsections we outline possible answers to these questions. Because there are only two previous studies investigating the subject preference in the context of topic marking (both on spoken Mandarin) that we are aware of, we necessarily limit the scope of the conclusions based on the comparison. The following discussion should be seen as first attempt at consolidation of available cross-linguistic and cross-modal findings.

Language modality

First, one could assume that the difference in findings of the present experiment and the study from He (2016) stems from the difference in language modality (auditory vs. visual), in that perhaps topic marking can eliminate the subject preference in spoken Mandarin, but not in a sign language such as ÖGS. Further, it also might make a difference in how, i.e. in which modality, topic marking is expressed. In ÖGS topic marking is expressed by visual cues (i.e. NMMs), but in Mandarin topics are marked by auditory cues. Thus, it could be argued that topic marking expressed by NMMs cannot eliminate the subject preference, but auditory topic marking can. Because we did not observe any modality effect regarding the processing of simple non-topic orders (i.e. the subject preference was observed for non-topic OSV vs. non-topic SOV orders in ÖGS; see Krebs, Malaia, et al., 2018; Krebs, Wilbur, et al., 2018), we do not assume that a modality difference per se caused the difference in findings between Mandarin and ÖGS. Despite the different modality of topic marking, the topics of both languages have the same focus function. Thus, it is unlikely that the form of how topic marking is expressed (i.e. in which modality) is responsible for the difference in findings.

Language-specificity

Second, it could be assumed that language specific differences between Mandarin and ÖGS might be responsible for the different results. Thus, there is the possibility that in ÖGS, factors such as topicalisation may not have as much impact on sentence processing as observed for Mandarin, i.e. leading to an elimination of the subject preference in Mandarin, but not in ÖGS. Although ÖGS shares important structural properties with Mandarin such as that both are pro-drop languages and use topic marking frequently, it may be the case that topic marking just does not influence sentence processing to the same extent in both languages. This difference may stem from the fact that Mandarin is an SVO language and ÖGS is an SOV language. However, the previous studies on Mandarin (He, 2016; Wang, 2011) do not support this language-specific assumption. For Mandarin, inconsistent results were reported in that in one study the subject preference was revealed independent of topic marking (Wang, 2011) and in the other the subject preference was eliminated in topic constructions (He, 2016). Hence, it is not plausible to suggest that language-specific aspects are responsible for the (non-)occurrence of the subject preference in the context of topic marking in Mandarin and ÖGS.

Type of topic

Third, it might be that a difference in the type of topic may have led to diverging results for Mandarin and ÖGS. Wang (2011) observed the subject preference in structures in which the (non-moved, i.e. base-generated) topic was identified only by the preceding discourse context. In contrast, He (2016) reports elimination of the subject preference if the sentences contain a topic that has been moved into topic position (fronted). One could, following He, suggest that only syntactically moved topics result in elimination of the subject preference. But the movement status of the topic seems not to be a crucial factor for finding the subject preference even in Mandarin. Whereas subject preference effects disappeared in Mandarin structures involving topicalisation, He (2016) did not observe similar loss of subject preference with wh-sentences that involved wh-movement. If syntactic movement were the triggering factor for subject preference elimination, there should have been similar loss in the wh-sentences as well. Thus, movement status of the topic does not seem to be a plausible cue triggering the difference between He’s (2016) and the present study.

Form of topic marking

Another difference between Wang and He is that He's topics are also preceded and followed by additional topic marking words [“THAT-CL(assifier)”, topic marker “Ah”]. This means that He's sentences begin with (at least 2) extra words before reaching the main clause. Presumably this could have engendered timing and processing changes independently of the syntactic movement status of the topic itself. In fact, He (2016) points out that a possible explanation for the difference between his findings and Wang’s (2011) might stem from the fact that the form of topic marking used by Wang was not “strong enough” to interfere with the subject preference. Since Wang's topics were identified by discourse, there was no overt topic marker on the topic itself.

Along these same lines, one could argue that a possible explanation for the subject preference in ÖGS topic marked structures may be that the topic marking NMMs were possibly not strong enough to influence syntactic processing of the main clause. Mandarin and ÖGS are not strictly comparable in their topic marking, because in Mandarin more topic markers are used (“THAT-CL(assifier)”, topic marker “Ah” and prosodic phrase boundary) in contrast to ÖGS (non-manual marking and prosodic phrase boundary). In the present study, however, we did use clear topic marking (similar to He, 2016, but in contrast to Wang, 2011) and yet still observed the subject preference effect (in contrast to He, 2016; and like Wang, 2011).¹⁶

Nevertheless it could be the case that in ÖGS topic marking alone is not a strong enough cue for influencing the subject preference. Perhaps a specific sentence context along with the topic marking could have influenced processing to a greater extent leading to elimination of the subject preference. Although the isolated sentences with sentence-initial topic marking were rated relatively high in acceptability, they are possibly rather untypical in the sense that such structures are usually not expressed without a specific sentence context, i.e. out of the blue. Aarons (1994, 1996) noted that ASL topicalised arguments (with tm1 marking) appear more naturally in the context of a closed set of known members, whereby the topic is one of the members and is highlighted with regard to the other members of this set (4) or when there is emphasis/focus on the topic which is contrasted with a previously established item in the discourse (5; examples from Aarons, 1996, p. 76).¹⁷

Table

____t

(4) FOUR WOMEN LIVE IN HOUSE IX. MARYi, JOHN LOVE ti

Four women live in that house over there. Mary, John loves.

____t

(5) JOHN NOT-LIKE JANE. MARY, IX3a LOVE.

John does not like Jane. Mary, he loves.

Therefore, it is possible with an appropriate sentence context, similar to (4) or (5) above, that the reanalysis effect for topic OSV sentences, i.e. the subject preference, would have been eliminated. If indeed such a combination of topic marking and context is required to eliminate the subject preference, this difference could be related to language- or even modality-specific characteristics. However, this assumption would have to be tested in future studies.

Sentence structure

Yet another possible difference which might explain the pattern of results for Mandarin and ÖGS as well as the inconsistent results reported for Mandarin is related to the structures. In Wang (2011) on Mandarin and in the present study on ÖGS, the influence of topic marking on subject preference was tested with simple word orders. In contrast, He (2016) used relative clause constructions (complex NPs). Thus, structural complexity may contribute to topic marking influences on the subject preference. Nonetheless, He (2016) reports effects reflecting a preference for subject-relative compared to object-relative clauses measured on the relativiser DE (occurring after the disambiguating verb in the topic sentences). In addition, the subject preference was also observed in the wh-sentences, which, like the topics, were also embedded in relative clause constructions. Thus, not (relative clause) complexity per se, but probably the combination of topic marking and relative clause constructions might have led to the elimination of the subject preference in Mandarin.

Presence vs. absence of context information

The present study might furthermore provide some additional information with respect to the interpretation of the results reported for Mandarin. The ÖGS results speak against the possibility that the difference between Wang’s (2011) and He’s (2016) studies stems from the fact that in one, isolated sentences, and in the other, non-isolated sentences, were tested. The present study shows the subject preference effect for isolated topic constructions, adding them to the report of subject preference in non-isolated constructions in Wang’s (2011) study. He (2016) used isolated sentences (as in the present study), but did not observe the subject preference (contrary to the present study). Thus, the presence/absence of context per se does not explain the difference between Wang's and He's studies.

Prosodic phrase boundary

He (2016) provided an alternative explanation for the subject preference with wh-clauses, but not for topicalisation in his studies on Mandarin. Rather than due to the factor of syntactic “topicalisation” (i.e. the sentence-external linguistic factor of topicality has a direct impact on sentence-level processing), the subject preference might have been eliminated simply through the phrase boundary present in topic structures. This phrase boundary isolates the topicalised argument from the following clause and therefore may be processed differently. But the present study also had sentence-initial topic marked arguments set apart from the rest of the sentence by prosodic phrase boundary and nonetheless observed a subject preference effect. This may speak against the possibility that the elimination of the subject preference in He's study is due to the prosodic boundary alone.

Effects of non-manual topic marking

Although the behavioural data did not reveal any interpretable effects for the sentences with topic compared to those without topic marking, effects for topic marking were observed at the neurophysiological level. An ERP effect (positivity) for sentences with topic marking compared to sentences without topic marking were observed with respect to the first argument (within the 0–150 ms time window). The effect for topic compared to non-topic orders with respect to the first argument is not implausible. Topics show a specific NMM absent in the non-topic structures (i.e. different eyebrow and chin position, enhanced mouthing, widening of eyes in topic compared to non-topic sentences). Although topic marking did not seem to play a role with respect to the assignment of argument roles, i.e. topic marking did not facilitate the processing of locally ambiguous OSV orders, the present findings suggest that topic marking may have an influence on the processing after reanalysis. Data analysis revealed a significant negative effect for topic compared to non-topic sentences overlapping in time with the processing of the disambiguating verb. We suggest that this observed effect (bound to the processing of the verb) may reflect processes related to the evaluation of the entire structure. In particular, the effect for topic compared to non-topic sentences may be interpreted as indexing enhanced processing costs during the evaluation/well-formedness phase (e.g. Bornkessel & Schlesewsky, 2006) for topic compared to non-topic orders. As mentioned above, sentences with topic marking usually occur in narrative context and not in isolation. Thus, the effect for topic orders might reflect the processing of implausible topic constructions presented without an appropriate sentence context. However, the behavioural data did not reveal any meaningful differences between topic and non-topic marked orders.

Linguistic structures and stimuli dynamics

Sign languages are four-dimensional, i.e. they are produced in the three dimensional signing space and unfold over time. Thus, when investigating the processing of natural sign language, one has to deal with a highly complex dynamic signal. The present study shows interesting differences in dynamics in the signal which are driven by linguistic factors. In more detail, the comparisons of specific time points and durations between these times revealed differences with respect to the time course within SOV and OSV orders with and without topic marking. Both topic marking and word order may influence the signing dynamics, resulting in latency shifts between conditions. For example, one component of topic marking, the pause after the topic, led to a shift in latency between topic and non-topic structures such that the onset of the second argument occurred later in topic stimuli sentences. This difference in timing was compensated for by the signer in order to catch up later during signing. In addition, word order caused a slight latency shift between conditions: the transition of the time when the verb handshape was established to the onset of the verb movement took longer in SOV orders, but this difference was then compensated for by the signer by shortening the duration from the onset to the offset of the verb movement in that condition. Note, however, whether these differences in timing are present in natural signing as well (i.e. in ÖGS signing not produced for creating stimulus material) has to be tested in further studies.

Crucially, to get an accurate picture of the language processing, these differences in timing have to be taken into account when interpreting ERP effects. In the present study the observed reanalysis effect overlaps with time intervals in the stimuli for which systematic differences in timing were observed. Therefore, one may argue that the effect may be at least partially the result of a shift in latency. In particular, the first part of the effect at Trigger “Transition” within the 200–300 ms time window overlaps with the phase in the stimulus material (i.e. the duration from Trigger “Transition” to the time point the handshape was established) for which a significant interaction ORDER × TOPIC was revealed. The resolution of this interaction by ORDER revealed a significant TOPIC effect for SOV orders (on average 20 ms shorter for topic SOV orders). The resolution of the interaction by TOPIC revealed a significant ORDER effect for topic sentences (on average 30 ms shorter for topic SOV orders). We do not assume that these differences in duration are reflected in the ERPs because if this were the case, we would expect an ORDER × TOPIC interaction as observed for differences in mean durations for the ERPs. This was not the case. With respect to Trigger “Transition” within the 200–400 ms time window only an ORDER effect, but no interaction was observed. The second part of the effect (in the 300–400 ms time window at Trigger “Transition”) overlaps with a period of time in the stimulus material for which a significant ORDER effect was revealed. The duration from the time at which the verb handshape was established to the time the verb movement starts was significantly shorter in OSV compared to SOV sentences (on average 30 ms shorter in topic and non-topic OSV). However, due to the fact that these differences in durations are relatively small (maximum 30 ms) it appears unlikely that the reanalysis effect is solely the product of a latency shift.

Conclusions

The present investigation extends the understanding of human cognitive processing during integration of visual and linguistic cues (non-manual topic marking in sign language) to identify neural parameters corresponding to linguistic features in the stimuli. We examined the influence of non-manual topic marking on the processing of word order variations in ÖGS. EEG and behavioural data indicated that the addition of topic marking (information structure) did not eliminate the subject preference phenomenon. Reanalysis effects were observed for OSV compared to SOV orders independent of whether the sentence-initial argument carried topic marking or not. The present study contributes to the literature by replicating and extending the previously reported results examining the subject preference in the context of topic marking. We used isolated topic constructions (similar to He, 2016, and in contrast to Wang, 2011) and observed the subject preference (in contrast to He, 2016; and similar to Wang, 2011). Because these are only two studies examining the subject preference in the context of topic marking that we are aware of, and they used different stimuli designs, it is difficult to conclusively determine what triggered the difference between He’s (2016) study and the present study. In addition to replicating earlier observations on the subject preference strategy (Krebs, Malaia, et al., 2018), we observed effects of topic marking on online processing of ÖGS. Non-manual topic marking influenced the processing of the topic argument, as well as later processing stages. Further studies on typologically different languages, as well as typologically unrelated sign languages, are needed to clarify processing mechanisms for locally ambiguous argument structures in the context of topic marking. The present study contributes to the larger question of multi-level processing in psycholinguistics – understanding of the mechanisms by which syntactic complexity, information structure, context, and language modality contribute to online comprehension.

Acknowledgements

We want to thank all Deaf informants taking part in the present study. Special thanks to Waltraud Unterasinger for signing the stimulus material. Additionally, we are grateful to Vadim Kimmelman for his helpful comments when reviewing this paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Evie Malaia http://orcid.org/0000-0002-4700-0257

Ronnie B. Wilbur http://orcid.org/0000-0001-7081-9351

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

1 In Wang (2011) the structures were disambiguated by animacy restrictions on the verb following the ambiguous argument. In He (2016), the verb after the ambiguous argument disambiguated the structures either through animacy or world knowledge.

2 There is general agreement that this process is a linguistic (grammatically controlled) process in sign languages (for a discussion see e.g. Lillo-Martin & Meier, 2011).

3 Other verb types include so-called “backwards verbs” (reverse path) and “spatial verbs” which take locative arguments instead of individuals. Recent analyses argue that these should not be considered irregular (Bos, 2016, 2017). They are not relevant to further discussion here.

4 At the moment there is no evidence of any form of case-marking on the arguments or of a default referencing (i.e. for example if subjects were always referenced at the ipsi- or contralateral side of the signer) in ÖGS.

5 Per convention Deaf with upper-case D refers to deaf or hard of hearing humans who define themselves as members of the sign language community. In contrast, deaf refers to audiological status.

6 Signers also show enhanced abilities of human motion/action processing (Corina et al., 2007; Corina & Grosvald, 2012).

7 In ASL the duration of signs may mark the phrase position of a particular sign, i.e. signs in phrase final position are lengthened (phrase final lengthening; e.g. Liddell, 1978; Wilbur & Malaia, 2018; Wilbur & Nolen, 1986).

8 Note that Aarons (1994, 1996) described additional non-manuals that may occur with tm1, tm2 and tm3.

9 Aarons (1994, 1996) pointed out that in ASL two topics can co-occur within one sentence, but they have to belong to different categories. If one of them is a moved topic (marked by tm1), it must appear after the other topic, i.e. in second position and thus adjacent to CP.

10 The definition Wilbur (2012) used for left dislocation is from McCawley (1988). The rest of the terminology is also based on previous works (e.g. Prince, 1984; Ziv, 1994).

11 Notation conventions: Signs are glossed with capital letters; IX = manual index sign; Subscripts indicate reference points within signing space; non-manual markings and the scope of non-manual markings are indicated by a line above the glosses; t at the end of the line stands for topic marking; the comma after the topic indicates the prosodic break, i.e. the pause after the topic. The notation conventions Aarons (1996) originally used were slightly adapted in this text.

12 Further, the background color as well as the light conditions in the video material were kept constant across conditions.

13 Mean time points are given in seconds; standard deviations are presented in parentheses.

14 Coded in R as lmer(Rating ∼ ORDER*TOPIC + (1 |Subject) + (1 |Item)).

15 Coded in R as lmer(log(Reaction time + 1) ∼ ORDER*TOPIC + (1 |Subject) + (1|Item)).

16 Although topic constructions, i.e. structures in which the sentence-initial argument bears a specific NMM and is prosodically set apart from the rest of the clause by a pause, have also been described for ÖGS (Hausch, 2008; Ni, 2014), there is – to the best of our knowledge – no study so far focusing on the question of whether (some of) these topic constructions involve syntactic movement. Given that ÖGS has a general basic word order of SOV, determining syntactic movement is also not straightforward. Therefore, it is an open question whether the non-manual topic marking used in the present study involves syntactic movement or not. Nonetheless, there is clear agreement on what the topic marking should look like, and we have used that in our study.

17 Original notation in Aarons (1996) was adapted for the present paper.