Validation: Knowledge- and Text-Based Monitoring During Reading

ABSTRACT

To create a coherent and correct mental representation of a text, readers must validate incoming information; they must monitor information for consistency with the preceding text and their background knowledge. The current study aims to contrast text- and knowledge-based monitoring to investigate their unique influences on processing and whether validation is passive or reader-initiated. Therefore, we collected reading times in a self-paced experiment using expository texts containing information that conflicts with either the preceding text or readers’ background knowledge. Results show that text- and knowledge-based monitoring have different time courses and that working memory affects only knowledge-based monitoring. Furthermore, our results suggest that validation could occur at different levels of processing and perhaps draw on different mixes of passive and reader-initiated processes. These results contribute to our understanding of monitoring during reading and of how different sources of information can influence such monitoring.

Introduction

A central tenet in reading research is that readers go through a text trying to create a coherent mental representation that is continuously updated when new information is encountered (e.g., Graesser, Singer, & Trabasso, 1994; Kintsch, 1988; O’Brien & Myers, 1999; Trabasso, Secco, & Van Den Broek, 1984; Van Den Broek, Young, Tzeng, & Linderholm, 1999; Fletcher & Bloom, 1988: Kintsch & Van Dijk, 1978; Van Dijk & Kintsch, 1983; McNamara & Magliano, 2009; Van Den Broek, 2010). In a high-quality mental representation, individual elements of the text are connected to each other and to relevant background knowledge by meaningful relations. Together, these elements and relations create a “situation model,” an interpreted description of the information in the text (Kintsch, 1998; Van Dijk & Kintsch, 1983; Zwaan & Radvansky, 1998). An essential aspect of constructing connections during reading and, thereby, updating the situation model is that the reader monitors the extent to which incoming information is accurate. Written materials frequently contain inconsistencies, misinformation, or even fake news, especially today. In some cases readers even integrate and use inaccurate information when they should know that what they are reading is incorrect because they most likely have the accurate knowledge (Eslick, Fazio, & Marsh, 2011; Fazio & Marsh, 2008a, 2008b; Marsh & Fazio, 2006; Marsh, Meade, & Roediger III, 2003; Fazio, Barber, Rajaram, Ornstein, & Marsh, 2013). It is important that we understand how readers deal with such inconsistencies and, more generally, how they monitor incoming information to create a mental representation of both coherent, “error-free” texts and texts that contain inconsistencies (e.g., Albrecht & O’Brien, 1993; Rapp & Braasch, 2014; Richter & Rapp, 2014).

Knowledge- and text-based monitoring

To create a coherent and correct mental representation and to protect it against inaccuracies, readers must validate incoming information. They can monitor such information for consistency with previous text information and with their own knowledge and beliefs (Singer, 2013). Considerable evidence indicates that validation against background knowledge occurs (e.g., O’Brien & Albrecht, 1992; Rapp, 2008; Richter, Schroeder, & Wöhrmann, 2009), at various levels of language processing (i.e., sentence vs. discourse level), in different text genres (e.g., narrative and expository texts), and through various research methods, such as behavioral and electrophysiological measures. For example, readers of narrative (e.g., O’Brien & Albrecht, 1992) and expository texts (e.g., Rapp, 2008) have been found to be slower when they read target sentences that were inconsistent with their world knowledge than when they read consistent target sentences, suggesting they indeed check the incoming information against background knowledge. Similar patterns have been observed in event-related potential studies, where an increase in the N400 event-related potential signal is taken to indicate detection of inconsistency. Such increases in N400 have been observed for sentences that contain information that is inconsistent with the reader’s background knowledge. For example, compare the sentences “the Dutch trains are yellow and very crowded” and “the Dutch trains are white and very crowded.” Although both sentences are linguistically correct, the second sentence elicited a N400 event-related potential in Dutch research participants, because it is a well-known fact among Dutch people that Dutch trains are yellow, and this sentence thus was inconsistent with their background knowledge (e.g., Hagoort, Hald, Bastiaansen, & Petersson, 2004). Hence, there is ample evidence that background knowledge influences validation.

In addition to validating information against background knowledge, readers should also validate information against prior text, because specific sections of a text can be inconsistent with information provided earlier or later in that same text. Previous research has shown that contextual information (i.e., provided within a text) affects processing and comprehension of texts, for example in studies investigating lexical access processes (Colbert-Getz & Cook, 2013), anaphoric references (O’Brien, 1987; O’Brien, Albrecht, Hakala, & Rizzella, 1995; O’Brien, Plewes, & Albrecht, 1990), bridging inferences (Myers, Cook, Kambe, Mason, & O’Brien, 2000), and those based on the contradiction paradigm developed by O’Brien and colleagues (Albrecht & O’Brien, 1993; O’Brien, Cook, & Guéraud, 2010; O’Brien, Cook, & Peracchi, 2004; O’Brien, Rizzella, Albrecht, & Halleran, 1998). Indeed, if contextual information is strong (e.g., recently encountered or more elaborate), it interacts with or may even fully override the influence of world knowledge during reading (e.g., Colbert-Getz & Cook, 2013; Cook & Myers, 2004; Leinenger & Rayner, 2013; Myers et al., 2000; Nieuwland & Van Berkum, 2006; Rizzella & O’Brien, 2002). Similarly, several studies using post-reading tasks such as statement-verification tasks or general knowledge tests showed that readers defer to text information (and not world knowledge), even when they are aware the information provided by the text is incorrect (e.g., Gerrig & Prentice, 1991; Marsh et al., 2003). For example, sometimes readers gave more incorrect answers on tests of general world knowledge after reading texts that contained incorrect information than when they read texts containing correct or neutral information (Marsh et al., 2003).

However, most studies on text-based monitoring do not explicitly exclude the influence of background knowledge to investigate the influence of prior text (Isberner & Richter, 2014). Often, the impacts of these two sources on processing and comprehension are studied in tandem, and as a result the unique influence of either background knowledge or prior text remains unknown. For example, O’Brien and Albrecht (1991) presented passages in which the contexts supported an explicitly mentioned antecedent (e.g., cat) or an unmentioned concept (e.g., skunk) followed by a sentence containing an anaphoric phrase (e.g., what had run in front of her car). Then they presented naming probes for either the correct antecedent (e.g., cat), or the unnamed concept (e.g., skunk). In this example the supportive context would mention “a terrific odor,” assuming that readers are more likely to associate “a terrific odor” with “skunk” than with “cat” based on their background knowledge. They found that the unnamed concept (skunk) was activated in memory, even despite the explicit reference to the correct antecedent (cat). This is one example that provides compelling evidence that contextual information influences processing but does not explicitly exclude the influence of background knowledge.

To summarize, it is clear that both knowledge-based and text-based monitoring influence text processing. However, it is unclear how to distinguish between the two sources for validation and how to define their unique influences.

Passive or reader-initiated

A second important issue that is debated in the literature is whether validation is a passive or a reader-initiated process. There is some evidence that validation against background knowledge is an automated, routine process (e.g., Hagoort & Van Berkum, 2007; Isberner & Richter, 2013; Richter et al., 2009; Singer, 2006, 2013). For example, Singer (2006) demonstrated with a reading time paradigm that readers verify information presented in everyday stories even when they do not follow an intentional validation strategy. Thus, individuals seem to routinely validate information they encounter in a discourse context. This is in line with the assumption of memory-based text-processing views that connections between text segments and background knowledge are formed via an effortless, autonomous spread of activation through existing associations in readers’ semantic and episodic memory (e.g., Myers & O’Brien, 1998; O’Brien et al., 1995; O’Brien & Myers, 1999). This would suggest that the detection of an inconsistency is also passive. However, Singer and colleagues observed that reading tasks may influence the degree to which readers rely on story contexts and prior knowledge, respectively, and, hence, the effect of these sources on readers’ processing of texts’ (e.g., Singer, 2006; Singer & Halldorson, 1996; Singer et al., 1992; see also Van Den Broek & Helder, 2017). If the degree to which readers rely on the text or prior knowledge can be altered by the task they are given, this would suggest that validation processes—or the subsequent processes that are triggered by the validation process—are not completely passive and might be at least partially reader-initiated. The above illustrates that the evidence on whether validation is passive or reader-initiated is ambiguous.

Whether a process is passive or reader-initiated also influences the processing capacity that is required to run it to completion. Passive processes are fast and relatively effortless and require fewer cognitive resources than reader-initiated processes do. Thus, it could be that if validation is a reader-initiated process, individual differences in processing capacity, for example in working memory, influence the validation process. If validation is a reader-initiated process, then perhaps working memory capacity serves as a bottleneck for the processing of inconsistencies. To illustrate how this could be the case, it is important to consider how models of reading comprehension define validation. They state that validation is one of three prominent processes that are active during reading, namely activation, integration, and validation. First, relevant concepts from memory are activated (activation), then the available concepts are connected with the content of working memory (integration), and finally the connections that are formed between concepts are validated to ensure they make sense (validation) (Cook & O’Brien, 2014; O’Brien & Cook, 2016). Thus, to successfully validate the connections both the newly read information and the relevant information from either background knowledge or the previous text have to be active and available in working memory. It is crucial for the detection of inconsistencies that the two are not only active but are co-activated in memory (Van Den Broek & Kendeou, 2008). Consequently, working memory could play a role in this co-activation (e.g., Hannon & Daneman, 2001; Singer, 2006) and, because its capacity is assumed to be limited (Miller, 1956; Simon, 1974), could serve as a bottleneck for the processing of inconsistencies.

Current study

Our discussion indicates that information from both background knowledge and the text can influence the validation process. However, to the best of our knowledge, prior studies have either not made a direct comparison between these two types of validation or, if they did include both text and background knowledge in their design, did not report on both types of validation. As a result, it is unclear whether the mechanisms of validating against background knowledge are the same as those validating against prior text or whether these mechanisms are fundamentally different. In the current study we aimed to tease apart these two types of validation to investigate their unique influence on processing by directly comparing them in a within-subjects design. Furthermore, we aimed to shed light on whether validation is passive or reader-initiated and elucidate the role of working memory during knowledge-based and text-based monitoring. We did so in a self-paced reading experiment in which the participants read expository texts about historical topics. The situations described in the texts either supported or called into question actual historical events. Half of the texts continued with information that was incongruent with the text or inconsistent with their background knowledge. The texts were presented sentence-by-sentence and reading times recorded, providing a measure of readers’ difficulty integrating statements into a discourse representation as texts unfold (e.g., Albrecht & O’Brien, 1993; Cook, Halleran, & O’Brien, 1998; Rapp, Gerrig, & Prentice, 2001; Rapp, 2008).

To investigate whether validation is a passive or reader-initiated process, we used two different approaches. First, participants received instructions during the reading task to focus them on either background knowledge or the text. If validation is a more reader-initiated process, we would expect participants to direct more resources to either validating against the text or validating against background knowledge and, hence, for the type of instruction to interact with the inconsistency effect: When the focus of the participants matches the type of inconsistency, the inconsistency effect would be larger than when the two are different. For example, when they are focused on the text, they would show a larger inconsistency effect for inconsistencies with text than for inconsistencies with background knowledge. However, if validation is a passive process, we would expect the reader to be unable to influence the validation process and their focus would not influence the inconsistency effect. Second, we consider the possible role of individual differences in working memory capacity in validation. As mentioned earlier, passive processes are seen as relatively effortless and require almost no cognitive resources, whereas reader-initiated processes are more effortful and require cognitive resources. Therefore, if working memory capacity is found to impact the processing of inconsistencies, then validation apparently requires cognitive resources and thus likely is at least in part a reader-initiated process.

Methods

Participants

Fifty-eight university students (11 men and 47 women, mean age 22.4 years) participated in this study for course credit or pay. All participants were native speakers of Dutch and had no diagnosed dyslexia and/or developmental disorders such as attention deficit hyperactivity disorder or autism-spectrum disorders. They also had normal or corrected-to-normal eyesight. Participants provided written informed consent before testing, and all procedures were approved by the Leiden University Institute of Education and Child Studies ethics committee and conducted in accordance with the Declaration of Helsinki.

Materials

Norming study

The texts used in this experiment were based on materials used by Rapp (2008). Each text contained a target statement that is either correct or incorrect with background knowledge and a context (the sentences preceding the target) that either supports or calls into question the information in the target. The texts were about historical topics that are well known to readers. To make sure that the facts described in the texts were common knowledge in our sample, we conducted a norming study among 30 native Dutch participants (8 men and 22 women, mean age 22.5 years). In the norming study 18 topics from the Rapp (2008) stimuli were translated and used. In addition, we wrote texts about 80 new topics to make them more suitable for the present sample of participants (e.g., a text about Babe Ruth was replaced by a text about Johan Cruyff, a famous Dutch soccer player). The participants read either the historically correct or the historically incorrect target sentence for the 98 topics and indicated (1) whether the sentence was true or false and (2) how sure they were of their answer on a visual analog scale ranging from “absolutely not sure” to “absolutely sure.” The certainty scores were calculated as a percentage of the visual analog scale line. To determine familiarity with the items, a threshold of 70% of participants selecting the correct answer was used as an indicator of high familiarity (e.g., Marsh & Fazio, 2006; Marsh et al., 2003). After eliminating the unfamiliar topics, a final sample of 80 topics remained. For the 80 items used in the main study, on average 87% of the participants answered them correctly (SD = 9.9) and were 77% certain of their answer (SD = 11.4).

Experimental texts

Four different versions of each of the 80 texts were constructed by orthogonally varying the context before the target sentence (i.e., congruent vs. incongruent with target sentence) and the target sentence itself (i.e., correct vs incorrect with the readers’ background knowledge). The context could either bias toward the correct or the incorrect target, making the context either congruent (i.e., bias-correct context paired with a correct target sentence or bias-incorrect context paired with an incorrect target sentence) or incongruent (i.e., bias-correct context paired with an incorrect target sentence or bias-incorrect context paired with a correct target sentence) with the target sentence. It should be noted that the contexts that were biased toward an incorrect target sentence did not state with certainty that the historical events (stated in the target sentence) would not occur or were impossible; rather, they called into question the certainty of those events. To make sure there were no inconsistencies with background knowledge before the presentation of the target sentence, all facts described in the context sentences were historically correct.

Each text consisted of 10 sentences (see Table 1 for a sample text). The first two sentences were identical among all conditions, providing an introduction to the topic. The next five sentences (Sentences 3–7) of the texts differed in content, depending on context condition (congruent/incongruent). On average, the bias-correct context consisted of 64 words (SD = 4.20) and 399 characters (SD = 23.48), and the bias-incorrect context consisted of 66 words (SD = 4.30) and 407 characters (SD = 22.79). The eighth sentence of the text was the target and provided one of two targets, depending on target condition (correct/incorrect). The final two sentences were identical among all conditions, providing a conclusion to the topic. On average, the texts contained 121 words (SD = 5.66) and 766 characters per text (SD = 37.63), across all four text versions.

Table 1. Sample text with the four text versions (translated from Dutch original).

		Background Knowledge
		Target correct	Target incorrect
Text	Target congruent with context	[Introduction]	[Introduction]
		In 1865, a Frenchman named Laboulaye wished to honor democratic progress in the U.S.	In 1865, a Frenchman named Laboulaye wished to honor democratic progress in the U.S.
		He conceptualized a giant sculpture along with artist Auguste Bartholdi.	He conceptualized a giant sculpture along with artist Auguste Bartholdi.
		[Context Bias Correct]	[Context Bias Incorrect]
		Their ‘Statue of Liberty’ would require extensive fundraising work.	Their ‘Statue of Liberty’ would require extensive fundraising work.
		They organized a public lottery to generate support for the sculpture.	Raising the exorbitant funds for the statue proved an enormous challenge.
		American businessmen also contributed money to build the statue’s base.	Because of financial difficulties France could not afford to make a gift of the statue.
		Despite falling behind schedule, the statue was completed.	Fundraising was arduous and plans quickly fell behind schedule.
		The statue’s base was finished as well and ready for mounting.	Because of these problems, completion of the statue seemed doomed to failure.
		[Target Correct]	[Target Incorrect]
		The Statue of Liberty was delivered from France to the United States.	The Statue of Liberty was not delivered from France to the United States.
		[Coda]	[Coda]
		The intended site of the statue was a port in New York harbor.	The intended site of the statue was a port in New York harbor.
		This location functioned as the first stop for many immigrants coming to the U.S	This location functioned as the first stop for many immigrants coming to the U.S
	Target incongruent with context	[Introduction]	[Introduction]
		In 1865, a Frenchman named Laboulaye wished to honor democratic progress in the U.S.	In 1865, a Frenchman named Laboulaye wished to honor democratic progress in the U.S.
		He conceptualized a giant sculpture along with artist Auguste Bartholdi.	He conceptualized a giant sculpture along with artist Auguste Bartholdi.
		[Context Bias Incorrect]	[Context Bias Correct]
		Their ‘Statue of Liberty’ would require extensive fundraising work.	Their ‘Statue of Liberty’ would require extensive fundraising work.
		Raising the exorbitant funds for the statue proved an enormous challenge.	They organized a public lottery to generate support for the sculpture.
		Because of financial difficulties France could not afford to make a gift of the statue.	American businessmen also contributed money to build the statue’s base.
		Fundraising was arduous and plans quickly fell behind schedule.	Despite falling behind schedule, the statue was completed.
		Because of these problems, completion of the statue seemed doomed to failure.	The statue’s base was finished as well and ready for mounting.
		[Target Correct]	[Target Incorrect]
		The Statue of Liberty was delivered from France to the United States.	The Statue of Liberty was not delivered from France to the United States.
		[Coda]	[Coda]
		The intended site of the statue was a port in New York harbor.	The intended site of the statue was a port in New York harbor.
		This location functioned as the first stop for many immigrants coming to the U.S	This location functioned as the first stop for many immigrants coming to the U.S

Overall, the target sentences were equated for length: both correct (SD = 1.92) and incorrect (SD = 1.90) targets contained on average nine words. When measured by number of characters (including spaces and punctuation), both correct (SD = 10.51) and incorrect (SD = 10.42) targets contained on average 60 characters. Half of the correct targets and incorrect targets included the word “not” or “never” (e.g., “Jack the Ripper was never caught and punished for his crimes.”) and half did not (e.g., “The Titanic withstood the damage from the iceberg collision.”).

Apparatus

Reading task

Participants read the texts on a computer screen. The texts were presented one sentence at a time, and reading times were recorded when participants pressed a key to advance to the next sentence. To implement a repeated measures design we used a Latin square to construct four lists, with each of the 80 texts appearing in a different version (as a function of text context, congruent with target vs. incongruent with target, and target, correct vs. incorrect) on each list. Each participant received one list, and hence read one version of each text. The texts were divided into two blocks of 40, and the order in which they were presented was randomized. Participants received instructions at the start of each block to focus them on either their background knowledge or the text. The order of the blocks was counterbalanced; participants could either receive the focus on text instruction in the first block and the focus on background instruction in the second block or vice versa. Focus on background knowledge was promoted by the instruction that they had to write down one thing they knew about the topic before reading each text. Focus on the text was promoted by the instruction to think of a short summary that reflects the content of the text best at the start of the block. After reading each text participants had to write down their summary. Halfway through a block they were able to take a short break. Each block of the experimental task was preceded by a short practice block of two texts to familiarize participants with the task. The structure of the practice texts mirrored the structure of the experimental items.

Measures

Working memory capacity

Working memory capacity was measured by means of the Swanson Sentence Span task (Swanson, Cochran, & Ewers, 1989). In this task the experimenter reads out sets of sentences, with set length varying from one to six sentences. At the end of each set a comprehension question is asked about one of the sentences in the set. Participants have to remember the last word of each sentence and recall these after answering the comprehension question. Demands on working memory vary because sets consist of two, three, four, five, or six sentences, with two sets at each working memory load. If participants successfully complete both tasks (recall of final words and correctly answer the comprehension question) for at least one of the two sets at a particular load, they advance to the next higher load. Participants earned 0.25 points for each correctly answered comprehension question or correctly recalled set of words, and the sum of these points is the index of working-memory capacity.

Reading skill

Although the sample in this study constituted of a relatively homogeneous group of skilled readers, we used the scores for a modified cloze task (CBM Maze Task) to control for the influence of overall reading proficiency (Deno, 1985; Espin & Foegen, 1996; Fuchs & Fuchs, 1992). In this task participants read two texts. For each text every seventh word is deleted and replaced with three options from which to choose. As participants proceed through the texts they choose the word that best completes the text as they read. They are given 90 seconds to provide as many correct answers as possible. Accuracy scores are averaged across the two texts.

Procedure

Participants were tested individually. First, they completed the experimental reading task. Participants were asked to read each sentence at their own pace. They started with a practice block, followed by one experimental block. After completing the first experimental block participants could take a 5- to 7-minute break before they started the second experimental block (with a different focus instruction). Halfway through each experimental block participants had the opportunity to take a short break if needed. After they finished the experimental reading task they completed the Swanson Sentence Span Task and the CBM Maze Task. The total duration of the experimental procedure was approximately 90 minutes.

Design

In the current study the factors text (target congruent or incongruent with context), background knowledge (target correct or incorrect), and focus (focus on text or background knowledge) varied as within-participants. The factor text was defined as congruent or incongruent based on the combination of context and target sentence (e.g., context bias correct followed by a correct target sentence is congruent and context bias correct followed by an incorrect target sentence is incongruent). The factor background knowledge was defined as correct or incorrect with the readers’ background knowledge and the factor focus consisted of two levels (focus on text or focus on background knowledge) depending on the instructions the participants received at the start of each block. The variables reading ability and working memory capacity were between-subjects variables.

Results

To investigate whether the participants completed the tasks as instructed, we analyzed their responses. Results for trials where participants had been instructed to focus on background knowledge (by writing down one thing they already knew about the topic) show that they were proficient in activating background knowledge: Participants responded with background information in 94% of the trials. Likewise, analysis of participants’ summaries for the trials where they had been instructed to focus on the text (by thinking of a summary and writing it down) revealed they completed the task as requested: They responded in 99.74% of the trials with a summary, with an average summary length of 133 characters (SD = 55).

To investigate the effects of our manipulations on the reading process, we analyzed the reading times for four regions of interest: the pretarget (sentence preceding the target sentence), the target sentence, the spill-over sentence (sentence following the target sentence), and the final sentence. Sentence reading times that were extremely short (shorter than 300 ms) or extremely long (longer than 10,000 ms) were excluded from the analyses, resulting in a loss of 1% of the data. Table 2 reports the means and standard deviations of the resulting data for reading times as a function of focus (focus background knowledge/focus text), text (congruent/incongruent with target sentence), background knowledge (correct/incorrect), and region (pretarget, target, spill-over and final).

Table 2. Mean reading times and standard deviations (in ms) at the regions of interest (pretarget, target, spill-over, and final sentence) for the experimental manipulations focus (focus on text or background knowledge), text (context consistent or inconsistent with target sentence), and background knowledge (target sentence correct or incorrect).

			Pretarget		Target		Spill-Over		Final
Focus	Text	Background	M	SD	M	SD	M	SD	M	SD
Focus backgroundknowledge	Consistent	Correct	2261	1289	1908	1043	2526	1743	2417	1497
		Incorrect	2347	1292	2104	1187	2565	1417	2457	1755
	Inconsistent	Correct	2359	1278	2026	1087	2441	1371	2475	2173
		Incorrect	2348	1282	2276	1317	2702	1670	2491	1833
Focus text	Consistent	Correct	2581	1312	2093	1023	2783	1443	2619	1601
		Incorrect	2589	1209	2373	1239	2989	1620	2668	1780
	Inconsistent	Correct	2548	1209	2317	1031	2837	1502	2634	1600
		Incorrect	2539	1242	2626	1453	2937	1510	2811	2002

First, we fitted a mixed-effects linear regression model to determine whether the experimental manipulations and their interactions were significant factors in the each of the regions of interest. Models were estimated with the R package LME4 version 1.1–12 (Bates, Maechler, Bolker, & Walker, 2015) and for all models Wald chi-square testing (Type II), as implemented in the R-package Car (version 2.1–4; Fox & Weisberg, 2011), was applied to select the most parsimonious structure of fixed effects by removing nonsignificant (p > .05) predictors. We considered the reading times (log transformed to correct for right skewness) for all regions together, with the factors region, background knowledge, text, and focus and the interaction of these factors. The model also included the maximal converging random structure for subjects and items. The model showed that the experimental manipulations were significant factors in the regions of interest. The results of the Wald chi-square tests revealed significant main effects of region (χ²(3) = 511.14, p < .001), focus (χ²(1) = 514.43, p < .001), background knowledge (χ²(1) = 52.03, p < .001), and text (χ²(1) = 15.31, p < .001) on the reading times. Moreover, both the interaction between region and text (χ²(3) = 26.928, p < .001) and the interaction between region and background knowledge (χ²(3) = 37.07, p < .001) were significant.

To further investigate the effects of the experimental manipulations, we conducted mixed-effects linear regression analyses for each region of interests separately. For each region of interest we started with a model that included the fixed factors background knowledge (target correct/target incorrect), text (congruent/incongruent), and focus (focus background knowledge/focus text) and the full interactional terms for these factors. Participants and items were included as crossed random effects. Next, we included reading ability and working memory capacity (both measures were median-centered) using a forward stepwise selection procedure (Viebahn, Ernestus, & McQueen, 2012), comparing models with and without each particular characteristic. Then we again selected the most parsimonious model as our final model. The final model included the maximal participant and item random-effect structure that resulted in a converging model (Barr, Levy, Scheepers, & Tily, 2013). For each region of interest we only report the final model. Furthermore, because it is not clear how to determine the degrees of freedom for the t-statistics estimated by the mixed models for continuous dependent variables (Baayen, 2008), we do not report degrees of freedom and p values for the fixed estimates. Instead, statistical significance at approximately the .05 level is indicated by values of the t-statistics ≥ 1.96 (see e.g., Schotter, Tran, & Rayner, 2014). We report both the Wald tests and the estimates of the fixed effects for all models. Unless mentioned otherwise, we only discuss effects that were significant (p < .05) in the Wald tests and the tests of the fixed estimates (t > 1.96 or t < −1.96), and for reading ability and working memory capacity we only discuss significant interactions with the fixed factors in our design (background knowledge, text, and focus).

Target sentence

Table 3 reports the results of the Wald tests as well as of the tests of the estimates of the fixed effects of the final model for the log transformed reading times on the target sentence. The results revealed significant main effects for both text (β = .08, SE = .01, t = 7.34) and background knowledge (β = .10, SE = .02, t = 6.47). The reading times for incorrect (M = 2,345 ms) or incongruent targets (M = 2,311 ms) were longer than the reading times for correct (M = 2,085 ms) or congruent targets (M = 2,119 ms), both for inconsistencies with background knowledge and for incongruences with prior text. Furthermore, we observed a main effect of focus (β = .15, SE = .04, t = 3.64): Participants read more slowly when they focused on the text (M = 2,352 ms) than when they focused on their background knowledge (M = 2,078 ms). In addition, there was a main effect of reading ability (β = −.02, SE = .01, t = −4.37), with skilled readers reading faster than less skilled readers. Finally, there was a significant interaction between working memory capacity and background knowledge (β = −.03, SE = .01, t = −2.32): Participants with a larger working memory capacity showed a smaller inconsistency effect than did participants with a smaller working memory capacity.

Table 3. Wald tests and estimates of fixed effects of the final model including maximum random slopes for the log transformed reading times on the target sentence. The following R code was used: TZ.RT_Log ~ 1 + focus * working memory capacity + text + background knowledge * working memory capacity + reading ability + (1 + focus + text + background | subject) + (1 + focus + text + background | item).

Wald tests	χ^2	Df	P
Focus	13.657	1	<0.001
Text	53.916	1	<0.001
Background	40.755	1	<0.001
Working memory	0.782	1	0.377
Reading ability	19.065	1	<0.001
Focus * working memory	1.050	1	0.305
Working memory * background	5.386	1	0.020
Estimates of the fixed effects	B	SE	T
Intercept	7.421	0.045	165.56
Focus	0.148	0.041	3.64
Text	0.077	0.011	7.34
Background	0.098	0.015	6.47
Working memory	−0.051	0.051	−1.01
Reading ability	−0.019	0.004	−4.37
Focus * working memory	0.050	0.049	1.02
Working memory * background	−0.029	0.012	−2.32

Spill-over sentence

Table 4 reports the results of the Wald tests as well as of the tests of the estimates of the fixed effects of the final model for the log transformed reading times on the spill-over sentence. For the reading times of the spill-over region there was a main effect of background knowledge (β = .056, SE = .013, t = 4.21), indicating a spill-over effect of background knowledge: When the preceding target sentence was incorrect (M = 2,799 ms) participants were slower on the spill-over sentence than when the preceding target sentence was correct (M = 2,647 ms). Furthermore, there was a main effect of focus (β = .12, SE = .05, t = 2.74): When participants focused on the text (M = 2,886 ms) they read more slowly than when they focused on background knowledge (M = 2,558 ms). Finally, there was an interaction of reading ability and working memory capacity (β = −.02, SE = .01, t = −2.23). However, because this interaction was of secondary interest and, moreover, not reliable in the Wald tests (χ²(1) = 1.71, p = .19), we refrain from further discussing this result.

Table 4. Wald tests and estimates of fixed effects of the final model including maximum random slopes for the log transformed reading times on the spill-over sentence. The following R code was used: Z9.RT_Log ~ 1 + focus × reading ability *; working memory capacity + background + (1 + focus *; background |subject) + (1 + focus * background |item).

Wald tests	χ^2	Df	P
Focus	11.60	1	<0.001
Reading ability	18.15	1	<0.001
Working memory	0.93	1	0.334
Background	17.76	1	<0.001
Focus * reading ability	1.71	1	0.191
Focus * working memory	0.10	1	0.754
Reading ability * working memory	1.70	1	0.192
Focus * reading ability * working memory	3.26	1	0.071
Estimates of the fixed effects	B	SE	T
Intercept	7.704	0.051	150.56
Focus	0.123	0.045	2.74
Reading ability	−0.012	0.008	−1.36
Working memory	−0.032	0.060	−0.54
Background	0.056	0.013	4.21
Focus * reading ability	−0.010	0.008	−1.32
Focus * working memory	0.005	0.055	0.09
Reading ability * working memory	−0.021	0.009	−2.23
Focus * reading ability * working memory	0.016	0.009	1.81

Taken together, the results show a main effect of background knowledge on both target and spill-over sentences, with participants reading both target and spill-over sentences more slowly when the target sentence was incorrect. For the target sentence this effect depended on participants’ working memory capacity as indicated by a significant interaction: Participants with a larger working memory capacity showed a smaller inconsistency effect than did participants with a smaller working memory capacity. With respect to the congruency of target and context, a main effect of text was observed for the target sentence but not the spill-over sentence: Participants were slower to read target sentences that were incongruent with the preceding text than target sentences that were congruent. On both the target and spill-over sentences we found a main effect of focus, indicating that participants were generally slower when they were instructed to focus on the text. We did not find any interaction effects between text and background knowledge on the target or the spill-over sentence.

Precritical sentence and final sentence

Tables 5 and 6 report the results of the Wald tests as well as of the tests of the estimates of the fixed effects of the final model for the log transformed reading times on the precritical sentence (Table 4) and the final sentence (Table 5). Both models include maximum random slopes. We found a main effect of focus on both the precritical sentence (β = .11, SE = .04, t = 2.62) and the final sentence (β = .09, SE = .04, t = 2.16): When participants were focused on the text (precritical: M = 2,546 ms, final: M = 2,683 ms) they read more slowly than when they were focused on background knowledge (precritical: M = 2,307 ms, final: M = 2,460 ms) for both sentences. Also, the main effect of reading ability was significant in the Wald tests for both the precritical (χ²(1) = 17.18, p < .001) and the final sentences (χ²(1) = 16.54, p < .001), indicating that a higher reading ability was associated with faster reading times. However, the estimate of the fixed effect for the predictor reading ability did not reach significance for both the precritical (β = −.02, SE = .01, t = −1.74) and the final sentence (β = −.01, SE = .01, t = −1.47), indicating that the results for this predictor have to be interpreted with caution. There were no effects of source (background knowledge or text) on the reading times for either region. Thus, we did not find any effects of the text and background knowledge manipulations on the pretarget or the final sentence, indicating that the manipulations did not have an effect before the presentation of the target sentence and that the spill-over effect of background knowledge was no longer present on the final sentence.

Table 5. Wald tests and estimates of fixed effects for the final model including maximum random slopes for the log transformed reading times on the pre-critical sentence. The following R code was used: Z7.RT_Log ~ 1 + focus × reading ability * working memory capacity + (1 + focus | subject) + (1 + focus | item).

Wald tests	χ^2	Df	P
Focus	10.999	1	<0.001
Reading ability	17.178	1	<0.001
Working memory	0.005	1	0.942
Focus * reading ability	1.413	1	0.235
Focus * working memory	2.131	1	0.144
Reading ability * working memory	0.565	1	0.452
Focus * reading ability * working memory	2.789	1	0.095
Estimated of the fixed effects	B	SE	T
Intercept	7.642	0.050	152.10
Focus	0.107	0.041	2.62
Reading ability	−0.015	0.008	−1.74
Working memory	−0.050	0.059	−0.85
Focus * reading ability	−0.009	0.007	−1.20
Focus * working memory	0.063	0.051	1.24
Reading ability * working memory	−0.017	0.009	−1.76
Focus * reading ability * working memory	0.013	0.008	1.67

Table 6. Wald tests and estimates of fixed effects for the final model including maximum random slopes for the log transformed reading times on the final sentence. The following R code was used: Z10.RT_Log ~ 1 + focus * reading ability * working memory capacity + (1 + focus |subject) + (1 + focus |item).

Wald tests	χ^2	Df	P
Focus	7.481	1	0.006
Reading ability	16.535	1	<0.001
Working memory	0.4532	1	0.465
Focus * reading ability	1.452	1	0.228
Focus * working memory	0.193	1	0.660
Reading ability * working memory	1.742	1	0.187
Focus * reading ability * working memory	2.265	1	0.132
Estimates of the fixed effects	B	SE	T
Intercept	7.665	0.053	144.19
Focus	0.094	0.043	2.16
Reading ability	−0.013	0.009	−1.47
Working memory	−0.033	0.062	−0.54
Focus * reading ability	−0.009	0.008	−1.22
Focus * working memory	0.014	0.054	0.25
Reading ability * working memory	−0.019	0.010	−2.00
Focus * reading ability * working memory	0.013	0.009	1.50

Discussion

In the present study we aimed to contrast validation against background knowledge (knowledge-based monitoring) and validation against prior text (text-based monitoring) to investigate their unique influences on processing. Additionally, we wanted to shed light on whether validation is a passive or a reader-initiated process and elucidate the possible role of working memory. First, in line with previous studies, participants took longer to read both the inconsistent and the incongruent targets (e.g., Albrecht & O’Brien, 1993; O’Brien et al., 1998; Rapp, 2008). Both prior text and background knowledge influenced readers’ moment-by-moment processing: Both types of inconsistencies elicited an effect on the target sentence. However, only inconsistencies with background knowledge elicited a spill-over effect on the next sentence. Thus, it seems that text-based monitoring and knowledge-based monitoring show distinct time courses, which suggests processing differences. Second, we investigated whether validation is a passive or reader-initiated process by considering both the influence of the task and the role of working memory. We examined the influence of the task by manipulating the focus of the readers by instructing them to focus on either their background knowledge or the text. The task influenced the reading process of the text as a whole (i.e., a main effect of focus: Readers were slower when instructed to focus on the text than when they were instructed to focus on background knowledge) but did not influence the validation process (i.e., there was no interaction between reading focus and inconsistency). Furthermore, we investigated the role of working memory in validation and observed that working memory influences processing of inconsistencies with background knowledge but not processing of inconsistencies with prior text.

The results show that both background knowledge and prior text have an influence on readers’ moment-by-moment processing. Moreover, they suggest distinct time courses for validation against background knowledge and prior text, because only inconsistencies with background knowledge elicit a spill-over effect. Similar results have been found by Rapp (2008) in a series of experiments where he used texts with familiar or unfamiliar topics to make the readers’ prior knowledge relevant or irrelevant. In two of these experiments he used familiar topics and in the third experiment unfamiliar topics, thereby minimizing—but not eliminating—the influence of background knowledge. Rapp found that minimizing the influence of background knowledge (more text-based monitoring) resulted in main effects of the inconsistencies only on the targets but no spill-over effects. In contrast, when prior knowledge was made relevant (more knowledge-based monitoring), he did observe spill-over effects. Although the aim of this study was not to compare the two types of monitoring, the pattern of spill-over effects when knowledge-based monitoring was important and no spill-over effects for text-based monitoring can be interpreted in light of the current study. In our study we made a direct comparison between the two types within a single design to investigate their differences. The results showed that indeed text-based and knowledge-based monitoring have different time courses and that inconsistencies with background knowledge seem to have a different -and perhaps larger- impact on processing than those with prior text.

There are several possible explanations for the time-course differences between text-based and knowledge-based monitoring. First, the time course differences may reflect differences in the possible repair processes that follow the validation process. It could be that the repair processes are different for inconsistencies with background knowledge than for inconsistencies with prior text. For example, if it is more difficult to repair an inconsistency with the more extensive background knowledge than one with a text representation, then this would lead to the observed longer reading times.

Second, the differences may be related to how easily relevant information for validation is accessed and activated. Earlier two-stage models of comprehension (Kintsch, 1988, 1998; Long & Lea, 2005; Rizzella & O’Brien, 1996, 2002; Sanford & Garrod, 1989) assume that comprehension consists of two stages: activation and integration. More recent models, such as the RI-Val model of reading (Cook & O’Brien, 2014; O’Brien & Cook, 2016), have included validation as an additional stage of comprehension. All aforementioned models assume that information needs to be activated before it can be integrated or validated. Relevant concepts are activated through spread of activation (e.g., Anderson, 1983): A passive and unrestricted process that continues until the system reaches an equilibrium (Kintsch, 1988). Presumably, the memory representation of background knowledge consists of a much larger and therefore richer network of possibly relevant concepts (especially for well-known topics) than the memory representation of the text (especially for relatively short texts). Because the memory representation of background knowledge consists of a much larger and richer network, it may be that the onset of knowledge-based validation is later because it takes longer for the system to reach an equilibrium during the activation and integration stages, resulting in spill-over effects for inconsistencies with background knowledge.

A third, related, possibility is that the time course differences are caused by differences in the duration of the validation processes. Because the memory representation of background knowledge consist of a larger and richer network, it seems likely that after the initial activation and integration stages more concepts are available as input for the validation process than for the memory representation of the text. Validation may take longer because there simply are more possible concepts against which the new information has to be validated, thus causing knowledge-based validation to take longer.

Common to these three explanations for the time-course differences between knowledge- and text-based monitoring is that the former requires more processing resources than the latter, either because the repair process are more demanding or because the amount of information that needs to be validated is larger. This interpretation of the results is supported by the observation that working memory capacity only played a role when readers validated inconsistencies with background knowledge. Interestingly, a larger working memory capacity decreased, but did not eliminate, the inconsistency effect for the target sentences. Although a smaller inconsistency effect could be taken to reflect less or even inferior validation, this interpretation seems unlikely in the current situation assuming that high working capacity readers are proficient validators. Hence, a more plausible explanation of the attenuation of the inconsistency effect for high working memory capacity readers is that they either validated the inconsistent information more quickly or that they repaired their mental representation more efficiently. Furthermore, our observation that working memory capacity only played a role on the target sentence and not on the spill-over sentence could indicate that its influence occurs relatively early during processing (i.e., when readers integrate and validate the information) rather than at later processing (i.e., repair processes that follow validation). However, given the temporal resolution of the current design, this account is speculative. Thus, although we can conclude that working memory capacity affects the processing of inconsistencies with background knowledge, more research is needed to clarify its specific role in validation and repair processes.

Whether validation is a passive or reader-initiated process remains an open question. Evidence for a passive account of validation is the finding that although the focus manipulation successfully altered the reading process as a whole (i.e., a main effect of focus), it did not influence the validation process (i.e., no interaction between focus and the type of inconsistency). This suggests that the validation process is not influenced by reading task and thus is a passive process, which would be in line with previous studies (e.g., Hagoort & Van Berkum, 2007; Isberner & Richter, 2013; Richter et al., 2009; Singer, 2006, 2013). Of course, one should be cautious interpreting this null effect: It also is possible that the specific reading tasks used in the present study were unable to influence the validation process. Evidence for a reader-initiated account of validation comes from the finding that working memory plays a role in the processing of inconsistencies with background knowledge but not prior text, This suggests that validation, at least against background knowledge, is not entirely effortless and thus not entirely passive.

The interpretation of these results depends on how one conceptualizes passive and reader-initiated processes. Passive processes generally are conceptualized as outside the reader’s conscious control, nonselective, and unrestricted in the kind of information they return (e.g., Anderson, 1983; McKoon & Ratcliff, 1992; Myers & O’Brien, 1998; O’Brien & Myers, 1999), whereas reader-initiated processes are conceptualized as requiring control and attentional resources. Based on this view our results create a complicated picture, because on the one hand validation was not influenced by the reading task, but on the other hand validation against background knowledge did require cognitive resources (i.e., working memory capacity). These seemingly contradictory findings may be reconciled by a more refined conceptualization, namely that validation consists of subcomponents that operate at different levels of processing (more passive or more reader-initiated). This interpretation would be in line with the suggestion that reader-initiated processes lie on a continuum reflecting the degree to which they are constrained by the text, ranging from processes that are close to the actual text itself (almost passive) to processes that go well beyond the information in the text (more interpretive) (Van Den Broek & Helder, 2017). If processes indeed range on such a continuum our task manipulation would tap into a relatively high level of reader-initiated processes, whereas the interaction between working memory and knowledge-based monitoring would tap into processes closer to the text itself (and thus closer to passive). The fact that we did not find an effect of focus but did find an interaction with working memory capacity could mean that validation, at least against background knowledge, indeed is a reader-initiated process but at a lower level on this continuum. Future research could try to determine whether there indeed is a continuum of reader-initiated processes and, if so, where exactly on this continuum the process of validation lies and whether this is the same for text-based and knowledge-based monitoring.

In summary, we have shown that both prior text and background knowledge have a unique influence on processing and that the processing of inconsistencies against these two sources follow different time courses and therefore may involve different mechanisms. The current study has taken a first step in elucidating the processing differences between text-based and knowledge-based monitoring. Future studies should examine why these processes differ and what exactly the differences are. Another future challenge is to design experiments that allow us to pinpoint exactly when the various component processes of validation start and finish by using more fine-grained measures to obtain better insight in their time course. Our results suggest that validation could occur at different levels of processing and perhaps draw on different mixes of passive and reader-initiated processes, but they call into question whether validation categorically can be described as passive or reader-initiated. Furthermore, the results show that working memory plays a role, in particular in the processing of inconsistencies with background knowledge. This is a first step to elucidate the possible role of working memory in comprehension monitoring and validation. An interesting research avenue would be to determine the conditions under which working memory does or does not play a role, for example by varying the demands a task places on working memory and determine whether and, if so, how this influences the processing of different types of inconsistencies.

The current results contribute to our understanding of monitoring processes during reading and how different sources of information, such as text or background knowledge, can influence this process. Many theoretical models of reading comprehension (e.g., Albrecht & O’Brien, 1993; Gerrig & McKoon, 1998; Gerrig & O’Brien, 2005; Johnson-Laird, 1983; McKoon & Ratcliff, 1998; O’Brien & Albrecht, 1992; Van Dijk & Kintsch, 1983) make a distinction in the origin (i.e., the text or background knowledge) of information that is included in a mental representation. The current study shows that a similar distinction can be made with respect to the origin of information, text or background knowledge, against which incoming information is validated during reading.

Acknowledgment

We thank Nina Kratina and Danielle Schouten for their assistance in data collection.