Comprehension of Multiple Documents With Conflicting Information: A Two-Step Model of Validation

Abstract

In this article, we examine the cognitive processes that are involved when readers comprehend conflicting information in multiple texts. Starting from the notion of routine validation during comprehension, we argue that readers' prior beliefs may lead to a biased processing of conflicting information and a one-sided mental model of controversial issues (text-belief consistency effect). An important distinction is that such biases occur routinely as a by-product of basic comprehension processes. However, readers can actively engage in strategies that work against the biasing effects of prior beliefs when they possess the relevant cognitive resources and are motivated to activate them. A review of published studies that examined belief effects in multiple text comprehension supports the two-step model of validation. We discuss implications of this model for multiple text comprehension and educational practice and delineate directions for future research.

People use the World Wide Web as a source for informal learning on socioscientific issues that are debated in public (e.g., causes and consequences of global warming, risks of genetically modified food, renewable energies, or nuclear power), medical topics (e.g., health risks of cell phones, cholesterol, or vaccination), or political issues (e.g., abortion or Persian Gulf War). More often than not, readers are confronted with documents that provide partial and one-sided information supporting only the authors' position on the controversy. Successful comprehension of multiple documents requires readers to integrate divergent perspectives into one coherent mental model of the controversial issue (Perfetti, Rouet, & Britt, 1999). One core aspect of this requirement is that readers need to evaluate the plausibility of the arguments presented in the various texts (Mason, Boldrin, & Ariasi, 2010). Prior research indicates that readers often have difficulties in dealing with this requirement (Rouet, 2006). In particular, they fail to consider and integrate information from alternative positions into their mental model, leading to a one-sided representation (Britt, Perfetti, Sandak, & Rouet, 1999). One explanation for failures to construct a balanced representation of controversial issues is that readers often endorse one argumentative position in a scientific or political dispute. Previous research suggests that prior beliefs impact readers' understanding and retention of controversial issues in such a way that belief-consistent information has a processing advantage over belief-inconsistent information in comprehension and memory (text-belief consistency effect, e.g., Eagly & Chaiken, 1993; Knobloch-Westerwick & Meng, 2011; Maier & Richter, 2013; Wiley, 2005).

In this article, we suggest that readers' prior beliefs affect multiple documents processing (partly) as a consequence of routine cognitive processes that continuously monitor incoming information for its consistency with currently active knowledge and beliefs, as well as currently active information from previously read text(s). In the text comprehension literature, these processes have been termed validation (Richter, 2015; Singer, 2013) or epistemic monitoring (Isberner & Richter, 2014a). The basic idea, which is elaborated in the next section, is that such validation processes are a regular component of comprehending texts. When comprehending multiple documents with conflicting information, this assumption implies that prior beliefs can have effects on comprehension and memory even when readers are not motivated in any particular way to defend their beliefs. After discussing the relationship between comprehension and validation, we present a simple two-step model of processing conflicting information in multiple documents (Richter, 2011). According to this model, the first step of routine validation may lead to the detection of text-belief inconsistencies, which in essence affects memory and comprehension in terms of a text-belief consistency effect. However, in a second step, the extent that readers engage in the strategic, resource-intensive elaboration of belief-inconsistent information, which optimally results in a balanced mental model of the controversial issue, depends on their goals and motivation. This simple theoretical model is consistent with the available empirical evidence from published studies that we review in this article.

VALIDATION IN TEXT COMPREHENSION

Traditionally, the comprehension of text information and the assessment of its plausibility or consistency have been conceptualized as separate stages of processing. In this view, a comprehension stage may be followed by an evaluation phase of the comprehended information, which is validated for further processing (Connell & Keane, 2006; Gilbert, 1991). However, a growing body of research suggests that this view may be too simplistic, because the comprehension and the evaluation of information are more closely interwoven than traditionally assumed. Readers use their knowledge and beliefs not only for interpreting and encoding text information but also for monitoring its validity. This process, which has been called epistemic monitoring (Isberner & Richter, 2014a) or validation (Richter, 2015; Singer, 2013), seems to be a routine part of comprehension activities. The modifier “epistemic” in epistemic monitoring clarifies that this kind of monitoring refers to the epistemic status of information (in contrast to the kind of comprehension monitoring that has been the focus of metacomprehension research; Baker, 1979).

The general assumption of routine epistemic monitoring during comprehension is supported by reading time and eye-tracking studies, demonstrating that readers tacitly verify incoming information even when they are not given a reading goal that involves scrutinizing the consistency or plausibility of this information. For example, Singer (2006) designed short stories that contained a target sentence that could be either consistent or inconsistent with the story context and general world knowledge. Participants' reading times were prolonged for inconsistent target sentences. Many experiments, based on an inconsistency paradigm of this type, consistently yielded similar results (e.g., Albrecht & O'Brien, 1993). This research also highlights the following two principles that guide the detection of inconsistent or implausible information during comprehension.

First, conflicting information must be coactivated in working memory to detect the inconsistency (van den Broek & Kendeou, 2008). Knowledge activation during comprehension is largely a passive, memory-based process. That is, knowledge is passively triggered by concepts and propositions in a text (e.g., Albrecht & O'Brien, 1993; O'Brien & Myers, 1999) but sometimes also through active retrieval processes. Knowledge and beliefs not activated cannot be used for validating the current text information. This may occur, for example, when two pieces of conflicting information are placed too far apart in a text, when they are only loosely connected in the text, or when the current text contains insufficiently strong cues to activate the information in long-term memory (e.g., in semantic illusions such as the Moses illusion; Erickson & Mattson, 1981). As a result, inconsistent or implausible information may not be detected (e.g., Otero & Kintsch, 1992).

Second, the likelihood and time course of implausible information detection seems to depend on the degree of plausibility. An eye-tracking experiment by Staub, Rayner, Pollatsek, Hyönä, and Majewski (2007) illustrated this principle. These authors used sentences with noun–noun compounds (e.g., cafeteria manager) as experimental material. The first noun of the compounds was either a plausible (e.g., the new principle visited the cafeteria …) or an implausible (e.g., the new principle talked to the cafeteria …) continuation of the sentence, creating a local implausibility at the first noun in the implausible version, that was immediately resolved when readers encountered the second noun of the compound (e.g., manager). Staub et al. found longer first fixations on the first noun in implausible sentences, suggesting early and immediate validation processes during comprehension (for similar eye-tracking results, see Matsuki et al., 2011, Experiment 3; Patson & Warren, 2010, Experiment 2). Important to note, the more implausible the sentence was rated in an off-line plausibility judgment, the larger the effect. The detection of relatively weak implausibilities may even be delayed and show up only in integrative processes, especially when no strong cues are available to activate validity-relevant knowledge and beliefs (Patson & Warren, 2010, Experiment 1). Cook and O'Brien (2014) found similar results using the inconsistency paradigm and self-paced reading. Participants in this experiment read a short story (e.g., on Mary being a vegetarian) followed by a consistent, a high-inconsistent (e.g., Mary ordered a cheeseburger), or a low-inconsistent (e.g., Mary ordered a tuna salad) target sentence. For both inconsistent target sentences, reading was disrupted, but the time point of this disruption varied. At the target sentence, longer reading times were found for the high-inconsistent condition indicating an immediate disruption in reading for high implausibility. However, one sentence after the target sentence, longer reading times were found in the low-inconsistent condition, indicating a delayed effect of validation for mild implausibility. Thus, readers seem to sense the degree of implausibility, and validation may be delayed for information that is only weakly implausible.

Another line of research supporting the idea of routine validation during comprehension is based on the epistemic Stroop paradigm (e.g., Isberner & Richter, 2013, 2014b; Richter, Schroeder, & Wöhrmann, 2009). In this method, participants read words presented in a rapid succession on a computer screen that successively form sentences. At some point, the presentation stops and participants are prompted to make a binary judgment that is unrelated to the content of the words/sentences. For example, participants' task is to judge the spelling of words (Richter et al., 2009), whether the word appears in color rather than in black (Isberner & Richter, 2013), or they react to two probe words (TRUE or FALSE) with different keys (Isberner & Richter, 2014b). In some experiments, the experimental sentences were either true or false (Richter et al., 2009), or plausible or implausible in the context of a preceding sentence (Isberner & Richter, 2013). Across these experiments, the results showed a consistency effect between the validity of a sentence and the required response in the judgment or identification task: When the sentence was false or implausible but the required response was positive, longer response times, and in some cases also more errors, occurred (epistemic Stroop-effect: Richter et al., 2009). This pattern of results suggests that false or implausible sentences created a negative response tendency (i.e., tendency to give a NO or FALSE response) that interfered with the positive response required in the task. These epistemic Stroop experiments, therefore, go beyond the previously reviewed reading time and eye-tracking studies by showing that readers not only passively detect information that is inconsistent with their prior knowledge and beliefs but also reject this information.

An important note is that epistemic monitoring of text information during comprehension can be based not only on previously encountered information or world knowledge but also on readers' beliefs; the general mechanism of validation evaluates the consistency of text information with all information that is currently active at a given point during comprehension, regardless of its type. For example, Voss, Fincher-Kiefer, Wiley, and Silfies (1993) found that readers can evaluate argumentative claims based on their beliefs as fast as they can comprehend them (for similar results, see Wyer & Radvansky, 1999). In sum, ample evidence has shown that readers routinely monitor the consistency of text information with prior knowledge and beliefs, provided that the knowledge and beliefs are strong and pertinent enough to be activated during comprehension. Information detected as being inconsistent with a reader's knowledge and beliefs is apparently rejected. This validation of information seems to be based on passive processes that co-occur with other comprehension processes and require few cognitive resources. In the following section, we explore potential consequences of the routine and early validation of information for the mental representation constructed from multiple texts on controversial issues.

A TWO-STEP MODEL OF PROCESSING CONFLICTING INFORMATION IN MULTIPLE DOCUMENTS

Theory and research on validation during comprehension implies that readers monitor the consistency of information with their prior beliefs during multiple text comprehension, provided that these beliefs are already activated or are triggered passively by concepts in the texts. In other words, readers continuously generate mostly implicit plausibility judgments that are based on general world knowledge but also on their prior and possibly partial beliefs. We propose that monitoring the plausibility of information affects comprehension in two ways, depending on whether readers take just one step or two steps of processing described in a simple two-step model of validation (Figure 1). According to this model, the first step of routine validation may lead to the detection of text-belief consistencies, which in essence affects memory and comprehension in terms of a text-belief consistency effect. However, in a second step, the extent that readers engage in the strategic, resource-intensive elaboration of belief-inconsistent information, which optimally results in the construction of a balanced mental model of the controversial issue, depends on their goals and motivation. We discuss the two steps in more detail in the following sections.

FIGURE 1 The two-step model of validation in multiple text comprehension.

Step 1: Epistemic Monitoring

We propose that readers use the perceived plausibility of arguments as a heuristic to allocate their cognitive resources during reading. Similar to other domains, people are cognitive misers when processing text for comprehension. They tend to construct only a sufficient rather than the best possible representation to economize their cognitive processing. In the case of multiple text comprehension with conflicting information, this means that readers tend to concentrate their cognitive resources on information that they find plausible. In contrast, information that has been detected as implausible (i.e., inconsistent with prior beliefs) by epistemic monitoring will, overall, receive less cognitive resources during comprehension. Note that this assumption is compatible with findings from many experiments based on the inconsistency paradigm, that inconsistent and implausible information increases reading times (Section “Validation in Text Comprehension”). This initial increase in reading times indicates a disruption of comprehension. The assumption here is that this initial disruption is usually not followed by an increase in strategic and resource-intensive comprehension processes (“repair processes”) but that readers simply move on in the text. The differential processing of belief-consistent and belief-inconsistent information should lead to a comprehension and memory advantage for belief-consistent information and arguments.

In sum, the proposed effects of epistemic monitoring on the processing of conflicting information may be considered a variant of the classical notions of selective exposure (Festinger, 1957) or confirmation bias (Nickerson, 1998). The common cognitive mechanism among these different models or theories is that people actively seek and favor information that confirms rather than contradicts their beliefs. However, the two-step model advocated in the present review adds three new aspects to this type of cognitive processing. First, the model assumes that the epistemic monitoring process is an involuntary and passive component of comprehension, that is, it operates independently from a specific goal or motivational state (such as the motivation to defend beliefs; Giner-Sorolila & Chaiken, 1997). Second, the nondeliberate preferential processing of belief-consistent information takes place during comprehension rather than, for example, in the stage of selecting information, which has been the focus of selective exposure theory research (Hart et al., 2009). Third, the model focuses on text-belief consistency effects in comprehension and memory rather than belief-stabilizing and belief-strengthening effects. Belief-stabilizing and belief-strengthening activities both suggest that readers are deliberately motivated to actively seek and favor information that confirms rather than contradicts their beliefs (see, e.g., the disconfirmation model by Edwards & Smith, 1996). In contrast, belief-consistency effects are supposed to occur due to passive and routine epistemic monitoring processes.

Research in different fields has revealed general comprehension and memory advantages for belief-consistent information. For example, studies on conceptual change have shown that readers hold fast to previously acquired knowledge and beliefs even when they are presented with new and contradicting evidence (Chinn & Brewer, 1993; Vosniadou, 1994). Research on the persistence of discredited or corrected information (Anderson, Lepper, & Ross, 1980; Ross, Lepper, & Hubbard, 1975) and on the continued influence of misinformation (Johnson & Seifert, 1994) yielded similar results. The fact that knowledge and beliefs persist even though they are discredited by later information may be partly explained by memory processes, for example, the passive reactivation of discredited, but nevertheless salient and easily accessible, concepts (Ecker, Swire, & Lewandowsky, 2014; Kendeou & O'Brien, 2014; see Richter & Singer, in press, for an overview). However, the epistemic monitoring of information during comprehension and the plausibility judgments that are generated by this process are likely to contribute to these effects as well (for a similar idea, see Lombardi, Nussbaum, & Sinatra, 2016).

A study by Schroeder, Richter, and Hoever (2008) directly addressed the relationship of epistemic monitoring and comprehension. Students read expository texts in their area of study (psychology) that contained implausible information (faulty arguments) and performed recognition and plausibility judgments on the same set of test items (paraphrases of text information and inferences that could be derived from the text). A multinomial models analysis of these judgments revealed a close bidirectional relationship of the plausibility judgments and comprehension. Plausible information was far more likely to be integrated into participants' mental model of the text content than implausible information. In contrast, information that was part of the mental model was more likely to be judged as plausible. Thus, once information has become part of a reader's mental model, it is used for monitoring the validity of incoming information. Basically the same mechanisms are assumed to take effect when readers process multiple texts with conflicting information, unless they are motivated and able to engage in an elaborative processing of belief-inconsistent information, as described in the next section.

Step 2: Elaborative Processing of Belief-Inconsistent Information

When epistemic monitoring has revealed an inconsistency between prior beliefs and incoming text information, readers may under certain circumstances engage in elaborative processing directed at resolving the inconsistency. For example, Blanc, Kendeou, van den Broek, and Brouillet (2008) found that readers were able to integrate two causal explanations for an event into one coherent mental representation by generating logical connections between the two causes. Apparently, readers sometimes actively resolve inconsistencies between incoming information and previously held knowledge. However, in contrast to the epistemic monitoring processes and their consequences for memory and comprehension, subsequent elaborative processing of inconsistent information is optional and understood to occur only when readers are motivated and cognitively capable. The reason for the conditional nature of this elaboration is that the underlying processes are assumed to be slow, resource demanding, and under the strategic control of the reader (Richter, 2003). Readers motivated to engage in these processes strive to achieve a justified and defensible point of view on a controversial issue. In other words, they follow an epistemic reading goal (Richter, 2011). According to the two-step model of validation, specific motivational states or goals in the second step of elaborative processing can affect the comprehension of multiple texts. These epistemic reading goals can appear in different forms. They can appear as a defense motivation, for example, when readers feel compelled to defend their views in front of others (fear of invalidity; Kruglanski & Webster, 1996) or wish to maintain their beliefs by actively finding reasons that discount alternative views (Edwards & Smith, 1996). However, epistemic reading goals can also manifest themselves as epistemic curiosity.

Elaborative processes instigated by inconsistencies may consist of various knowledge-based comprehension processes such as elaborative and bridging inferences that readers use to establish hypothetical truth conditions or to search for evidence that could support dubious information. As a consequence, these elaborative processes, if readers engage in them at all, can strongly support comprehension of belief-inconsistent information. For example, readers who follow an epistemic reading goal and possess the relevant prior knowledge and working memory capacity might actively search for additional information in the texts, or actively retrieve pro- and con-arguments from long-term memory. Ideally, they would arrive at an informed decision based on a rich and balanced mental model of the controversial issue in which the better-justified option is adopted. Thus, elaborative processing can greatly benefit comprehension. Experiments by Wiley and Voss (1999) on learning with multiple texts in history are a case in point. In these experiments, students wrote more coherent essays with stronger causal links and scored better on comprehension tasks when they had received the instruction to write an argumentative essay, as compared to the instruction to write a summary or a narrative text. The task to write an argumentative essay should create an epistemic reading goal that, according to the two-step model of validation, should promote the elaborative processing of belief-inconsistent information.

At the metacognitive level, epistemological beliefs can promote elaborative processing of belief-inconsistent information in multiple texts. Epistemological beliefs, which are also called epistemic beliefs in the literature (Kitchener, 2002), refer to characteristics, criteria, and justification conditions of knowledge and may be considered as the epistemological part of declarative metacognition (Hofer & Pintrich, 1997). According to Bråten, Britt, Strømsø, and Rouet (2011), epistemological beliefs can have a profound influence on the integration of information from multiple documents. In terms of our model, a well-developed epistemological position (such as commitment within relativism, Perry, 1970; or reflective judgment, King & Kitchener, 1994) is more likely to create the condition in which readers follow an epistemic reading goal and engage in the elaboration processing of belief-inconsistent information (Richter, 2011). Consider, for example, epistemological beliefs about the certainty of knowledge. Readers endorsing the belief that knowledge is certain and never changing (as opposed to fallible and changing) will see no point in resolving conflicts between their beliefs and text information and in constructing a balanced mental model of controversial issues. A self-report study by Richter and Schmid (2010, Study 2) illustrates the effects of epistemic reading goals. In this study, university students were more likely to report the application of epistemic strategies (such as actively checking whether knowledge claims are supported by sound reasons) in their everyday reading activities when they endorsed the belief that knowledge is uncertain and changing. This effect was mediated by epistemic curiosity, that is, by a prototypical epistemic reading goal (measured with self-report items such as I want to know which theory is correct in the explanation of a certain phenomenon). Thus, the belief that knowledge is uncertain or changing seems to predispose readers to be curious about learning the objective nature of phenomena. This epistemic reading goal, in turn, enhances elaborative processing of information presented in multiple texts.

Taken together, the two-step model of validation posits that readers use the belief consistency of information as a heuristic for the allocation of resources and processing capacities during reading. This assumption implies that by default (i.e., if no additional elaborative processing is used by readers), readers will evaluate belief-consistent information in multiple documents as more plausible, will process belief-consistent information in multiple documents more deeply, and will achieve a stronger mental model for documents that are consistent with their prior beliefs. Moreover, subsequent elaborative processing stimulated by inconsistencies allows readers to achieve a justified and defensible point of view on a controversial issue. In the next section, we review empirical evidence on belief effects in multiple document comprehension to investigate whether the assumptions of the two-step model hold. In particular, we examine how belief consistency and validation effects may manifest in the context of multiple text processing.

BELIEF CONSISTENCY EFFECTS IN MULTIPLE TEXT COMPREHENSION: EMPIRICAL EVIDENCE

In this section, we review published studies on the processing of belief-consistent versus belief-inconsistent information from multiple documents. The studies were selected from PsycINFO and Google Scholar using different combinations of the keywords multiple texts/documents/sources, reading comprehension, beliefs, conflicting/controversial/contrary information. Only articles published in English and published before October 2016 were considered in the literature review. The comprehensive search identified 107 articles (journal articles and empirical book chapters). Full-text copies of all articles were screened by both authors and were selected for the final systematic review based on three main inclusion criteria (Figure 2). The two-step model of validation posits that readers use the belief consistency of information as a heuristic for information selection and for the allocation of resources during reading. This assumption implies that by default readers will evaluate belief-consistent information in multiple texts as more plausible, will process belief-consistent information in multiple texts more deeply, and will achieve a stronger mental model for texts that are consistent with their prior beliefs.

FIGURE 2 Flow diagram of the research and selection of studies for the review.

To evaluate these assumptions, studies were selected only when they included (a) an assessment of readers' beliefs; (b) texts or arguments that varied in their consistency with these beliefs; and (c) an assessment of comprehension outcomes, belief ratings, argument evaluation tasks, essay tasks, or measures of processes during reading such as reading times, eye-tracking, or think-aloud data. Based on our screening criteria, some prominent studies from the field of multiple document comprehension were excluded. For example, the studies by Britt, Rouet, and colleagues on reasoning with multiple historical documents on the Panama Canal (e.g., Rouet, Britt, Mason, & Perfetti, 1996) were not included in the review because effects of students' prior beliefs on the historical controversy were not investigated. Following the criteria just described, a subset of 18 studies from 14 articles were investigated in the review. Table 1 provides an overview of the studies, including information on measures and distributions of prior beliefs, text materials, and measures of dependent variables. In the next sections, we first discuss studies identified in the systematic review that report belief consistency effects on comprehension outcomes, followed by studies based on argument evaluation tasks and comprehension processes. We then summarize the evidence on text, reader, and task characteristics that moderate text-belief consistency effects.

TABLE 1 Studies Included in the Review on Text-Belief Consistency Effects

Authors and Year of Publication	Sample	Assessment of Prior Beliefs	Belief Distribution in Sample	Material	Online and Offline Measures
Anmarkrud et al., 2014	51 university students	Belief position assessed in essays with holistic rubric	NA	Six documents presenting different perspectives on health risks of cell phones	Essay task
Kardash & Howell, 2000	38 educational psychology undergraduates	One item on 9-point Likert scale	26 participants believed that HIV causes AIDS, 7 believed that HIV does not cause AIDS, and 5 unsure	Dual-positional text (1,354-words) on the relationship between HIV and AIDS, eight arguments for each side	Think-aloud protocols comments coded in seven categories (e.g., developing awareness)
Kardash & Scholes, 1996	96 psychology undergraduates	One item on 9-point Likert scale collapsed in three categories: 1–4 = “HIV causes AIDS”; 5 = “uncertain”; 6–9 = “HIV does not cause AIDS”, and four categories of belief strength	64% believed that HIV causes AIDS, ∼18% were uncertain, ∼ 18% believed that HIV does not cause AIDS	Dual-positional text (1,354 words) on the relationship between HIV and AIDS, eight arguments for each side	Conclusion task (scored for degree of certainty)
Kobayashi, 2010	96 undergraduate students	Two pro and two con (reverse-coded) items on 7-point Likert scale. Higher scores indicate pro position	53 were opposed to the introduction of daylight savings time (score < 4), 4 were neutral (score = 4), and 39 favored it (score > 4)	Two contrary texts (535–547 Japanese letters) on the introduction of daylight savings time in Japan, one pro, one con	1) Argument evaluation 2) Argumentative essay task
Kobayashi, 2010	97 undergraduate students	Two pro and two con (reverse-coded) items on 7-point Likert scale. Higher scores indicate pro position	53 were opposed to the introduction of daylight savings time (score < 4), 7 were neutral (score = 4), and 37 favored it (score > 4)	Two original articles (3,019–3,345 Japanese letters) that argued for or against the introduction of daylight savings time in Japan	1) Argument evaluation 2) Argumentative essay task
Kobayashi, 2014	154 undergraduate students	Two pro and two con (reversed) items on 7-point Likert scale. Higher scores indicate more favorable attitudes toward blood type personality	Mean ratings of prior attitudes 4.26 (SD = 1.15)	Two texts (983/986 Japanese characters) about the relationship between ABO-blood type and personality, one pro, one con	1) Essay task coded as intertextual conflict resolution 2) Evaluation of text quality
Maier & Richter, 2013	79 psychology undergraduates	Five pro and five con items per issue on 6-point Likert scale	Preference for pro text stance for both topics, sample size on both sides not presented	Four texts for each topic (average length 899 words), two texts pro (human driven / vaccination beneficial) and two texts con (nature / vaccination harmful)	1) Reading times 2) Sentence recognition task 3) Source memory
Maier & Richter, 2014	85 university students	1) Prior beliefs: Five pro and five con items on 6-point Likert scale 2) Certainty ratings for belief items on 6-point Likert scale	17 participants more strongly agreed with the con text stance, remaining sample preferred pro text stance	Two texts (average length 888 words) discussing the risks and benefits of vaccinations, one text pro (vaccination beneficial) and one text con (vaccination harmful)	1) Reading times 2) Sentence verification and sentence recognition task
Maier & Richter, 2016b	39 university undergraduates	Five pro and five con items on 6-point Likert scale	20 participants more strongly agreed with the stance of the contra text and 19 with the stance of the pro text	Two texts (1076-1091words) on health risks of electromagnetic radiation from cell phones, one pro (causes health risks), one con (no health risks)	1) Think-aloud protocols coded as a) strategic validation, b) memorization 2) Reading times 3) Sentence verification task
Mason & Boscolo, 2004	65 high school students (10th and 11th grade)	One item on 5-point Likert scale (1 = totally con; 5 = totally pro)	Belief scores in groups indicate a preference for contra stance (M = 2.14- 2.71), information on sample size on both sides not presented	Dual-position expository text (2,727 words) about genetically modified food	1) Belief change 2) Text conclusion
McCrudden & Barnes, 2016	72 male secondary high school students	One item on a 6-point Likert scale	64 participants were more accepting, 8 participants were less accepting	Eight evidence-based arguments on the topic of whether humans affect the climate	1) Argument strength rating 2) Interview data (N = 8)
McCrudden & Sparks, 2014	45 male (ages 14–15) 10th grade high school students	1) Prior beliefs: One item on 9-point Likert scale. Higher scores indicate greater approval of widening the tunnel 2) Topic belief justification (open question)	Only participants selected for analyses that were proponents of widening the tunnel	One text (1,392 words) with 10 arguments interleaved presentation, five supporting and five opposing the widening of the tunnel	1) Belief change 2) Belief justification 3) Interview data (N = 8)
Van Strien et al., 2014	61 11th grade students	Six items assessed prior attitudes on 5-point Likert scale. High scores represent the view that violent videogames do not negatively affect behavior	Information on sample size on both sides not presented Only one essay negatively biased (excluded)	13 texts on the relationship between violent video games and aggressive behavior with a total number of 2,984 words (one neutral, six texts “not harmful”, six texts “harmful”)	Essay task analyzed for 1) Perspective (neutral, slightly or strongly biased, biased, other) 2) Position (positively or negatively biased, neutral) 3) Origin of information (borrowed, added, transformed)
Van Strien et al., 2016	79 university students	1) Prior attitudes: 14 items on 5-point Likert scale 2) Attitude strength: Three items (Taber & Lodge, 2006) on 6-point Likert scale	NA	Eight websites (212-220 words) on four aspects of organic foods. For each aspect, one website was pro and one website con, each website contained one argument	1) Total fixation duration on text 2) Essay Task 3) Credibility rating
Wiley, 2005	20 psychology undergraduates, 20 graduate students, 20 law students	1) Beliefs: One item on a 10-point Likert scale 2) Extremity of beliefs based on belief ratings	24 participants believed abortion should remain legal, 16 were against taking military action in Iraq in 1991	One text about abortion (533 words) and one text on Persian Gulf War (548 words), Each text 10 supporting arguments on each side of the issue	Recall
Wiley, 2005	24 psychology undergraduates	One item on a 10-point Likert scale	In each knowledge group (high vs. low) six participants on each side of the issue	One text (545-words) with ten arguments on either side about “Is OJ Simpson guilty?”	Recall
Wiley, 2005	48 psychology undergraduates	One item on a 10-point Likert scale	In each knowledge /presentation format subgroup an equal number of subjects on each side of the issue	One text about abortion (533 words) with ten supporting arguments on each side of the issue	Recall
Wiley, 2005	48 psychology undergraduates	One item on a 10-point Likert scale	In each knowledge presentation format subgroup an equal number of subjects on each side of the issue	One text on Persian Gulf War (451 blocked, 496 interleaved words) with 10 supporting arguments on each side of the issue	Recall

Text-Belief Consistency Effects on Comprehension Outcomes

A number of studies identified in the review investigated text-belief consistency effects by using essay-writing tasks to assess comprehension outcomes. In a study by van Strien, Brand-Gruwel, and Boshuizen (2014), 11th graders read 13 documents (one neutral, six pro, and six con) on the link between violent video games and aggression. In a hypertext environment, participants could choose the texts they wished to read and were given 35 min for reading and for writing an essay on the issue. The results revealed a strong text-belief consistency effect: Students were far more likely to take a position in the essay that was consistent with their prior beliefs than to take a position that was inconsistent with their prior beliefs (odds ratio = 2.85). Students with strong prior beliefs also added more information (based on their prior knowledge and beliefs) to their essays, whereas students with neutral prior beliefs relied more on information from the texts they had read. An important point is that van Strien et al.'s method imposed a mild time pressure on participants. According to the two-step model of validation presented in this article, time pressure should reduce the opportunity to engage in elaborative epistemic processing, thus creating conditions favoring a text-belief consistency effect.

Another experiment by van Strien, Kammerer, Brand-Gruwel, and Boshuizen (2016) included belief strength (composite score of interest, strength, and certainty) as an additional predictor. University students read eight documents that took a positive or negative stance toward organic food, and they wrote an essay on the question of whether organic food is better than conventional food. A text-belief consistency effect emerged, but this effect was qualified by an interaction with attitude strength. Participants with strong prior beliefs (either in favor of or against organic food) used far more arguments from belief-consistent web pages compared to belief-inconsistent web pages (d = 0.68, estimated at 1 SD above the sample mean of the belief strength score), whereas no such effect was found for participants with weak prior beliefs (estimated at 1 SD below the sample mean of the belief strength score). Thus, consistent with the two-step model of validation, the text-belief consistency effect depended on strong prior beliefs. These results also concur with those of an exploratory study by Anmarkrud, Bråten, and Strømsø (2014). In this study, undergraduates read six texts providing different views on whether the use of cell phones is associated with health risks, and they produced an argumentative essay after reading. Anmarkrud et al. did not assess participants' prior beliefs, but the participants' essays were scored on the quality of argumentative reasoning using a rubric (7-point rating scale) proposed by Reznitskaya and Anderson (2002). The mean score across all participants indicated that most participants produced essays that contained few reasons supporting their position and provided no discussion or reasons for alternative perspectives. This finding can be taken as indirect evidence for the text-belief consistency effect.

In some studies that have presented texts on controversial issues, participants were asked to write a concluding paragraph, which may be considered a variant of the essay-writing task. Kardash and Scholes (1996) used a dual-position text that included two contradictory positions about whether a causal link between HIV and AIDS exists. The two positions were clearly separated, appearing in different paragraphs, and each position was clearly linked to one of two sources (names of scientists), resulting in a situation that resembled multiple documents comprehension. Almost all of the undergraduate participants who produced a conclusion (only these were included in the analyses) held the belief that HIV causes AIDS prior to reading the text. Of these, 37.2% still advocated this position in their written conclusion, apparently ignoring the counterarguments that they had read in the dual-position texts. A similar proportion (46.5%) wrote conclusions that were uncertain, tentative, or ambiguous, thus acknowledging the controversial nature of the issue.

Studies with essay or conclusion writing tasks are an important source of evidence for text-belief consistency effects, because they demonstrate that readers spontaneously rely more on belief-consistent information obtained from texts when prompted to write their point of view on controversial issues. However, these tasks might also strengthen reading goals directed at bolstering prior beliefs, that is, to establish motivated text-belief consistency effects. Thus, to reveal the routine and default character of text-belief consistency effects, research should show that such effects also occur in experiments with general memory and comprehension tasks such as recall or recognition tasks that are presumably less obtrusive. Wiley (2005, Experiment 1) presented undergraduate and graduate students with 10 pro and 10 con arguments on two controversial issues after assessing their prior knowledge and beliefs. The sample was divided according to less-educated/low-knowledge, more-educated/low-knowledge, and more-educated/high-knowledge participants. Less-educated/low-knowledge participants recalled more arguments that were consistent with their beliefs than arguments inconsistent with their beliefs (d = .50).¹ In more-educated/low-knowledge participants, a similar but smaller difference was found that marginally failed to reach significance in a one-tailed test (p = .08, d = .40). No significant difference was found in the high-knowledge group. This finding could indicate that more knowledge shields against text-belief-consistency effects, which would be consistent with the assumption of the two-step model of validation, that greater knowledge enables readers to elaborate on arguments from both sides of a controversy. However, this finding might have also been due to a ceiling effect, because the recall of the high-knowledge participants was almost perfect (9.6 arguments out of 10), leaving little room for a difference between belief-consistent and belief-inconsistent arguments to emerge. Wiley's second experiment, based on 20 arguments on a different controversial topic and with high- versus low-knowledge groups of participants, ruled out this alternative explanation. Again, a strong text-belief consistency effect emerged only in the low-knowledge group (p > .01, d = .94), whereas no such effect occurred in the high-knowledge group, that recalled approximately five arguments on each side of the issue. In two additional experiments, Wiley also identified a text-based factor that reduced the text-belief consistency effect. When the arguments representing opposing stances were presented in an interleaved instead of in a blocked manner, the text-belief consistency effect disappeared.

One might argue that the experiments by Wiley (2005) did not really target multiple documents comprehension, because the arguments representing each side of the controversy could be conceived of as being part of one single (albeit internally inconsistent) text. Other studies, however, found similar effects using multiple texts. Of note is Maier and Richter's study (2013), in which a text-belief consistency effect emerged in undergraduates for situation model strength, measured with a recognition task based on inference items (corrected for response bias), when two texts supporting one side of the controversy were presented in a blocked manner (d = 0.87). In contrast, the text-belief consistency effect disappeared when two additional texts with different stances were read in an interleaved sequence. This result provided further evidence that an interleaved presentation of opposing stances in a controversy seems to help readers achieve a balanced mental representation of a controversial issue. The text-belief consistency effect found by Maier and Richter (2013) was replicated in a study by Maier and Richter (2014), who presented undergraduates with just two texts that took opposing stances on vaccination. In this training experiment (described in more detail in the upcoming section Conditions Fostering Elaborative Processing of Conflicting Information), participants received either a metacognitive strategy training or no training and received either positive or negative performance feedback for a practice phase or no feedback. A text-belief consistency effect emerged on situation model strength in all groups except for the group that received positive performance feedback and the metacognitive strategy training, with a median effect size (Cohen's d for dependent measures) of .55 (Figure 3).

FIGURE 3 Example of a text-belief consistency effect on general memory and comprehension tasks: Effects of text-belief consistency and training on situation model strength. From “Training Multiple Text Comprehension: How Metacognitive Strategies and Motivation Moderate the Text-Belief Consistency Effect” by J. Maier and T. Richter, 2014, Metacognition and Learning, Vol. 9, No. 1, p. 66. © Springer International Publishing AG. Reproduced by permission of Springer International Publishing AG. Permission to reuse must be obtained from the rightsholder.

A third group of studies employed evaluations of arguments or evaluations of argumentative statements as dependent variables. When such studies differentiate between different types of arguments (e.g., pro vs. con arguments, weak vs. strong arguments) or assess justifications for evaluations of argument strength, they can provide a means to further understand the contribution of validation processes to text-belief consistency effects. For example, a text-belief consistency effect in argument evaluations also emerged in a study by Kobayashi (2010, Experiment 1) who presented Japanese undergraduates with two texts that presented arguments for or against the introduction of daylight savings time in Japan. Participants rated belief-consistent arguments as more convincing than belief-inconsistent arguments. In a subsequent essay task, the more that participants were in favor of daylight savings time, the fewer favorable statements they produced in response to the counterarguments provided in the text. McCrudden and Barnes (2016) provided additional insight in this line of research. High school students evaluated eight evidence-based arguments that represented opposite views on human contributions to climate change. The arguments also varied in argument strength (strong arguments were based on evidence from long periods of time, such as several decades, whereas weak arguments were based on evidence from short periods of time, such as only 1 year). McCrudden and Barnes found a medium-sized effect of text-belief consistency on argument evaluation (η² = .06), suggesting that participants relied on simple belief-consistency judgments as a myside bias heuristic for evaluating conflicting information. However, they also found a main effect of argument strength in the same order of magnitude (η² = .06), which indicates that participants also weighed the extent that the empirical evidence can be considered as valid reasons that support the claims about climate change. In addition, an interaction effect emerged, because the text-belief consistency effect was larger for strong arguments as compared to weak arguments.

In another study by McCrudden and Sparks (2014), high school students read a dual-position text providing arguments for and against widening a local tunnel. The researchers' goal was to identify reading goals as a potential moderator. The instruction was orthogonally varied to direct readers' focus on evidence and reasons (vs. no instruction) and to alert readers to the presence of multiple viewpoints. Most participants were in favor of widening the tunnel (only data from these participants were included in the analyses), but their beliefs on this issue were weaker after reading when they had received the evidence instruction (d = .94). No changes in beliefs were observed in the other groups. In addition, both the evidence and the multiple viewpoints instruction led to the inclusion of more opposing ideas in the justifications students provided for their beliefs. These results can be interpreted in light of text-belief consistency effects. Beliefs may have remained stable in participants due to routine validation processes, unless they were explicitly told to focus on evidence and reasons. Finally, Kobayashi (2014) found strong text-belief consistency effects in a study with Japanese undergraduates who read two texts that presented opposite stances on whether a relationship between blood type and personality exists. Participants holding prior beliefs in favor of this assumption rated the pro text as more acceptable than the con text. Moreover, prior beliefs were a strong predictor of whether participants attempted to resolve the conflict in a particular direction in an essay-writing task. In a multinomial logistic regression, the likelihood of resolving the debate in favor of the blood type—personality link (vs. in favor of the assumption that no such link exists) increased with more favorable prior beliefs (odds ratio = 1.53). This relationship held even in a model in which the acceptability ratings for the two texts (plus a number of other ratings of the text) were statistically controlled.

Text-Belief Consistency and Comprehension Processes

To date, relatively few studies have focused on process data that can provide information on the cognitive processes underlying the text-belief consistency effect. From the five studies identified in the systematic review that assessed processing, most of them collected reading times or eye-tracking data that indicate how readers allocated their cognitive resources to belief-consistent versus belief-inconsistent texts. In their experiment that revealed text-belief consistency effects in situation model strength for multiple texts on two controversial issues, Maier and Richter (2013) also examined the relationship between the reading time differences for belief-consistent versus belief-inconsistent texts and comprehension. They found that the text-belief consistency effect of situation model strength that occurred when the texts were presented block-by-block became stronger when participants devoted less time to the belief-inconsistent text (Figure 4). This finding is consistent with the two-step model of validation suggesting that superficial processing promotes a comprehension advantage for belief-consistent texts. In the experiment by van Strien et al. (2016), which exhibited a text-belief consistency effect for participants with strong prior beliefs, the sum of the total fixation durations was assessed for each of the eight web pages arguing for or against organic food. Van Strien et al. did not find an absolute text-belief consistency effect that would manifest itself in longer fixation durations for belief-consistent versus belief-inconsistent web pages. However, they found a relative text-belief consistency effect that depended on belief strength. Participants with weak prior beliefs (i.e., low self-rated interest, belief strength, and certainty, estimated 1 SD below the sample mean) spent more time reading the belief-inconsistent web pages, whereas the reading times were equal in participants with strong prior beliefs (estimated at 1 SD above the sample mean). This result that only this relative text-belief consistency effect occurred is difficult to explain. Notably, most participants (90%) were in favor of organic food. As a result, text-belief consistency was not balanced across texts. Therefore, the absence of an absolute text-belief consistency effect might have simply occurred because of differences in the pro and con texts.

Another source of information about cognitive processes potentially underlying the text-belief consistency effect are studies that collected think-aloud data during reading. Maier and Richter (2016a) measured reading times while undergraduates read two texts that took opposite stances on the issue of health risks caused by cell phone use. When reading the text with the goal to write a summary, participants focused their cognitive resources on belief-consistent information as indicated by longer reading times (d = 0.62) and the use of more memorization strategies for the belief-consistent text (d = 0.46). However, when participants were given the task to write an argumentative essay, they allocated their cognitive resources in a more balanced way, as indicated by equal reading times for the belief-consistent and the belief-inconsistent texts. Participants given the task to write an argumentative essay also engaged more often in strategic, elaborative validation strategies when reading the belief-inconsistent text (d = 0.33). The use of these strategies, in turn, was associated with a stronger mental model for the belief-inconsistent text (assessed with a verification task and inference items). Consistent with the two-step model of validation, the text-belief consistency effect occurred when participants worked on a reading task that required receptive processing and a relatively superficial interaction with the claims and reasons provided in a text but not when they were given an argument task that was more epistemic in nature. This task prompted readers to elaborate on the belief-inconsistent information, leading to a better comprehension of the controversial issue and a more balanced mental model of the scientific controversy.

FIGURE 4 Example of a text-belief consistency effect on comprehension processes: Effects of text-belief consistency and training on reading times (per syllable) for belief-consistent and belief-inconsistent information. From “Training Multiple Text Comprehension: How Metacognitive Strategies and Motivation Moderate the Text-Belief Consistency Effect” by J. Maier and T. Richter, 2014, Metacognition and Learning, Vol. 9, No. 1, p. 66. © Springer International Publishing AG. Reproduced by permission of Springer International Publishing AG. Permission to reuse must be obtained from the rightsholder.

Kardash and Howell (2000) collected think-aloud data based on the dual-position text debating the possibility of a causal link between HIV and AIDS (same material as in Kardash & Scholes, 1996). Think-aloud data for belief-consistent and belief-inconsistent paragraphs were analyzed only for the majority of undergraduate participants (26 of 38) that held the prior belief that HIV causes AIDS. Kardash and Howell found that participants made more statements indicating metacognitive awareness (called “developing awareness” by the authors) while reading the belief-consistent paragraphs as compared to the belief-inconsistent paragraphs (d = 0.41). In contrast, participants made more judgments and evaluations while reading belief-inconsistent, as compared to belief-consistent, passages (d = 0.64). From the perspective of the two-step model of validation advocated in the present review, the greater frequency of metacognitive statements in the protocols for the belief-consistent paragraphs suggests a deeper processing of belief-consistent information. Furthermore, the greater frequency of judgments and evaluations in the belief-inconsistent passages indicates that participants monitor text information for belief consistency, which provides evidence for the idea that readers validated incoming information. Finally, in the study by Anmarkrud et al. (2014), previously discussed as providing indirect evidence for text-belief consistency effects, think-aloud protocols obtained during reading were scored for statements indicating strategic processing during reading of the six texts on cell phone risks. The more participants used sophisticated evaluation strategies (such as using information from the present source to evaluate text content, using text characteristics to evaluate their trustworthiness, or evaluating one text in relation to another one) and monitoring strategies (such as perceiving that the different texts provide diverse views), the more their argumentative essays were balanced and justified (with correlations of .42 and .43, respectively). This is in line with the assumption of the two-step model of validation that elaborative processing fosters the comprehension of belief-inconsistent information and decreases text-belief consistency effects.

Conditions Fostering Elaborative Processing of Conflicting Information

Text-belief consistency effects lead to the biased comprehension of conflicting multiple texts, making it desirable to engage readers in the strategic evaluation of the validity and quality of information presented in all texts. The research reviewed in the previous section suggests several conditions that might reduce the influence of beliefs on multiple text comprehension and foster elaborative processing.

Text presentation

Wiley (2005) and Maier and Richter (2013) found that an interleaved presentation of texts or arguments with opposing claims can eliminate text-belief consistency effects (that occurred in a block-by-block presentation of the same texts or arguments). To our knowledge, these two studies are the only ones that consistently varied the sequence in which more than two texts or arguments were presented. Typical studies of multiple text comprehension are based on just two texts, a random sequence of texts, or they let participants choose the order in which different texts are studied. Given the relative paucity of research on this issue, replication studies on the moderating effect of text presentation with different types of samples and text material are needed. This research could also focus on potential mechanisms underlying a moderating effect of text sequence, which are basically unknown.

Task instructions

The purpose or reading goal that readers receive prior to reading seems to decrease or increase the occurrence of text-belief consistency effects. Reading goals that prevent the occurrence of text-belief consistency effects are rationale and evidence instructions (McCrudden & Sparks, 2014) as well as argument instructions (Maier & Richter, 2016a). These reading instructions have in common the effect of increasing the relevance of processing belief-inconsistent information during reading. For example, the rationale instruction used by McCrudden and Sparks called for participants' attention to evidence and reasons for both sides (rather than only one side) of a controversial topic. Similarly, the rationale instruction highlighted the benefit of reading multiple viewpoints. The argument instruction, used in one of our own studies, assigned readers to build a justified point of view about the controversial issue. Even if presented more indirectly, such a reading task can be fulfilled only if both sides on the controversial issue are understood. Hence, reading tasks that highlight the importance of or directly require the understanding of belief-inconsistent information seem to prevent the occurrence of text-belief consistency effects. In terms of the two-step model of validation proposed in this article, such instructions create epistemic reading goals that foster elaborative epistemic processing.

Reader characteristics

A number of studies with multiple texts on socioscientific issues found that reader characteristics such as epistemological beliefs (Mason & Boscolo, 2004), belief strength (Kardash & Scholes, 1996; McCrudden & Barnes, 2016) and prior knowledge, and level of education (Wiley, 2005) can moderate text-belief consistency effects. In particular, an advanced epistemological understanding, strong prior knowledge, a higher level of education, and weaker beliefs seem to prevent the occurrence of text-belief consistency effects. For example, Kardash and Scholes (1996) identified epistemological beliefs as a reader characteristic that works against a text-belief consistency effect. In a multiple regression analysis with prior beliefs, need for cognition, and epistemological beliefs in the certainty of knowledge (assessed with the questionnaire by Schommer, 1990) as predictors and the conclusion score as criterion, prior beliefs and certainty of knowledge emerged as significant predictors, each explaining 9% of variance in the criterion. The more participants believed that knowledge is tentative and evolving, the more their written conclusions reflected the uncertain and inconclusive nature of the issue.

The results from Mason and Boscolo (2004) support the conclusions that can be drawn from Kardash's and Scholes's (1996) results. In the study from Mason and Boscolo, high school students read a dual-position expository text on genetically modified food and wrote a concluding paragraph for the dual-position text after reading. Participants' beliefs about this issue were assessed before and after reading, and the sophistication of their epistemological beliefs was measured with an instrument proposed by Kuhn, Cheney, and Weinstock (2000) that differentiates between an absolutist, multiplist, and evaluativist level of epistemological thinking. According to the prior belief scores reported by the authors, participants, on average, endorsed a critical stance toward genetically modified food. After reading the dual-position text, only participants with a more sophisticated epistemological understanding changed their beliefs toward a more neutral position, whereas those with absolutist epistemological beliefs held fast to their original beliefs.

Moreover, independent of given task instructions, interview data provided in the study from McCrudden and Sparks (2014) suggests that readers' personal reading intentions influence the strength of text-belief consistency effects. A belief-protection goal, which includes a reaffirmation of prior beliefs based on one's prior knowledge and experiences, seems to facilitate text-belief consistency effects, whereas a belief-reflection goal, which includes critically evaluating arguments and evidence from both sides of a scientific controversy and being more open to belief-inconsistent information, increases the incorporation of opposing arguments into readers' understanding.

The results from these studies support the proposition of the two-step model of validation that readers' reading goal modifies the extent to which readers rely on epistemic monitoring or engage in additional elaboration processes. A belief-protection goal might include low standards of coherence, resulting in less engagement in elaboration processes. Further, the belief that knowledge is uncertain or changing seems to predispose readers to be curious about learning the objective nature of phenomena. This epistemic reading goal, in turn, enhances elaborative processing of information presented in multiple texts.

Trainings

Several types of interventions have been proposed to increase readers' comprehension of conflicting information in multiple texts. One promising approach is to enhance readers' metacognitive knowledge about the processing of conflicting information, which can increase the monitoring and regulation of relevant cognitive processes in a functional way (Ku & Ho, 2010; Maier & Richter, 2014; Stadtler & Bromme, 2007). One approach that we have followed in our own research investigating cognitive processes involved in routine versus strategic validation is to create an awareness of potential biases resulting from these processes and to teach strategies that can be used to control the outcomes of these processes (Maier & Richter, 2014, 2016b). For example, Maier and Richter (2016b) trained participants' metacognitive strategies that were expected to increase participants' awareness of the cognitive processes involved in routine validation (i.e., the effects of routine epistemic monitoring) and foster strategic validation (i.e., the active use of prior knowledge to validate information). The effectiveness of the metacognitive training was compared to the PQ4R training (Thomas & Robinson, 1972), which can be viewed as a well-situated and effective reading strategy training that increases elaborative processing.² Results revealed that participants who had received the metacognitive strategy training were able to build similarly strong mental models for both belief-consistent and belief-inconsistent texts. In contrast, participants who had received the PQ4R training still suffered from a less complete understanding of the controversial scientific issue, because their situation model for belief-inconsistent texts was significantly weaker as compared to their situation model for belief-consistent texts. These results indicate that classical cognitive and metacognitive strategies (which are the focus of the PQ4R training) are not sufficient to assist readers in the successful comprehension of controversial texts. Instead, multiple text comprehension requires readers to pay attention to routine cognitive processes involved in the immediate validation of information and the biasing effects of these processes (such as text-belief consistency effects) and to actively use their prior knowledge to strategically validate the plausibility of information (Richter, 2015; Richter & Schmid, 2010). Instructing readers to take a neutral perspective prior to reading and to activate prior knowledge to enrich textual information does not seem to be sufficient to overcome the belief bias that occurs in multiple text comprehension.

A second approach that might be effective in promoting the comprehension of multiple texts with conflicting information and in reducing text-belief consistency effects are trainings that focus on developing knowledge about the structure of arguments and the identification of different argument components such as claims and data (Toulmin, 1958). Multiple texts present arguments of varying quality. Thus, text-belief consistency corresponds to text-claim consistency at a more nuanced level. Readers who are aware of the structure of an argument are better able to evaluate the quality of the argument (Britt & Larson, 2003; Wolfe, Britt, & Butler, 2009) and to focus on its internal consistency rather than focusing on whether the claim is consistent with their beliefs (von der Mühlen, Richter, Schmid, Schmidt, & Berthold, 2016). For example, Wolfe et al. (2009, Study 3) found that readers benefited from a tutorial about the structure of written arguments to the extent that they were better in expressing a clear main claim statement and provided elaborated ground and counterarguments in written essays. For multiple text comprehension, knowledge of the structure of written arguments might be similarly beneficial, because this knowledge enables readers to validate the internal consistency of the arguments provided within each text (e.g., evaluate the relationship between claim and evidence) and to relate competing claims across texts (e.g., to identify rebuttals). In addition, attending to and elaborating on belief-inconsistent counterarguments is an important strategy in reducing the effect of immediate validation processes. For this aim, training the identification of markers (e.g., modals and qualifiers) that identify argument components or indicate the relationship between different arguments (Britt & Larson, 2003; Larson, Britt, & Larson, 2004) might assist readers in the comprehension of multiple texts. However, a direct test of these assumptions made by the two-step model of validation is still lacking.

A third approach to foster multiple document comprehension is training readers to become more aware of source information and to evaluate the quality and validity of the sources (e.g., Braasch, Bråten, Strømsø, Anmarkrud, & Ferguson, 2013; Brem, Russell, & Weems, 2001; Britt & Aglinskas, 2002; Macedo-Rouet, Braasch, Britt, & Rouet, 2013, Stadtler & Bromme, 2007, 2008; Wiley et al., 2009). Such trainings aim at educating readers on how to discriminate reliable from unreliable documents based on source attributes, such as publication type or date of publication. Paying attention to source information and reliably discriminating trustworthy from untrustworthy sources may be an important step in achieving a balanced and unbiased mental model of controversial issues, especially under circumstances in which readers lack the ability or motivation to engage in elaborative processing of the content provided in multiple texts. However, strategies that focus solely on source information, taught in these trainings, can counter text-belief consistency effects only when the trustworthiness of texts differs and is correlated with text-belief consistency.

CONCLUSION

Readers use the World Wide Web as a tool for information search to gain a better understanding of topics they are interested in, which often revolve around controversial issues. The two-step model of validation, presented here, and the accompanying review of published studies suggest that prior beliefs can impede such an understanding. Beliefs influence the evaluation of arguments and the processing and comprehension of multiple documents, which in turn lead to text-belief consistency effects. First, readers evaluate belief-consistent arguments and claims as more plausible, process belief-consistent information from multiple documents more deeply, and achieve a stronger mental model for documents that are consistent with their prior beliefs. Second, reader and task characteristics, such as reading goals or prior knowledge, can reduce text-belief consistency effects. The reader should note that the results we identified in the systematic review of the existing literature are fully in line with the assumptions of the two-step model of validation.

Text-belief consistency effects occurred in research regardless of whether prior beliefs were assessed only with one item (Kardash & Howell, 2000; Kardash & Scholes, 1996; Mason & Boscolo, 2004; McCrudden & Barnes, 2016; McCrudden & Sparks, 2014; Wiley, 2005) or with multiple items representing the two divergent argumentative stances on the controversy (Kobayashi, 2010, 2014; Maier & Richter, 2013, 2014, 2016a; van Strien et al., 2014; van Strien et al., 2016). Moreover, text-belief consistency effects also occurred even when they were not directly the focus of the research (Anmarkrud et al., 2014), and they were even stronger than the effects of the interventions (Kobayashi, 2010). The disadvantage in the comprehension of belief-inconsistent information was not accompanied by heightened metacognitive awareness or a strategic validation of information (Maier & Richter, 2016a), and reading goals enhancing critical thinking reduced the text-belief consistency effect (McCrudden & Sparks, 2014). These findings support the assumption that text-belief consistency effects are not (or at least not completely) due to motivated reasoning but are also a consequence of cognitive processes inherent in comprehension, that is, epistemic monitoring.

Text-belief consistency effects may be regarded as a well-established phenomenon. In our systematic review, none of the studies that fulfilled the inclusion criteria failed to find a text-belief consistency effect. However, still relatively little is known about the cognitive processes underlying these effects. The available studies that assessed process data mostly focused on reading times or think-aloud protocols that are limited to strategic processes. To expand current knowledge of processing differences between belief-consistent and belief-inconsistent information, eye tracking might be used for an immediate and nonintrusive investigation of information processing (Just & Carpenter, 1980). Moreover, future research should also pay more attention to the distribution of prior beliefs in samples and to a balanced design. For most scientific topics that have been used in published studies, participants agreed more strongly with one argumentative position in the controversy prior to the intervention (such as the relationship between HIV and AIDS, Kardash & Howell, 2000; or the causes of global warming; Maier & Richter, 2013). Even if the likelihood of effects from material differences is low across all studies, balanced designs (i.e., a similar amount of believers and disbelievers in each experimental condition) should be carried out by researchers. Finally, studies that aim at investigating both steps of the two-step model of validation should first include at least one experimental condition that is likely to bring about a text-belief consistency effect, for example, by creating a receptive or no reading goal in readers with strong beliefs. Moreover, at least one other experimental condition should be included that creates a condition that counteracts the text-belief consistency effect, for example, by creating an epistemic reading goal, by providing background knowledge to participants, or by teaching them metacognitive strategies. Ideally, such a study would assess not only comprehension outcomes but also indicators of routine validation processes and the strategic cognitive processes that are supposed to be involved in the optional elaboration of belief-inconsistent information.

The two-step model of validation suggests that all information that is currently held in working memory or that is passively activated by concepts or propositions in the text might be used by readers to validate new information. According to our model, prior beliefs, prior knowledge or previous text information are all possible sources that can be used for validation. Thus, the psychological differences that are often ascribed to knowledge and beliefs, for example, are not relevant for their functional role in validation. It does not matter, for example, whether the information that is used as epistemic background for validation is intersubjectively shared knowledge or part of a subjective belief system. Aspects of belief strength such as certainty, extremity, or accessibility (Petty & Krosnick, 1995) are relevant only insofar they may affect the likelihood that the belief is passively activated during comprehension and, thus, can be used for validation. In this review, we were especially interested in the effects of prior beliefs as we assume that readers primarily rely on these beliefs as epistemic background for validation when they comprehend multiple documents on socioscientific issues. Nevertheless, investigating how validation based on prior knowledge and/or previously read text information influences multiple documents processing is an interesting avenue for future research. For example, using earlier encountered text information for validation might contribute to misinformation effects where readers continue to rely on earlier learned information even if this information is discredited or corrected by subsequent information (see Ecker et al., 2014, for a review). The two-step model of validation might also be used for designing classroom instruction when educational practitioners use multiple documents for teaching. For example, the two-step model of validation proposes that reading instructions calling for neutral perspectives and the activation of relevant prior knowledge before reading might not be suitable for the comprehension of controversial documents but might rather lead to potential biases, such as text-belief consistency effects. If awareness of validation during comprehension was created, such biases could be avoided. For example, including a training of specific metacognitive strategies (see Maier & Richter, 2014) or epistemic reading goals (e.g., argument tasks or belief-reflection tasks) in classroom instructions should be able to enhance critical thinking and to avoid biased processing.

The two-step model of validation assumes that general cognitive mechanisms contribute to the processing of conflicting information in multiple texts. Accordingly, the scope of the two-step model is, in principle, not limited to multiple texts but covers the processing of controversial information presented in other formats as well. However, single texts seldom present controversial information without an attempt of resolving, reconciling, or integrating controversial positions or conflicting evidence. Thus, we view multiple documents comprehension as the primary scenario for applying this model.

In sum, ample evidence has shown that readers' beliefs serve as a special kind of gatekeeper that determines the extent that readers engage in the comprehension of belief-relevant information. According to the two-step model of validation advocated here, this biased processing is not always due to motivated reasoning, such as disconfirming belief-inconsistent information but rather due to passive and routine validation processes (epistemic monitoring) that are part of basic comprehension processes. Considering the concept of validation in future research and practical interventions has the potential to broaden and enrich our understanding of multiple documents comprehension.

FUNDING

This research was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG; grants RI 1100/5–1, RI 1100/5–2, and RI 1100/5–3) and the German Ministry of Education and Research (BMBF; grant 01PK15009B).

Notes

¹ For estimations of effect size d in within-subject designs a correlation of .5 between consistent and inconsistent indicators (such as argument recall, reading times, or category frequencies) or between pre- and postmeasures of beliefs was assumed (unless the exact correlation was reported in the publication).

² The acronym PQ4R stands for the six overlapping stages of Preview, Question, Read, Reflect, Recite, and Review that are central to the reading skill training from Thomas and Robinson (1972).