Comprehension of Multiple Documents With Conflicting Information: A Two-Step Model of Validation

Figures & data

FIGURE 1 The two-step model of validation in multiple text comprehension.

FIGURE 2 Flow diagram of the research and selection of studies for the review.

TABLE 1 Studies Included in the Review on Text-Belief Consistency Effects

Authors and Year of Publication	Sample	Assessment of Prior Beliefs	Belief Distribution in Sample	Material	Online and Offline Measures
Anmarkrud et al., 2014	51 university students	Belief position assessed in essays with holistic rubric	NA	Six documents presenting different perspectives on health risks of cell phones	Essay task
Kardash & Howell, 2000	38 educational psychology undergraduates	One item on 9-point Likert scale	26 participants believed that HIV causes AIDS, 7 believed that HIV does not cause AIDS, and 5 unsure	Dual-positional text (1,354-words) on the relationship between HIV and AIDS, eight arguments for each side	Think-aloud protocols comments coded in seven categories (e.g., developing awareness)
Kardash & Scholes, 1996	96 psychology undergraduates	One item on 9-point Likert scale collapsed in three categories: 1–4 = “HIV causes AIDS”; 5 = “uncertain”; 6–9 = “HIV does not cause AIDS”, and four categories of belief strength	64% believed that HIV causes AIDS, ∼18% were uncertain, ∼ 18% believed that HIV does not cause AIDS	Dual-positional text (1,354 words) on the relationship between HIV and AIDS, eight arguments for each side	Conclusion task (scored for degree of certainty)
Kobayashi, 2010	96 undergraduate students	Two pro and two con (reverse-coded) items on 7-point Likert scale. Higher scores indicate pro position	53 were opposed to the introduction of daylight savings time (score < 4), 4 were neutral (score = 4), and 39 favored it (score > 4)	Two contrary texts (535–547 Japanese letters) on the introduction of daylight savings time in Japan, one pro, one con	1) Argument evaluation 2) Argumentative essay task
Kobayashi, 2010	97 undergraduate students	Two pro and two con (reverse-coded) items on 7-point Likert scale. Higher scores indicate pro position	53 were opposed to the introduction of daylight savings time (score < 4), 7 were neutral (score = 4), and 37 favored it (score > 4)	Two original articles (3,019–3,345 Japanese letters) that argued for or against the introduction of daylight savings time in Japan	1) Argument evaluation 2) Argumentative essay task
Kobayashi, 2014	154 undergraduate students	Two pro and two con (reversed) items on 7-point Likert scale. Higher scores indicate more favorable attitudes toward blood type personality	Mean ratings of prior attitudes 4.26 (SD = 1.15)	Two texts (983/986 Japanese characters) about the relationship between ABO-blood type and personality, one pro, one con	1) Essay task coded as intertextual conflict resolution 2) Evaluation of text quality
Maier & Richter, 2013	79 psychology undergraduates	Five pro and five con items per issue on 6-point Likert scale	Preference for pro text stance for both topics, sample size on both sides not presented	Four texts for each topic (average length 899 words), two texts pro (human driven / vaccination beneficial) and two texts con (nature / vaccination harmful)	1) Reading times 2) Sentence recognition task 3) Source memory
Maier & Richter, 2014	85 university students	1) Prior beliefs: Five pro and five con items on 6-point Likert scale 2) Certainty ratings for belief items on 6-point Likert scale	17 participants more strongly agreed with the con text stance, remaining sample preferred pro text stance	Two texts (average length 888 words) discussing the risks and benefits of vaccinations, one text pro (vaccination beneficial) and one text con (vaccination harmful)	1) Reading times 2) Sentence verification and sentence recognition task
Maier & Richter, 2016b	39 university undergraduates	Five pro and five con items on 6-point Likert scale	20 participants more strongly agreed with the stance of the contra text and 19 with the stance of the pro text	Two texts (1076-1091words) on health risks of electromagnetic radiation from cell phones, one pro (causes health risks), one con (no health risks)	1) Think-aloud protocols coded as a) strategic validation, b) memorization 2) Reading times 3) Sentence verification task
Mason & Boscolo, 2004	65 high school students (10th and 11th grade)	One item on 5-point Likert scale (1 = totally con; 5 = totally pro)	Belief scores in groups indicate a preference for contra stance (M = 2.14- 2.71), information on sample size on both sides not presented	Dual-position expository text (2,727 words) about genetically modified food	1) Belief change 2) Text conclusion
McCrudden & Barnes, 2016	72 male secondary high school students	One item on a 6-point Likert scale	64 participants were more accepting, 8 participants were less accepting	Eight evidence-based arguments on the topic of whether humans affect the climate	1) Argument strength rating 2) Interview data (N = 8)
McCrudden & Sparks, 2014	45 male (ages 14–15) 10th grade high school students	1) Prior beliefs: One item on 9-point Likert scale. Higher scores indicate greater approval of widening the tunnel 2) Topic belief justification (open question)	Only participants selected for analyses that were proponents of widening the tunnel	One text (1,392 words) with 10 arguments interleaved presentation, five supporting and five opposing the widening of the tunnel	1) Belief change 2) Belief justification 3) Interview data (N = 8)
Van Strien et al., 2014	61 11th grade students	Six items assessed prior attitudes on 5-point Likert scale. High scores represent the view that violent videogames do not negatively affect behavior	Information on sample size on both sides not presented Only one essay negatively biased (excluded)	13 texts on the relationship between violent video games and aggressive behavior with a total number of 2,984 words (one neutral, six texts “not harmful”, six texts “harmful”)	Essay task analyzed for 1) Perspective (neutral, slightly or strongly biased, biased, other) 2) Position (positively or negatively biased, neutral) 3) Origin of information (borrowed, added, transformed)
Van Strien et al., 2016	79 university students	1) Prior attitudes: 14 items on 5-point Likert scale 2) Attitude strength: Three items (Taber & Lodge, 2006) on 6-point Likert scale	NA	Eight websites (212-220 words) on four aspects of organic foods. For each aspect, one website was pro and one website con, each website contained one argument	1) Total fixation duration on text 2) Essay Task 3) Credibility rating
Wiley, 2005	20 psychology undergraduates, 20 graduate students, 20 law students	1) Beliefs: One item on a 10-point Likert scale 2) Extremity of beliefs based on belief ratings	24 participants believed abortion should remain legal, 16 were against taking military action in Iraq in 1991	One text about abortion (533 words) and one text on Persian Gulf War (548 words), Each text 10 supporting arguments on each side of the issue	Recall
Wiley, 2005	24 psychology undergraduates	One item on a 10-point Likert scale	In each knowledge group (high vs. low) six participants on each side of the issue	One text (545-words) with ten arguments on either side about “Is OJ Simpson guilty?”	Recall
Wiley, 2005	48 psychology undergraduates	One item on a 10-point Likert scale	In each knowledge /presentation format subgroup an equal number of subjects on each side of the issue	One text about abortion (533 words) with ten supporting arguments on each side of the issue	Recall
Wiley, 2005	48 psychology undergraduates	One item on a 10-point Likert scale	In each knowledge presentation format subgroup an equal number of subjects on each side of the issue	One text on Persian Gulf War (451 blocked, 496 interleaved words) with 10 supporting arguments on each side of the issue	Recall

FIGURE 3 Example of a text-belief consistency effect on general memory and comprehension tasks: Effects of text-belief consistency and training on situation model strength. From “Training Multiple Text Comprehension: How Metacognitive Strategies and Motivation Moderate the Text-Belief Consistency Effect” by J. Maier and T. Richter, 2014, Metacognition and Learning, Vol. 9, No. 1, p. 66. © Springer International Publishing AG. Reproduced by permission of Springer International Publishing AG. Permission to reuse must be obtained from the rightsholder.

FIGURE 4 Example of a text-belief consistency effect on comprehension processes: Effects of text-belief consistency and training on reading times (per syllable) for belief-consistent and belief-inconsistent information. From “Training Multiple Text Comprehension: How Metacognitive Strategies and Motivation Moderate the Text-Belief Consistency Effect” by J. Maier and T. Richter, 2014, Metacognition and Learning, Vol. 9, No. 1, p. 66. © Springer International Publishing AG. Reproduced by permission of Springer International Publishing AG. Permission to reuse must be obtained from the rightsholder.

*Anmarkrud, Ø., Bråten, I., & Strømsø, H. I. (2014). Multiple-documents literacy: Strategic processing, source awareness, and argumentation when reading multiple conflicting documents. Learning and Individual Differences, 30, 64–76. https://doi.org/10.1016/j.lindif.2013.01.007

 Web of Science ®Google Scholar

*Kardash, C. M., & Howell, K. L. (2000). Effects of epistemological beliefs and topic-specific beliefs on undergraduates' cognitive and strategic processing of dual-positional text. Journal of Educational Psychology, 92, 524–535. https://doi.org/10.1037/0022-0663.92.3.524

 Web of Science ®Google Scholar

*Kardash, C. M., & Scholes, R. J. (1996). Effects of preexisting beliefs, epistemological beliefs, and need for cognition on interpretation of controversial issues. Journal of Educational Psychology, 88, 260–271. https://doi.org/10.1037/0022-0663.88.2.260

 Web of Science ®Google Scholar

*Kobayashi, K. (2010). Strategic use of multiple texts for the evaluation of arguments. Reading Psychology, 31, 121–149. https://doi.org/10.1080/02702710902754192

 Google Scholar

*Kobayashi, K. (2014). Students' consideration of source information during the reading of multiple texts and its effect on intertextual conflict resolution. Instructional Science, 42, 183–205. https://doi.org/10.1007/s11251-013-9276-3

 Web of Science ®Google Scholar

*Maier, J., & Richter, T. (2013). Text-belief consistency effects in the comprehension of multiple texts with conflicting information. Cognition and Instruction, 31, 151–175.

 Web of Science ®Google Scholar

*Maier, J., & Richter, T. (2014). Training multiple text comprehension: How metacognitive strategies and motivation moderate the text-belief consistency effect. Metacognition & Learning, 9, 54–71.

 Web of Science ®Google Scholar

Maier, J., & Richter, T. (2016b). Training metacognitive strategies on multiple text: When elaboration is not enough. Manuscript in preparation.

 Google Scholar

*Mason, L., & Boscolo, P. (2004). Role of epistemological understanding and interest in interpreting a controversy and in topic-specific belief change. Contemporary Educational Psychology, 29, 103–128. https://doi.org/10.1016/j.cedpsych.2004.01.001

 Web of Science ®Google Scholar

*McCrudden, M. T., & Barnes, A. (2016). Differences in student reasoning about belief-relevant arguments: A mixed methods study. Metacognition and Learning, 11, 275–303. https://doi.org/10.1007/s11409-015-9148-0

 Web of Science ®Google Scholar

*McCrudden, M. T., & Sparks, P. C. (2014). Exploring the effect of task instructions on topic beliefs and topic belief justifications: A mixed methods study. Contemporary Educational Psychology, 39, 1–11. https://doi.org/10.1016/j.cedpsych.2013.10.001

 Web of Science ®Google Scholar

*van Strien, J. L. H., Brand-Gruwel, S., & Boshuizen, H. P. A. (2014). Dealing with conflicting information from multiple nonlinear texts: Effects of prior attitudes. Computers in Human Behavior, 32, 101–111. https://doi.org/10.1016/j.chb.2016.02.057

 Web of Science ®Google Scholar

*van Strien, J. L. H., Kammerer, Y., Brand-Gruwel, S., & Boshuizen, H. P. A. (2016). How attitude strength biases information processing and evaluation on the web. Computers in Human Behavior, 60, 245–252. https://doi.org/10.1016/j.chb.2016.02.057

 Web of Science ®Google Scholar

*Wiley, J. (2005). A fair and balanced look at the news: What affects memory for controversial arguments? Journal of Memory and Language, 53, 95–109. https://doi.org/10.1016/j.jml.2005.02.001

 Web of Science ®Google Scholar

Isberner, M. B., & Richter, T. (2014b). Does validation during language comprehension depend on an evaluative mindset? Discourse Processes, 51, 7–25.

 Web of Science ®Google Scholar