Comprehension, Diagram Analysis, Integration, and Interest: A Cross-Sectional Analysis

Abstract

The current study examines a multidimensional set of outcome variables to understand whether different pre-reading scaffolds influence students’ text comprehension, diagram analysis, text integration, and interest; and investigates these constructs cross-sectionally to identify any progression as students move across grades. One-hundred fifty-six 3rd through 6th grade students enrolled in a public laboratory school were randomly assigned to one of three pre-reading conditions intended to activate or build students’ topic knowledge. Students completed a series of before, during, and after reading activities while engaging with grade appropriate texts about the topics of ecosystems and living things. Results indicate that there were no significant differences between the three pre-reading conditions on any of the four constructs of interest. Students across grade levels performed well on multiple-choice comprehension questions, but not as well on diagram analysis questions or an open-ended integration task. Implications and future directions are discussed.

Keywords:

• Diagram analysis
• knowledge activation
• knowledge building
• multiple text comprehension
• reading comprehension
• reading development

The elementary years are among the most important in children’s reading development. Indeed, the reading process becomes more complex as students learn to decode words, understand a single perspective, and ultimately comprehend multiple viewpoints expressed through texts (Alexander, 2005; Alexander & DRLRL, 2012; Chall et al., 1990). Although growth in reading development throughout elementary school is presupposed, relatively few studies have directly examined this progression. In this manuscript, we report on a cross-sectional study examining the reading performance of students in grades three through six. Moreover, consistent with theoretical understanding of reading as becoming more complex across grade levels (Alexander & DRLRL, 2012), we conceptualize reading performance in four inter-­related ways. First, we examine students’ comprehension of science texts addressing two different topics (i.e., ecosystems and living things). Second, we consider students’ diagram analysis, with diagrams embedded within the textual content introduced. Third, in a unique investigation, we consider the extent to which students at different grade levels are successful in integrating, or connecting, distinct information presented across texts. Further, consistent with more comprehensive views of reading, including both motivational and dispositional factors (Guthrie et al., 2004), we examine students’ individual interest in reading, as well as their situational interest after reading each of two expository texts.

Reading Comprehension

Although a variety of factors, including decoding accuracy, fluency, and vocabulary, contribute to effective comprehension, in this study, we are concerned with students’ reading comprehension per se (Kendeou et al., 2009; Lai et al., 2014; White et al., 1990). Successful comprehension refers to students being able to generate an accurate and coherent mental model or cognitive representation of the central information or situation described in text. According to Kintsch (1988, 1998; Kintsch & Van Dijk, 1978), this process of comprehension or mental model development occurs through students’ construction and integration of the information presented in text. Construction refers to students’ parsing the textual information (e.g., words, propositions) directly included in text (i.e., construction of a text-base model). This construction includes students’ understanding of both local textual coherence and macro-structural text features. Integration refers to students combining of text-based information with prior knowledge, to fill in gaps in understanding, resulting in learners’ development of a situation model of the central issue or topic discussed in text.

Given the importance of readers’ background knowledge in constructing a situation model, we examine how various scaffolds might support 3^rd through 6^th-grade students in activating and building knowledge during reading. Multiple types of knowledge, including cultural, linguistic, strategic, or metacognitive knowledge, may serve as scaffolds for comprehension (e.g., Afflerbach et al., 2020; Alexander et al., 1991; Hattan & Lupo, 2020). However, in this study, we focus on the role of topic or content knowledge in supporting comprehension and on the role of content knowledge activation, in particular. Prior (content) knowledge activation entails cueing the retrieval from memory of what the reader already knows about a given topic or concept (Hattan et al., 2023; Hattan et al., 2015). Such retrieval can be elicited through instructional supports such as class discussions, the introduction of visual representations, or targeted questioning. One common knowledge activation technique, referred to as mobilization, is when students are prompted to bring to mind everything they know about a particular topic prior to reading (Peeck et al., 1982), such as “what do you know about the solar system?” However, there are times when students may not have sufficient topic knowledge for mobilization to be effective; in those cases, an alternative strategy, referred to as knowledge building, may be more appropriate. Knowledge building involves pre-teaching or introducing topic-relevant information prior to reading.

Previous studies comparing the process of knowledge activation to knowledge building have yielded conflicting results. For example, Lupo et al. (2020) found that students performed significantly better on comprehension assessments when they activated their knowledge via a Know-Want to Know-Learned chart (Ogle, 1986) compared to building their knowledge via a Listen-Read-Discuss (Manzo & Casale, 1985) approach. Conversely, Dole et al. (1991) found that students’ comprehension was significantly stronger when teachers utilized a knowledge building rather than a knowledge activation technique. The knowledge building instructional technique required that teachers directly explain relevant textual information prior to reading, whereas the knowledge activation technique involved an interactive strategy where the teacher facilitated class discussions meant to elicit students’ prior topic knowledge.

Given these conflicting findings, in this study we examine the effectiveness of three different types of scaffolds, as mechanisms for prior topic knowledge activation and knowledge building, for students across four elementary grades (i.e., 3^rd through 6^th). Moreover, consistent with contemporary views of reading (Afflerbach & Cho, 2009), we examine not only students’ text-based comprehension, but also their (a) reasoning about diagrams embedded in text, (b) abilities to integrate information across texts, and (c) situational interest after reading two expository science texts.

Expository Text and External Representations

As learners progress in school, they are increasingly asked to learn not only from narrative texts, but also from expository texts. For instance, the National Assessment of Educational Progress (NAGB, 2010) and the Common Core State Standards (CCSS, 2017) recommend that 4^th-grade students read 50% fiction, or narrative, texts and 50% nonfiction, or expository, texts, with these percentages shifting to 30% narrative and 70% expository texts by 12^th-grade. Expository, or informational, texts differ from their narrative counterparts in a number of ways (Coté et al., 1998; McNamara et al., 2011). For one, expository texts often introduce students to complex or abstract concepts, about which students may have limited prior topic knowledge, using technical or specialized vocabulary. For another, expository texts, unlike their narrative counterparts, have more limited cohesion or lack in intra-textual connectedness when elements of a story are not linked via the actions of a character. Indeed, expository texts, as compared to their narrative counterparts, require more self-explanation and inferencing on the part of learners to fill in conceptual gaps in text coherence (Best et al., 2008). This may be why expository text comprehension has been found to be more challenging for students than reading narrative texts (Best et al., 2008; Graesser et al., 2003) and more demanding of world knowledge vis-à-vis decoding skills alone (Best et al., 2008; Wolfe & Mienko, 2007).

In this study, we examine students’ comprehension of two expository texts in science. In particular, we examine a challenge unique to learning from such texts, namely that expository texts, particularly those in science, often include non-textual content, like charts, graphs, and diagrams, referred to as external representations (Ainsworth, 2008). External representations, or non-textual elements presented alongside text, serve a variety of functions (Ainsworth, 2006). These include providing complementary information to that introduced in text, constraining the interpretation of textual information, and aiding comprehension by illustrating abstract concepts or facilitating students’ connection formation (e.g., relating concepts to one another). Despite these intended facilitative functions, students have been found to struggle with learning from external representations, either because they ignore such representations, even when these are explicitly referred to in text, or because they do not yet have the ability to build appropriate inferences between these varying modalities (Butcher, 2006; Cromley et al., 2013; Cromley et al., 2010; Schwonke et al., 2009). Nevertheless, students’ learning from external representations has primarily been examined among high school and university students (see Van Meter & Stepanik, 2020 for a review), rather than in younger, elementary-aged samples. In this study, in addition to examining students’ comprehension of two expository texts in science, we further examine how students reason about diagrams presented as complementary and illustrative external representations.

Integration Across Texts

In addition to reading throughout elementary school requiring that students increasingly comprehend expository texts, such reading further requires that students read more than one text to understand complex topics. That is, as students are introduced to increasingly complex and multidimensional content, information about such content cannot be expected to be contained within one text alone. Rather, students must consult multiple texts and form connections among these texts for deep-level understanding (Britt et al., 2012). This need is demonstrated by Firetto and Van Meter (2018) who asked college students to read texts about the endocrine and urinary systems and examined the extent to which students’ written responses integrated information across texts or described connections across systems. Firetto and Van Meter found only 5% of biology undergraduates to be able to compose written responses demonstrating cross-system integration, with this proportion rising to only 32% when students received task instructions and scaffolds (i.e., a graphic organizer) intended to foster integration. This reflects a need to better support students in understanding how scientific concepts work together and interact with one another, based on information presented across texts. Similar limitations in undergraduates’ multiple text integration have likewise been found across domains (Britt & Aglinskas, 2002; Cromley et al., 2021; List et al., 2017).

Investigations of younger students’ multiple text integration have been rare (Barzilai et al., 2018), although some notable exceptions do exist. For instance, Wolfe and Goldman (2005) found that 6^th-grade students were able to form inter-textual connections when reading two isomorphic, or parallel structured, texts presenting conflicting explanations for the fall of the Roman Empire, albeit to a limited extent. Kiili et al., (2020) likewise found that 6^th-grade students were limited in their multiple text integration when asked to read a set of four texts to decide whether or not to add a soda vending machine to their school. Limitations in integration included the substantial number of students considering only one text or no texts at all in justifying their soda vending machine preferences (29.7%) and the limited number of inter-textual connections that students formed (M = 0.46), particularly in comparison to information directly copied or paraphrased from texts (M = 2.20). Moreover, students’ overall degree of integration (including intra-textual integration, inter-textual integration, and connection of text-based information with prior knowledge) was found to be significantly associated with an independent measure of reading comprehension, although the effect was small.

Interest

Given work documenting the important role of interest and motivation in students’ reading achievement, particularly as students progress throughout elementary school, we further examine students’ individual and situational interest (Guthrie et al., 2005; Guthrie & Wigfield, 1999). Although individual interest is a more stable construct and not easily manipulated, situational interest is a temporary, emotional, and effortless state of engagement that can be influenced by environmental factors (Hidi & Renninger, 2006; Schiefele, 2009). Schraw et al. (2001) suggest that teachers can increase students’ situational interest through purposeful instructional techniques and planning, such as choosing texts that students know something about or offering meaningful choices to students. However, there may be times when teachers would like all students to read the same text, regardless of their prior topic knowledge. Therefore, in addition to examining students’ a priori individual interest in reading, we were further curious as to whether different pre-reading knowledge scaffolds might influence students’ situational interest when reading expository science texts, particularly as interest and other motivational factors have been found to support students’ engagement in the cognitively demanding processing needed for comprehension (Ainley et al., 2002; List & Alexander, 2017).

Present study

In the current study, we examine upper elementary students’ (a) text comprehension, (b) diagram analysis, (c) integration, and (d) interest in several ways. First, we investigate whether different forms of scaffolded knowledge activation or knowledge building led to different results on each of these target outcomes. In particular, we compare the facilitative effects of three distinct pre-reading activities: (a) open-ended questioning (i.e., knowledge activation, KA), (b) presenting students with a diagram to support activation (i.e., scaffolded/cued knowledge activation, CKA), and (c) presenting students with both a diagram and a short paragraph foregrounding text-relevant content (i.e., knowledge building, KB). The knowledge activation condition mirrors what is more typically seen in studies that elicit students’ topic knowledge prior to reading (e.g., Peeck et al., 1982; Hattan & Alexander, 2018), whereas the knowledge building condition, including both diagrammatic and textual frontloading, was meant to provide students with background information prior to reading, in addition to potentially supporting activation. The cued knowledge activation condition, with only a diagram provided, was intended to elicit prior knowledge activation but with more scaffolding than typical open-ended questioning alone. Additionally, the provision of a diagram was intended to (potentially) support students’ generation of diagram-based inferences (e.g., comparing plant and animal cells under a microscope) even in the absence of specific prior knowledge of the topic. Moreover, we examine the facilitative effects of each of these pre-reading activities, cross-sectionally, for students in grades three through six.

We have the following research questions:

1. What are the effects of pre-reading activity type (i.e., KA, CKA, KB) and grade-level (i.e., 3^rd through 6^th) on students’ reading comprehension?

2. What are the effects of pre-reading activity type (i.e., KA, CKA, KB) and grade-level (i.e., 3^rd through 6^th) on students’ diagram analysis?

3. What are the effects of pre-reading activity type (i.e., KA, CKA, KB) and grade-level (i.e., 3^rd through 6^th) on students’ text integration performance?

4. What are the effects of pre-reading activity type (i.e., KA, CKA, KB) and grade-level (i.e., 3^rd through 6^th) on students’ situational interest after reading?

5. What is the nature of the associations among reading comprehension, diagram analysis, text integration, and interest for students in grades three through six?

We hypothesized that students in the KB condition, which provided the highest level of scaffolded support, would outperform students in the KA or CKA conditions. Additionally, we hypothesized that there would be developmental differences across grade levels. In other words, students in 6^th-grade would outperform students in 5^th, 4^th, and 3^rd-grades on comprehension, diagram analysis and text integration measures, with 5^th-grade students outperforming 4^th and 3^rd-grade students and 4^th grade students outperforming 3^rd-grade students. However, since interest tends to decline as students advance in their schooling experiences (Wigfield et al., 2016), we expected that 3^rd-grade students would have a higher level of interest than students in 4^th, 5^th, and 6^th-grades, with 6^th-grade students demonstrating the lowest level of interest.

Methods

Participants

Participants were 156 elementary school students enrolled in a laboratory school affiliated with a mid-sized, public university in the Midwestern United States. Students enrolled in the school primarily reside in two local districts. Students of all academic abilities are admitted to the school and the school is free for families. There are two classes of students per grade level. Grades one through four include two general education teachers per grade level, with one teacher per class. Starting in grade five, students follow a rotating schedule and have a different teacher for each of the main content areas (Language Arts, Math, Science, and Social Studies).

On average, students were 10.16 years old (SD = 1.22), ranging from age 8 to 12. In particular, 19.87% (n = 31) of students were in 3^rd-grade, 28.21% (n = 44) were in 4^th-grade, 25.00% (n = 39) were in 5^th-grade, and 26.92% (n = 42) were in 6^th-grade. The sample was evenly split by gender (49.36% male and female, n = 77), with two students declining to report gender. Across grade levels, 76.92% (n = 120) of students identified as White, 3.85% (n = 6) were Black, 3.85% (n = 6) were Asian, 2.56% (n = 4) were Latin, and 11.54% (n = 18) of students reported biracial or multiracial status. Two students did not report race/ethnicity. Three students were excluded from analyses because they were assigned to two different conditions for the two science topics, such that our analysis sample included 153 students.

All 3^rd through 6^th-grade students at the school were invited to participate in the study. A parental letter and consent form were sent home with each student. Additionally, students assented to participate in the study and were informed that their participation was voluntary and that they would not be in trouble if they did not participate. All students were rewarded with a class pizza party as a thank you for their hard work.

Procedures

This study adopted a 4 × 3 mixed-effects design, with pre-reading activity (i.e., KA, CKA, or KB) and grade level (i.e., 3^rd, 4^th, 5^th, or 6^th) serving as between-subjects factors. Students were asked to complete a series of four tasks (See Figure 1).

Figure 1. Study procedures.

First, students were prompted to complete introductory activities, asking them to report demographic information and prior knowledge, corresponding to each of the two science topics examined in this study (i.e., ecosystems and living things), as well as individual interest in reading. Second, students were asked to complete a reading activity for the ecosystems topic. This involved four sub-parts. These were: (a) students completing a knowledge activation (KA), cued knowledge activation (CKA), or knowledge building pre-reading task (KB), corresponding to their assigned experimental condition; (b) reading an expository text on ecosystems, including an external representation; (c) responding to a series of multiple-choice and open-ended questions tapping comprehension and diagram analysis; and (d) reporting situational interest for the reading topic. Third, students were asked to complete this same reading activity, including KA, CKA, or KB, reading, comprehension and diagram analysis performance questions, and situational interest for the living things topic. Fourth and finally, students were asked to complete a multiple text integration task by responding to a series of multiple-choice and open-ended questions, which prompted students to draw connections across the two topics that they read about.

Students completed all research activities in small groups (i.e., two to seven students), with the researcher present. If a student was absent on the day of the study, they were pulled into a different small group during subsequent days. Experimental assignment was done at the individual level, with each student within a class randomly assigned to a pre-reading condition, with the intent of having equal groups. Further, students were assigned to the same condition for both science topics examined (e.g., a student would have been in the KA condition for both the ecosystems and living things topics). Study sessions took approximately one hour to complete.

Measures

Measures are described, first, for the introductory activities and then for the two reading activities about ecosystems and living things, followed by the integration activity.

Introductory Activities

The introductory activities first asked students to report demographic information.

Prior Topic Knowledge

Students were asked to respond to two open-ended questions to assess their prior knowledge of each of the two topics examined in this study (i.e., ecosystems and living things). In particular, students were asked to respond to the questions: what is an ecosystem and what are some characteristics of living things. Responses to each question were coded according to the number of correct idea units included. Specifically, students’ responses to the ecosystems question could have included a maximum of three idea units for specifying that ecosystems (a) consisted of living and non-living things, (b) were related to one another, and (c) co-existed within a particular environment. For the living things question, students could have included a maximum of four correct idea units. For example, students earned points if they identified that living things (a) had needs for oxygen, food, water, and shelter; (b) grew, (c) reproduced, and (d) responded to their environment. Students averaged 1.06 idea units (SD = 1.07, range = 0 to 4) for the ecosystems topic and 1.04 idea units (SD = 0.56, range = 0 to 3) for the living things topic. Across both topics, students averaged 2.11 idea units (SD = 1.26, range = 0 to 6).

Individual Interest in Reading

Further, students were asked to report their interest in reading by responding to the items: I am interested in reading, I enjoy reading informational texts, and I enjoy reading stories using a five-point Likert scale. Cronbach’s alpha reliability for reading domain interest, across these items, was 0.76.

Reading Activities

Students were asked to complete two reading activities, one associated with the ecosystems topic and the other associated with the living things topic. Each reading activity involved (a) a pre-reading KA, CKA, or KB task and an eliciting of students’ situational interest, (b) a reading task, (c) students’ completion of measures of text comprehension and diagram analysis, and (d) students reporting their situational interest.

Pre-Reading Activities

Students were randomly assigned to one of three pre-reading conditions.

Knowledge activation

Students were asked to respond to an open-ended question intended to elicit prior topic knowledge. For the ecosystems topic, all students were given the question: what makes up an ecosystem. For the living things topic, the question was What are some similarities and differences between living and non-living things, for 3^rd-graders, and What are some similarities and differences between plant and animal cells, for students in 4^th through 6^th-grades.

Cued knowledge activation

In this condition, prior to being asked to respond to the open-ended questions that students received in the KA condition, students were first presented with a diagram intended to support response composition. For instance, prior to being asked what makes up an ecosystem, students were presented with a picture of an ecosystem. For the living things topic, students were presented with two photographs of microscope slides of plant and animal cells. Visible on these slides were similarities in the nuclei and membranes of plan and animal cells and differences in the shape, color, and structure of these (i.e., the presence of a cell wall); similarities and differences that could have been used to inform students’ responses to the open-ended knowledge activation question.

Knowledge building

Similar to the diagram CKA condition, students in the knowledge building condition were presented with a diagram meant to support students’ responses to the open-ended question. However, students were also presented with a brief text that essentially answered the question provided. For instance, for the ecosystems topic, students were provided both with a picture of an ecosystem and with a text describing its composition. Likewise, for the living things topic, the text identified similarities and differences between plant and animal cells apparent in the pictures provided. In this way, the textual frontloading condition explicitly provided students with necessary and relevant prior topic knowledge, rather than eliciting such knowledge, relative to the two other experimental conditions. See Appendix A for an example of the 4^th grade knowledge building condition.

Texts

Students were asked to read a text on each of two topics, ecosystems and living things. These topics were chosen after consultation with the teachers and the curriculum director, and after closely examining the science scope and sequence across grade levels. Rather than utilizing a particular textbook series, the school developed its own scope and sequence for the science curriculum. The topics of ecosystems and living things were covered in varying detail and sophistication across grade levels. At the time of data collection, the 3^rd-grade classes were completing a unit on ecosystems, whereas the 6^th-grade classes were in the middle of a unit on cells.

These topics were chosen for three primary reasons. First, these were topics in the domain of biology, covered across grade levels (i.e., 3^rd to 6^th-grade). Indeed, given the commonality with which these topics are taught in life science, we expected students, across grade levels, to have some relevant prior knowledge about these topics. Second, these were rich topics describing complex relationships in the natural world (i.e., interdependence in the ecosystems text; compare/contrast in the living things text). Finally, these were topics about which appropriate pictorial and diagrammatic external representations could be generated.

Texts were taken from a common science textbook series, aligned with the Next Generation Science Standards, and were modified for inclusion in this study. Texts were formatted to be uniform in length and style and were further modified for coherence by increasing sentence-to-sentence cross-referencing and adding headings, as needed. Moreover, each text included six key terms, with these bolded and defined in-text. As such, texts were created to reflect the topographic and organizational features common in science textbooks (e.g., headings, bolding).

Finally, each text included an external representation (i.e., a diagram). Diagrams were created for the purpose of this study to illustrate content in texts. For instance, the ecosystems text included a food chain, paralleling that discussed in the text. Likewise, the living things text included a diagram of a cell with labels for each of the organelles discussed in the text. Each external representation was explicitly referred to in text (e.g., Picture 1: Plant and Animal Cells).

See Table 1 for a summary of text characteristics.

Table 1. Readability statistics for the study passages.

Ecosystem
Grade level	Word count	Flesch reading ease	Flesch-Kincaid grade level	Coh-Metrix
3	480	67.96	5.80	27.08
4	525	58.82	7.16	21.99
5/6	763	55.23	8.22	17.63
Living Things
Grade Level	Word Count	Flesch Reading Ease	Flesch-Kincaid Grade Level	Coh-Metrix
3	486	78.95	4.12	30.41
4	507	86.9	3.13	31.97
5/6	740	72.83	5.40	22.56

Texts were created to be both parallel across grade levels and to increase in difficulty, with students in 5^th and 6^th-grade receiving the same study materials. For the ecosystems topic, the 3^rd-grade text described producers, consumers, and decomposers, yet the text for 5^th and 6^th-grade further included information about different types of consumers (i.e., herbivores, omnivores, and carnivores) and scavengers. Likewise, for the living things topic, wherein the text for 3^rd-grade compared living and non-living things; the text for 5^th and 6^th-grade compared plant and animal cells. The 4^th grade text constituted a hybrid, including information both about living and non-living things and about plant and animal cells, albeit to a more limited extent than the information provided to students in 5^th and 6^th-grade. That is, while the 4^th grade text only discussed nuclei, membranes, cell walls, and chloroplasts, the 5^th and 6^th-grade text further described the cytoplasm, mitochondria, and vacuoles. At the same time, the compare and contrast relationships discussed stayed the same for students in 3^rd through 6^th-grade. Paralleling greater complexity in texts across grade levels, diagram complexity also increased. For instance, the food chain represented in the 5^th and 6^th-grade text included more organisms than did the diagram for 3^rd-grade and more organelles were labeled in the 5^th and 6^th-grade diagram than in the diagram for 4^th-grade.

Reading Comprehension

After reading, students were asked to respond to a number of questions tapping comprehension. These included 12 to 14 multiple-choice items based on information presented in the texts, as well as two open-ended application questions, one for each text. Multiple-choice items were created based on key words included within each text and were scored as correct or incorrect. As was the case with texts, multiple-choice items were created to include some overlap across grades as well as to also progress in difficulty. In particular, the multiple-choice items presented to students in 4^th grade included items overlapping with those given to students in both 3^rd and 5^th/6^th grade. See Appendix B for sample reading comprehension questions. Cronbach’s alpha reliabilities were 0.48, 0.67, 0.74, and 0.70 for the multiple-choice questions for 3^rd, 4^th, 5^th, and 6^th-grade students, respectively. We attribute these lower than ideal reliabilities to the relatively limited number of items used and to the multi-dimensional nature of the scales, given that these tapped students’ comprehension of texts about two different topics (e.g., McCarthy & McNamara, 2021).

In addition to responding to multiple-choice items, students were further asked to respond to application questions in association with each text. These were scored according to the number of correct idea units they included. For instance, for the 5^th and 6^th-grade ecosystems topic, students were asked to respond to the question: What would happen to a land ecosystem if there was a disease that caused all of the carnivores to become sick and die out. Please identify two effects. Please explain what would happen to herbivores and what would happen to producers. Students’ responses were scored using a four-point scale that ranged from students providing general responses, to identifying that herbivores would increase and producers would decrease, to specifying an inverse relationship between these. For the living things topic, students in 4^th-grade were asked to respond to the following question: Claire is looking at cells under a microscope, but she is not sure whether what she sees is a plant cell or an animal cell. What are two ways that Claire can decide whether what she is seeing is a plant cell or an animal cell? Student responses were again scored using a four-point scale, according to whether students introduced one or two contrasts between plant and animal cells and whether students elaborated each of these. Students could earn up to eight points on the application questions. Interrater reliability across grade levels was 76% with discrepancies discussed and corrected.

Diagram Analysis

In addition to responding to comprehension questions, students were asked to complete a set of questions tapping diagram analysis. Two types of questions were included. First, students were asked two open-ended questions requiring them to identify information in the external representation included in the text. For instance, 5^th and 6^th-graders were asked: Please identify one example of a producer, one example of a consumer, and one example of a decomposer in Picture 1, for the ecosystems topic; while 4^th graders were asked: Please use Picture 1 to name two parts of a plant cell and two parts of an animal cell. The parts in the plant and animal cells could be the same or different, for the living things topic. Students’ diagram identification responses were scored as correct or incorrect, according to the information included in each diagram. Interrater reliability was 94% across grade levels and texts.

Second, students were asked to complete a diagram or graphic organizer based on information presented in texts for each topic. For the ecosystems topic, students were asked to complete a food chain. For the living things topic, students were asked to complete Venn diagrams comparing and contrasting the focal relations discussed in text (i.e., comparing and contrasting living and non-living things in 3^rd-grade; comparing and contrasting plant and animal cells in 4^th through 6^th-grades).

The diagram completion tasks that students were assigned increased in complexity, with age. For instance, students in 5^th and 6^th-grade were asked to identify more comparisons and contrasts in their Venn diagrams than their younger counterparts and to include more organisms in the food chains they constructed. At the same time, across grade levels, the diagram completion items that students responded to paralleled the diagrams provided in-text (e.g., students were asked to complete a food chain, similar to the one included in text). Diagram completion items were scored componentially, according to the accuracy of each element in the diagram that students were asked to complete. For instance, students received one point for each organism correctly located within a food chain. Interrater reliability was 66% across grade levels. This low reliability rate is due to a systematic error, which was discussed and resolved.

Situational Interest

After responding to the comprehension and diagram analysis questions, students were asked to report their situational interest in each reading topic. Students identified their level of agreement with four statements such as: I am interested in the topic of ecosystems and I enjoyed reading the text about living things, using a seven-point Likert scale ranging from not at all to very. Cronbach’s alpha reliability for post-reading situational interest was 0.92.

Integration Activity

After completing reading activities for both topics, students were asked to complete a measure of multiple text integration. This involved both multiple-choice items and an open-ended question. Multiple-choice items were questions constructed to tap the overlap between the topics discussed across the two texts. For instance, one integration item asked: Plant cells are most likely to be the building blocks of: (a) carnivores; (b) herbivores; (c) omnivores; (d) producers. This question required student to connect that plants’ chloroplasts allow them to make their own food, information provided in the living things text, and that making one’s own food from the sun was the definition of a producer, information provided in the ecosystems text. Reliability for the multiple-choice integration items was 0.47 for 3^rd-grade; 0.62 for 4^th-grade; 0.73 for 5^th-grade; and 0.59 for 6^th-grade.

In addition to these multiple-choice questions, students were further asked to respond to the question: You read two texts. One was about ecosystems and one was about living things. Please explain how ecosystems and living things fit together. What is the connection between them. This open-ended integration item was scored for the number of idea units students included. Interrater reliability was 96% across grade levels.

Analyses

Because all measures included a variable number of items or points used in scoring, all results are presented as percentages. Additionally, although students read texts about two different life science topics, we collapsed topics in our analyses to ensure sufficient power.

Results

Research Question 1: What Are the Effects of Pre-Reading Activity Type (i.e., KA, CKA, KB) and Grade-Level (i.e., 3^rd through 6^th) on Students’ Reading Comprehension?

We conducted two mixed-effects (3 × 4) ANOVAs for the two reading comprehension outcomes examined (i.e., scores on the multiple-choice items and the open-ended application questions). Condition was a between-subjects factor with three levels (i.e., KA, CKA, and BK) and grade was a between-subjects factor with four levels (i.e., 3^rd through 6^th).

Multiple Choice

For the multiple-choice comprehension questions, the mixed-effects ANOVA showed no statistically significant main effect for condition (p = 0.57). However, there was a significant main effect for grade level [F(3, 141)=11.08, p < 0.001, $\eta$² = 0.19], with no interaction effect between condition and grade identified (p = 0.25). Post-hoc analyses using Tukey’s HSD determined that, overall, 6^th-grade students performed significantly better (M = 0.86, SD = 0.14) than students in 3^rd-grade (M = 0.73, SD = 0.15, p < 0.01) and 5^th-grade (M = 0.67, SD = 0.22, p = 0.01), while 4^th-grade students performed significantly better (M = 0.84, SD = 0.16) than students in 3^rd grade (p = 0.03) and 5^th-grade (p < 0.001), with no significant difference with 6^th-graders. Please see Table 2 for descriptive information.

Table 2. Performance across grade levels.

	Comprehension MC		Application question
	M	SD	M	SD
3rd grade	0.73	0.15	0.48	0.15
4th grade	0.84	0.16	0.53	0.22
5th grade	0.67	0.22	0.56	0.26
6th grade	0.86	0.14	0.73	0.22
	Diagram Identification		Diagram Completion
	M	SD	M	SD
3rd grade	0.72	0.28	0.61	0.24
4th grade	0.61	0.31	0.73	0.21
5th grade	0.58	0.28	0.69	0.20
6th grade	0.74	0.19	0.78	0.13
	Integration MC		Integration Open-Ended
	M	SD	M	SD
3rd grade	0.72	0.22	0.22	0.18
4th grade	0.71	0.28	0.35	0.28
5th grade	0.61	0.32	0.17	0.13
6th grade	0.78	0.23	0.29	0.21

Note: Because different measures included different numbers of items or point options, all scores are presented as percentages.

Application

For the open-ended application questions, the ANOVA did not find a significant main effect for condition (p = 0.32) but did find a significant main effect for grade level [F(3, 141)=9.34, p<.001, $\eta$² = 0.17]. Post-hoc analyses using Tukey’s HSD determined that, overall, 6^th grade students performed significantly better (M = 0.73, SD = 0.22) than students in 3^rd grade (M = 0.48, SD = 0.15, p < 0.001), 4^th grade (M = 0.53, SD = 0.22, p < 0.001), and 5^th grade (M = 0.56, SD = 0.26, p = 0.01). No other significant differences were found. Please see Table 2 for descriptive information.

Research Question 2: What Are the Effects of Pre-Reading Activity Type (i.e., KA, CKA, KB) and Grade-Level (i.e., 3^rd through 6^th) on Students’ Diagram Analysis?

We further conducted two mixed-effects (3 × 4) ANOVAs to examine differences in diagram analysis (i.e., including diagram identification and diagram completion questions) across conditions and grade levels. Again, condition and grade level were between-subjects factors.

Diagram Identification

We first examined differences in diagram identification performance. The ANOVA did not find a significant main effect for condition (p = 0.99), but did find a significant main effect for grade level [F(3, 141)=3.54 p = 0.02, $\eta$² = 0.07]. There was no significant interaction effect between condition and grade level (p = 0.36). Post-hoc analyses using Tukey’s HSD determined that, overall, 6^th grade students performed significantly better (M = 0.74, SD = 0.19) than students in 5^th grade (M = 0.58, SD = 0.28, p = 0.04) with no additional significant differences between grade levels identified. Please see Table 2 for descriptive information.

Diagram Completion

We then examined students’ performance on a diagram completion task, asking them to use text-based information to complete a corresponding graphic organizer (i.e., Venn diagram or food chain). The ANOVA found no significant main effect for condition (p = 0.76) but found a significant main effect for grade level [F(3, 141)=4.38, p = 0.01, $\eta$² = 0.09]. There was no interaction effect between pre-reading activity condition and grade level (p = 0.91). Post-hoc analyses using Tukey’s HSD determined that 6^th-grade students had significantly better performance (M = 0.78, SD = 0.13) than students in 3^rd-grade (M = 0.61, SD = 0.24, p = 0.003). Please see Table 2 for descriptive information.

Research Question 3: What Are the Effects of Pre-Reading Activity Type (i.e., KA, CKA, KB) and Grade-Level (i.e., 3^rd Through 6^th) on Students’ Integration Performance?

We conducted two mixed effects (3 × 4) ANOVAs to examine differences between conditions and grade levels in multiple text integration, with integration assessed via both multiple-choice items and an open-ended question.

Multiple-Choice Integration Questions

For the multiple choice items, the ANOVA did not find significant differences across conditions (p = 0.84) or grade levels p = 0.06). There was also no significant interaction effect (p = 0.64).

Open-Ended Integration Question

For the open-ended integration question, the ANOVA found no significant difference across conditions (p = 0.51), but did find a significant main effect for grade levels [F(3, 141)=5.49, p = 0.001, $\eta$²=0.11]. Post-hoc analyses using Tukey’s HSD determined that 4^th grade students (M = 0.35, SD = 0.28) had significantly better performance than students in 3^rd (M = 0.21, SD = 0.19, p = 0.05) and 5^th grade (M = 0.17, SD = 0.13, p = 0.001). Please see Table 2 for descriptive information.

Research Question 4: What Are the Effects of Pre-Reading Activity Type (i.e., KA, CKA, KB) and Grade-Level (i.e., 3^rd Through 6^th) on Students’ Individual Interest in Reading and Situational Interest After Reading?

We first conducted a one-way ANOVA to examine cross-grade differences in individual interest in reading. The ANOVA found a significant difference across grade levels [F(3, 149)=8.35, p < 0.001, $\eta$²=0.14]. Post-hoc analyses using Tukey’s HSD determined that 4^th-grade students (M = 5.95, SD = 0.76) had significantly higher overall reading interest than students both in 5^th (M = 4.88, SD = 1.62, p = 0.001) and 6^th-grade (M = 4.68, SD = 1.29, p < 0.001).

We further conducted a mixed-effects (3 × 4) ANOVA to examine differences in situational interest across conditions and grade levels, after students completed the pre-reading activities and read the two biology texts. As with other analyses, condition (i.e., KA, CKA, BK) and grade level (i.e., 3^rd through 6^th) were between-subjects factors. The ANOVA found no significant main effect for condition (p = 0.83). However, the ANOVA found a significant main effect for grade level [F(3, 141)=7.36, p < 0.001, $\eta$²=0.14] with no interaction effect identified (p = 0.48). Post-hoc analyses using Tukey’s HSD determined that, overall, 6^th-grade students (M = 4.02, SD = 1.53) had significantly lower interest than students in 3^rd (M = 5.15, SD = 1.25, p = 0.01), 4^th (M = 5.36, SD = 1.34. p < 0.001) and 5^th-grade (M = 4.90, SD = 1.43, p = 0.03), with no other significant differences identified. Please see Table 3 for descriptive information.

Table 3. Individual and situational interest across grade levels.

	Individual interest		Situational interest
	M	SD	M	SD
3rd grade	5.40	1.36	5.15	1.25
4th grade	5.95	0.76	5.36	1.34
5th grade	4.88	1.62	4.90	1.43
6th grade	4.68	1.29	4.02	1.53

Research Question 5: What Was the Nature of the Associations Among Reading Comprehension, Diagram Analysis, and Text Integration Performance and Interest For Students in Grades Three Through Six?

We found that the (a) reading comprehension (i.e., including multiple choice comprehension items and application question performance), (b) diagram analysis (i.e., including diagram identification and diagram completion items), and (c) multiple text integration (i.e., including multiple choice and open-ended integration items) were significantly correlated with one another. Specifically, students’ performance on the reading comprehension multiple choice measure was significantly correlated with responses to the application questions [r(153)=0.45, p < 0.001], diagram task performance [diagram identification: r(153)=0.29, p < 0.001, diagram completion: r(153)=0.37, p < 0.001], and with integration performance [integration multiple choice: r(152)=0.58, p < 0.001, open-ended integration question: r(153)=0.28, p = 0.001]. Also, performance on the reading comprehension application measure was significantly associated with diagram task performance [diagram identification: r(153 = 0.23, p = 0.004, diagram completion [r(153)=0.37, p < 0.001] and integration performance in multiple choice questions [r(152)=0.41, p < 0.001].

Performance in two diagram analysis tasks (i.e., diagram identification and diagram completion) was significantly correlated with each other [r(153)=0.34, p < 0.001] as well as with integration performance. Specifically, diagram identification was associated with integration multiple choice performance [r(152)=0.33, p < 0.001], while diagram completion was correlated with both integration multiple choice [r(152)=0.22, p = 0.01] and open-ended integration [r(153)=0.21, p = 0.01] performance.

Individual interest in reading and situational interest in each reading topic were significantly associated with each other [r(153)=0.54, p < 0.001]. While situational interest was not correlated with performance on any of the comprehension, diagram analysis, or integration measures, individual interest in reading showed significant correlation with performance on the multiple choice reading comprehension [r(153)=0.19, p = 0.02], diagram completion [r(153)=0.19, p = 0.02], and multiple choice integration [r(152)=0.17, p = 0.04] questions. Please see Table 4 for correlations.

Table 4. Associations among reading comprehension, diagram analysis, multiple text integration variables, and interest.

	RC MC	RC App	Diagram ID	Diagram Comp	Integrate MC	Integrate open	Ind interest	Sit interest
RC MC	1	.447***	.291***	.370***	.579***	.276**	.186*	.056
RC App		1	.230**	.369***	.413***	.121	.099	−.001
Diagram ID			1	.341***	.332***	.075	.034	−.026
Diagram Comp				1	.221**	.211**	.191*	−.027
Integrate MC					1	.259**	.170*	.036
Integrate Open						1	.128	−.034
Ind Interest							1	.543***
Sit Interest								1

***. Correlation is significant at the 0.001 level (2-tailed).

**. Correlation is significant at the 0.01 level (2-tailed).

*. Correlation is significant at the 0.05 level (2-tailed).

Key: RC MC = Reading Comprehension Multiple Choice, RC App = Reading Comprehension Application, Diagram ID = Diagram Identification, Diagram Comp = Diagram Completion, Integrate MC = Text Integration Multiple Choice, Integrate Open = Text Integration Open-Ended, Ind Interest = Individual Interest, Sit Interest = Situational Interest.

Discussion

Our goal in conducting this study was to examine a multidimensional set of outcome variables, including assessments of comprehension, diagram analysis, multiple text integration and individual and situational interest, to understand the range of processes that students may be expected to engage when presented with expository texts in science. Specifically, we investigated the effects of varying topic knowledge activation and knowledge building scaffolds on these four constructs, as well as examined these constructs cross-sectionally to identify any progression across grades. Moreover, we were interested in the nature of the associations among the constructs. Based on results from this study, two overarching conclusions can be drawn related to knowledge activation or building and grade level differences.

These findings suggest that the brief pre-reading scaffolds introduced in this study did not influence students’ text comprehension, diagram analysis, text integration, or situational interest. These findings stand in contrast to our expectations that, given students’ relatively low levels of prior topic knowledge, additional pre-reading supports, beyond activation, would lead to stronger comprehension outcomes. There are a number of reasons why this could have occurred, all of which should be considered in light of the fact that, overall, students performed fairly well on the comprehension questions administered, with more limited performance on the diagram completion and open-ended integration tasks. There are at least three possibilities for the limited results associated with our experimental manipulation.

First, it is possible that either the open-ended topic knowledge activation questions alone were enough to activate students’ knowledge, and that the additional supports were not needed, or that deeper knowledge building supports would have enhanced students’ textual understanding. We favor the latter explanation. Although students performed fairly well on the comprehension multiple-choice, diagram identification, and integration multiple-choice questions, they did not perform as well on the open-ended application, diagram completion, and integration questions, which require deeper text processing. Further, students’ scores on the prior topic knowledge assessment were fairly low, meaning that students did not have a lot of content specific knowledge on which they could draw. Therefore, in order to move the needle on text comprehension and the other constructs, more substantial knowledge building may need to take place. Instead of brief texts, students may have benefited from lessons that delve deeply into related topics and build conceptual knowledge over time.

Another possible reason why we did not see differences across pre-reading techniques is because some work has demonstrated that knowledge activation is more useful when engaged during, rather than solely before reading. This hypothesis is supported by Kintsch’s (1998) Construction Integration Model, which emphasizes the importance of a continuous interaction between students’ prior knowledge and the text throughout the reading process. This possibility is further supported by recent research findings that a novel knowledge activation technique, prompting relational reasoning, which included during reading prompts, was more beneficial to students than typical, pre-reading knowledge activation prompts, like the ones used in our KA condition (Hattan & Alexander, 2021). Instead of activating and attempting to build students’ knowledge prior to reading, without encouraging students to go back to their initial responses and the initial scaffolds provided, future studies should include during reading supports that guide students to reflect on how their initial knowledge may be similar to or different from what they read in the text.

Third, it is important to note that all of the scaffolds were presented to students via written instructions. Although the researcher clarified students’ questions during the study, the pre-reading activities did not include teacher-led instruction or class discussions. Given the social nature of reading (Guthrie & Wigfield, 2000; Pressley et al., 1992) and the value of classroom discourse (Almasi, 1996; Murphy et al., 2018), future studies can investigate whether similar pre-reading scaffolds differentially support students’ textual understanding when done via class discussion, instead of through students individually responding to written prompts.

A second overarching conclusion is that there were significant grade level differences on seven of our eight measures, although not always in the ways we predicted. We expected that students’ performance on the comprehension, diagram analysis, and text integration measures would progress as students moved from 3^rd up through 6^th-grade, but this was not always the case. For example, 5^th-grade students (perhaps idiosyncratically) demonstrated somewhat lower levels of performance. In fact, they had the lowest scores of any grade level for the comprehension multiple-choice, diagram identification, and integration multiple-choice questions. It is possible that 5th grade students performed worse than their peers because they were tasked with reading the same texts as the 6th grade students, meaning that the texts may have been too challenging for them. Another possibility is that 5^th-grade students struggled due to contextual factors. At the school site, 3^rd and 4^th-grade students are considered part of the elementary school, while 5^th and 6^th-grade students are part of the middle school. It is possible that the transition from elementary to middle school put undue stress on this group of 5^th-grade students, resulting in lower performance when compared to other grade levels (Alspaugh, 1998).

Despite the surprising results regarding 5^th-grade students’ performance, scores on both the diagram completion and open-ended integration measures followed a more traditional developmental trajectory. Specifically, 3^rd-grade students seemed to struggle more with open-ended questions that required deeper-level processing, in comparison to their performance on the multiple-choice measures. Students at the lower grade levels may not yet know how to approach these more challenging questions that required deeper strategic processing, as compared to the multiple-choice questions asking students to identify information explicitly stated in text.

Students’ modest performance on the diagram analysis and open-ended integration tasks aligns with previous research showing the challenges that learning from external representations poses for students, despite the commonality with which such representations are included in science texts (Cromley et al., 2013; Renkl & Scheiter, 2017. This may indicate that students need more support to reason about diagrams effectively, with Cromley et al. (2013) suggesting that students need particular instruction in understanding diagrammatic conventions or the “grammar” of external representations.

When considering open-ended integration performance, students’ responses were limited. This may be the case for at least three reasons. First, abilities in integration may emerge only at later stages of schooling. At the same time, recent work (Kiili et al., 2020) suggests that students as young as 6th-grade are capable of engaging in multiple-text integration, at least to some extent. Second, students may keep conceptual knowledge about various scientific topics fairly inert, as demonstrated by students, even at the undergraduate level, experiencing difficulties integrating various organ systems in biology (Firetto & Van Meter, 2018). Third, it may be that students were relatively inexperienced with responding to this type of question. We favor this last explanation and would call for more work asking students to integrate information, not only when studying history and social studies, but science as well. Indeed, the importance of integration has been demonstrated in curricula introducing students to texts on various controversial socio-scientific topics, albeit implemented in higher grades. More generally, we would argue that students understanding the connection between the cellular structure of plants and their role as producers within an ecosystem, represent the type of foundational and integrated scientific understanding that we should seek to foster.

Finally, similar to previous studies (Kush & Watkins, 1996; McKenna et al., 1995; Wigfield et al., 2016), there was a decline in interest by 6th grade. Although we were curious as to whether the varying pre-reading tasks would influence students’ situational interest, this was not the case.

Limitations and Future Directions

Despite taking a nuanced look at a variety of variables that are crucial to text processing, there are several limitations in the current study, which should be addressed in future work. First, the pre-reading experimental conditions were all prompted via written instructions instead of through more elaborate class conversations and teacher-led instruction. Our decision to do this was twofold. For one, we were curious as to whether these simple, text-based instructions would provide a shift in students’ performance. For another, we were limited in terms of the time we were allotted in the participating classrooms. A second, related limitation, as mentioned earlier, was that the manipulations examined focused on before reading prompts, rather than incorporating during reading scaffolds. It is possible that we would have seen differences between groups had we supplemented initial knowledge activation and knowledge building with complementary during reading supports.

Finally, it would have been interesting to more deeply investigate students’ diagram analysis and abilities to integrate information across texts. Semi-structured interviews or other qualitative methodologies could help researches better understand where the breakdown in understanding occurs. Additional research should also consider what instructional approaches are most facilitative for developing young students’ abilities to analyze diagrams and develop integrated, meaningful connections across texts. Our study indicates that these processes are challenging for elementary students. Next, researchers should seek to better understand why these processes are challenging and what specific scaffolds can support students in developing these crucial literacy skills.

Conclusion

Despite these limitations, the present study has enhanced our conception of elementary students’ understanding of science texts, including their comprehension, diagram analysis, text integration, and situational interest. In particular, this study signifies that short pre-reading knowledge activation or building techniques may not be sufficient in supporting students’ deep-level text comprehension. Further, additional research should investigate how teachers can more effectively support students in analyzing external representations and integrating information across texts, given that this is a potential area for concern. Finally, the results indicate clear cross-sectional differences across grade levels, yet highlight that these differences may be more nuanced than initially perceived.

Disclosure statement

We have no known conflicts of interest to disclose.

Appendix A

4th Grade Knowledge Building Condition for Living Things Topic

All living things are made of cells. Cells are the smallest unit of living things or the building blocks for plants and animals. Plant and animal cells have some things in common. Both plant cells and animal cells have a membrane that surrounds the cell and controls what goes in and out. But, plant cells include some features that you will not find in animal cells. Plant cells have a cell wall that gives them a rectangular shape. Animal cells are round. Plant cells also have chloroplasts that have a green chemical called chlorophyll. Plants use this chemical to make food using the sun’s energy.

Below are pictures of plant and animal cells that were taken using a microscope.

Based on the pictures and text, what do plant and animal cells have in common? How are they different? Write your response in the box below.

Appendix B

Ecosystem Reading Questions for 3rd Grade

Please answer each question based on the text you read.

1. A food chain shows:

2. All of the things that one specific organism eats

3. How animals make their own food

4. How energy comes from non-living objects, like the sun

5. How energy is passed from one organism to another that eats them

6. Producers are organisms that:

7. Breaks down dead organisms

8. Do not require energy

9. Eat other organisms

10. Make their own food

11. In a food chain, plants can best be described as:

12. Consumers

13. Decomposers

14. Predators

15. Producers

16. In most food chains, energy originally comes from:

17. Animals

18. Humans

19. Plants

20. The sun

21. Organisms that eat other organisms are referred to as:

22. Producers

23. Predators

24. Decomposers

25. Consumers

26. Animals that eat both producers and consumers are referred to as:

27. Carnivores

28. Herbivores

29. Omnivores

30. Producers