iSTART: Adaptive Comprehension Strategy Training and Stealth Literacy Assessment

Abstract

The Interactive Strategy Training for Active Reading and Thinking (iSTART) game-based intelligent tutoring system (ITS) was developed with a foundation of comprehension theory and principles of learning science to improve students’ comprehension of complex scientific texts. iSTART has been shown to improve reading comprehension for learners from middle school through adulthood, particularly lower knowledge readers, through strategy instruction and game-based practice. This paper describes iSTART, the theoretical foundations that have guided iSTART development, and evidence for the feasibility of game-based practice to improve learning outcomes. This paper also introduces a novel method of assessing students’ reading comprehension through game-based literacy assessments that have been incorporated in iSTART. The development of these stealth assessments was guided by recent work emphasizing the need for rapid, dynamic, and low stakes assessments that evaluate students’ reading skills in the context of brief, dynamic games. Stealth assessments can generate estimates of multiple aspects of students’ reading comprehension quickly and within a motivating environment. The work described in this paper is a promising method to assess students’ literacy in an unobtrusive and authentic way that may lead to improved learning outcomes for students.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The research reported here was supported by the Office of Naval Research, through Grants [N00014-17-1-2300, N00014-20-1-2623, N00014-19-1-2424, and N00014-20-1-2627] and the Institute of Education Sciences, U.S. Department of Education, through Grants [R305A190050 and R305A190063] to Arizona State University. The opinions expressed are those of the authors and do not represent views of the Office of Naval Research, the Institute, or the U.S. Department of Education.

Notes on contributors

Danielle S. McNamara

Danielle S. McNamara develops educational technologies (iSTART, iSTART-ME, Coh-Metrix, Writing-Pal) and conducts research to better understand cognitive processes of comprehension, learning, text coherence, and individual differences. She has solidified herself as a one of the world’s premier experts in cognitive psychology, publishing hundreds of scholarly works.

Tracy Arner

Tracy Arner is a postdoctoral research scholar at Arizona State University. Her research interests include the development and use of instructional technologies such as multimedia instruction, intelligent tutoring systems, and artificial intelligence to improve learning outcomes for struggling students.

Reese Butterfuss

Reese Butterfuss is a postdoctoral research scholar at Arizona State University. His research focuses on the cognitive processes that underlie learning from texts, as well as improving students’ literacy skills using technology-based literacy instruction.

Ying Fang

Ying Fang is an associate professor of Faculty of Artificial Intelligence in Education at Central China Normal University. Her research interests include artificial intelligence in educational systems, the development, implementation, and assessment of intelligent tutoring systems, and promoting learning in electronic learning environments.

Micah Watanabe

Micah Watanabe is a PhD candidate in Cognitive Psychology at Arizona State University. His research is primarily on the role of educational interventions and intelligent tutoring systems in promoting conceptual change. He also studies the structure and quality of students’ prior knowledge and how that affects their learning.

Natalie Newton

Natalie Newton is a Research Specialist at Arizona State University. She leads training for expert raters to score constructed responses and contributes to other research projects through data analysis, writing, and conceptualization. Her research interests in the lab center around comprehension strategy use and how instructional prompts impact text understanding.

Kathryn S. McCarthy

Katie S. McCarthy is an Assistant Professor of Educational Psychology in the Department of Learning Sciences at Georgia State University. Her research explores the higher-order processes involved in reading comprehension and how they vary across disciplines and readers and how in-person interventions and educational technology can support learning from text.

Laura K. Allen

Laura K. Allen is an Assistant Professor of Educational Psychology at University of Minnesota. The primary aim of her research is to examine how individuals learn and communicate with text and to apply those insights to educational practice through the development of interventions and educational technologies.

Rod D. Roscoe

Rod D. Roscoe is an Associate Professor of Human Systems Engineering in the Ira A. Fulton Schools of Engineering at Arizona State University. His work combines insights from learning science, cognitive science, design science, and equity science to implement effective educational technologies that are inclusive for all learners.