Measuring Reading Strategy Use

ABSTRACT

This study describes the development and validation of a multidimensional measure of preadolescent and adolescent readers’ abilities to apply reading comprehension strategies necessary for understanding challenging academic texts. The Strategy Use Measure (SUM) was designed with the intention of being pedagogically informative to the increasingly multilingual student population in the U.S. in grades 6 through 8. The SUM aims to measure four areas of knowledge and skill that are widely purported to support the use of reading strategies: (a) morphological awareness, (b) knowledge of cognates, (c) ability to relate micro- and macro- ideas within a text, and (d) the ability to use intra- and inter-sentential context clues for defining unfamiliar words. The test was developed following a principled, iterative process to instrument development, employing Rasch models and qualitative investigations to test hypotheses related to the instrument’s validity. Findings suggest promising evidence for the validity and fairness of this multidimensional measure.

Acknowledgments

The authors would like to acknowledge the substantial intellectual contribution of Dr. Janette Klingner to the development and study of Collaborative Strategic Reading. Dr. Klingner’s legacy in special education, reading comprehension strategy instruction for culturally and linguistically diverse students, and enhancing the sustainability of evidence-based and culturally responsive instructional practices will inform teachers and researchers for years to come. Dr. Janette Klingner passed away in March 2014.

Notes

¹ Claims are expressed here in terms of scale scores (i.e., person estimates from the multidimensional Rasch model described in more detail in later sections) rather than raw scores. Development of a computerized adapted test (CAT) for the SUM is currently underway using Concerto (Scalise & Allen, 2015), a free and open-source testing platform that interfaces with the catR library. Ultimately, the intention is that teachers will have access to a free, online version of the SUM that estimates and presents such scale scores along with confidence bands expressed in relation to construct maps, along with guidance documents and sample items to aid interpretation.

² The forms were designed such that each contained items encompassing the full range of expected difficulties.

³ It may be worth noting that while findings of DIF can provide evidence of potential bias or systematic differences at the item level, such analyses are not well-suited to detecting systematic differences at the level of an entire test (at least in the absence of a known external criteria), given that such differences may be equally present in all items.

⁴ Another 15% of students either skipped the question or selected “other” when reporting home language; these students were not included in DIF analyses. Another 17 primary home languages were reported in the dataset, the largest group of which (Arabic) comprised only 1% of the sample.

⁵ Correlations among raw scores are not disattenuated for measurement error, and thus are somewhat lower than model-estimated correlations among dimensions, as reflected in Table 5.

Additional information

Funding

This research was supported in part by grant U396B100143, from Investing in Innovation, U.S. Department of Education. The content is solely the responsibility of the authors and does not necessarily represent the official views of the U.S. Department of Education.