Evaluating a Model for Developing Cognitively Diagnostic Adaptive Assessments: The Case of Young Children’s Length Measurement

ABSTRACT

We report on an innovative computer-adaptive assessment, the Comprehensive Research-based Early Math Ability Test (CREMAT), using the case of 1st- and 2nd-graders’ understanding of geometric measurement. CREMAT was developed with multiple aims in mind, including: (1) be administered with a reasonable number of items, (2) identify the level(s) of thinking at which the child is operating for each developmental progression (domain) of significant mathematics, and (3) provide information about specific concepts and skills that constitute the cognitive components of thinking involved in various tasks. CREMAT relies on Q-Matrix theory and a resulting blend of sophisticated psychometric and statistical techniques (Rule Space Method, poset models) to determine not only students’ overall performance but also their specific level of thinking along developmental progressions. Results indicated that the current tests range from satisfactory to excellent in their ability to diagnose 1st- and 2nd-graders’ understanding of measurement, but also suggest revisions and implications for the approach. We conclude with a description of why CREMAT is important for early childhood practitioners’ ability to formatively assess and document children’s progress in learning math.

KEYWORDS:

• Assessment
• computer-adaptive testing
• early childhood education
• learning trajectories
• mathematics education

Acknowledgments

The findings and statements in this manuscript do not represent the views of the National Science Foundation. The authors wish to express appreciation to the school districts, teachers, and children who participated in this research, and to Douglas Van Dine, who oversaw initial data collection and helped with initial analyses.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The research reported in this manuscript was supported by the National Science Foundation under Grant No. [DRL-1313695].