Group comparisons of mathematics performance from a cognitive diagnostic perspective

Abstract

Traditional comparisons of test score means identify group differences in broad academic areas, but fail to provide substantive description of how the groups differ on the specific cognitive attributes required for success in the academic area. The rule space method (RSM) allows for group comparisons at the cognitive attribute level, which consists of the cognitive knowledge, skills, and abilities an individual can employ to solve a problem. In the current study, we extend RSM group comparison methods to include comparisons of the attribute characteristic curves (ACCs) and provide a method for estimating and plotting the ACCs using SAS. We further investigated Taiwanese mathematics performance on TIMSS-1999 by comparing cognitive attributes between students of different achievement levels as well as between male and female students. The results indicated the highest and lowest achieving students differed most on mastery probabilities for Algebra (C3), Open-ended items (S10), and Rule application in algebra (P4). Gender differences in mathematical skills were quite minimal for Taiwanese students. Detailed discussion of these findings is provided in the paper.

Keywords:

• rule space methodology
• attribute characteristic curve
• group comparisons
• mathematics assessment