Separating Cognitive and Content Domains in Mathematical Competence

Abstract

The present study investigates the empirical separability of mathematical (a) content domains, (b) cognitive domains, and (c) content-specific cognitive domains. There were 122 items representing two content domains (linear equations vs. theorem of Pythagoras) combined with two cognitive domains (modeling competence vs. technical competence) administered in a study with 1,570 German ninth graders. A unidimensional item response theory model, two two-dimensional multidimensional item response theory (MIRT) models (dimensions: content domains and cognitive domains, respectively), and a four-dimensional MIRT model (dimensions: content-specific cognitive domains) were compared with regard to model fit and latent correlations. Results indicate that the two content and the two cognitive domains can each be empirically separated. Content domains are better separable than cognitive domains. A differentiation of content-specific cognitive domains shows the best fit to the empirical data. Differential gender effects mostly confirm that the separated dimensions have different psychological meaning. Potential explanations, practical implications, and possible directions for future research are discussed.

Notes

¹ Content domains can be specified at different levels (i.e., with different grain size), such as mathematics versus science (subject), algebra versus geometry (content area), or linear equations versus theorem of Pythagoras (content unit). The present article does not deal with the subject level, as it is focused on mathematics. In our research review, the term content domain is used for mathematical areas as well as units, whereas our own study focuses on two specific content domains defined on the level of units.

² It should be noted that other findings also exist. In contrast to Hyde et al. (1990), findings from the TIMSS context (Mullis, Martin, & Foy, 2008) show an advantage of boys in the content domain of number sense but not in geometry. Adversely, girls outperformed boys in geometry, data and chance, and algebra. The superiority of girls in algebra has also been demonstrated by Kaiser and Steisel (2000, also based on TIMSS data) and in the meta-analysis of Lindberg, Hyde, Petersen, and Linn (2010).

³ Although the described findings on problem solving and computation are well documented, it should be noted that other findings have also been reported (e.g., Kaiser & Steisel, 2000).

⁴ In contrast, the content-specificity of very basic cognitive processes (like judgment and decision making) is currently strongly debated in various fields of psychology (for an overview, see Roberts, 2007). For educational assessment purposes, however, the content-specificity of less basic, more complex cognitive assessment categories (such as Niss's, 2003, cognitive domains) appears to be of greater practical relevance.

⁵ Beta-coefficients were standardized using the variances of the respective latent outcome variables (y-standardization).

⁶ These findings are in line with previous studies that show an advantage of boys in geometry and a relative strength of girls in algebra (e.g., Hyde et al., 1990). The finding that within the domain of linear equations girls and boys did equally well in technical but not in modeling competence (here, boys outperformed girls) corresponds to the well documented advantage of boys in problem solving (e.g., Hyde et al., 1990) and word problems (e.g., Ryan & Chiu, 2001) and relative strengths of girls in mathematical operations (e.g., Hyde et al., 1990; Ryan & Chiu, 2001).

⁷ This could be the reason why Winkelmann and Robitzsch (2009) used MIRT models with a compensatory (instead of a noncompensatory) within-item structure (see the corresponding discussion in Winkelmann, 2009).