The Language Factor in Elementary Mathematics Assessments: Computational Skills and Applied Problem Solving in a Multidimensional IRT Framework

Abstract

The results of an exploratory study into measurement of elementary mathematics ability are presented. The focus is on the abilities involved in solving standard computation problems on the one hand and problems presented in a realistic context on the other. The objectives were to assess to what extent these abilities are shared or distinct, and the extent to which students' language level plays a differential role in these abilities. Data from a sample of over 2,000 students from first, second, and third grade in the Netherlands were analyzed in a multidimensional item response theory (IRT) framework. The latent correlation between the two ability dimensions (computational skills and applied mathematics problem solving) ranged from .81 in grade 1 to .87 in grade 3, indicating that the ability dimensions are highly correlated but still distinct. Moreover, students' language level had differential effects on the two mathematical abilities: Effects were larger on applied problem solving than on computational skills. The implications of these findings for measurement practices in the field of elementary mathematics are discussed.

ACKNOWLEDGMENTS

The research was supported by CITO, National Institute for Educational Measurement in the Netherlands. I am indebted to Jan Janssen from CITO for collecting the data, Ronald Krom from CITO for his advice on scoring the linguistic complexity of the items, Rinke Klein Entink for programming the MCMC-algorithm in R, and Norman Verhelst, Kees van Putten, Willem Heiser, and Claire Stevenson for their helpful suggestions.

Notes

¹With the general term “ability”, that is used throughout the article, I refer to a dimension quantifying different levels of performance or achievement, on which individuals may differ.

²The number of missing data on the reading comprehension measure and/or the home language variable was substantial, with 109, 130, and 120 students with missing data on one or both of the predictor variables in grade 1, 2, and 3, respectively. The potential effects of these missing data were investigated by estimating the measurement MIRT models based on the data of students who had non-missing scores on both student background variables, that is with N = 540 in grade 1, N = 592 in grade 2, and N = 544 in grade 5, and comparing these estimates with those of the models based on all students as reported in the previous section. The item parameter estimates and of these models correlated very highly . That implied that the same measurement model holds for the complete sample as for the sample of students without missing values on the background variables, and the model parameters are thus robust against leaving out students with missing data.