The impact of students’ test-taking effort on growth estimates in low-stakes educational assessments

ABSTRACT

This study investigated the impact of students’ test-taking effort on their growth estimates in reading. The sample consisted of 7,602 students (Grades 1 to 4) in the United States who participated in the fall and spring administrations of a computer-based reading assessment. First, a new response dataset was created by flagging both rapid-guessing and slow-responding behaviours and recoding these non-effortful responses as missing. Second, students’ academic growth (i.e., daily increase in ability levels) from fall to spring was calculated based on their original responses and responses in the new dataset excluding non-effortful responses. The results indicated that students’ growth estimates changed significantly after recoding non-effortful responses as missing. Also, the difference in the growth estimates varied depending on the grade level. Overall, students’ test-taking effort appeared to be influential in the estimation of students’ reading growth. Implications for practice were discussed.

KEYWORDS:

• Growth
• reading
• test-taking effort
• rapid guessing
• slow responding
• response time effort

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 For researchers who are more familiar with the original NT10 approach, we also performed the same analyses using the NT10 method by detecting only rapid guessing (i.e., no slow responding) in the items. These results are presented in Appendix 1.

Additional information

Notes on contributors

Seyma Nur Yildirim-Erbasli

Seyma Nur Yildirim-Erbasli, MA, is a doctoral student in the Measurement, Evaluation, and Data Science programme at the University of Alberta in Edmonton, Alberta. Her research interests include item response modelling, computer-based assessments, and educational data mining.

Okan Bulut

Okan Bulut, PhD, is an Associate Professor of Measurement, Evaluation, and Data Science and a member of the Centre for Research in Applied Measurement and Evaluation at the University of Alberta in Edmonton, Alberta. His research interests include psychometrics, educational data mining, big data modelling, and digital assessments.