Assessment in the Time of COVID-19: Understanding Patterns of Student Disengagement during Remote Low-Stakes Testing

ABSTRACT

The arrival of the COVID-19 pandemic had a profound effect on K-12 education. Most schools transitioned to remote instruction, and some used remote testing to assess student learning. Remote testing, however, is less controlled than in-school testing, leading to concerns regarding test-taking engagement. This study compared the disengagement of students remotely administered an adaptive interim assessment in spring 2020 with their disengagement on the assessment administered in-school during fall 2019. Results showed that disengagement gradually increased across grade level. This pattern was not meaningfully different between the two testing contexts, with the exception of results for American Indian/Alaska Native students, who showed higher disengagement under remote testing. In addition, the test’s engagement feature – which automatically paused the test event of a disengaged student and notified the test proctor – had a consistently positive impact whether the proctor was in the same room as the student or proctoring was done remotely.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ Previous research with MAP Growth has found comparable levels of mean engagement between the fall and spring testing terms (Wise et al., 2010) with mean RTE slightly higher (.003) in the spring.

² Because MAP Growth is an adaptive test, many different items are administered at each item position. Hence, the values in Table 6 reflect the average rapid guessing rate at each item position set, and not the rapid guessing received by particular items. Moreover, the trends in rapid guessing rates do not reflect relative item difficulty, as the average difficulty of administered items was nearly equivalent across item positions.

³ The leveling off of the trend in Table 6 reflects both the auto-pauses’ incomplete elimination of rapid guessing as shown in Table 4 and the analyses’ exclusion of test events for which first auto-pauses occurred late in the test.

⁴ For test events that triggered auto-pauses, response times tended to initially decrease more quickly, with a much slower decrease occurring during the second half of the positions (reflecting the effects of the auto-pauses).