Are Online and Paper Tests Comparable? Evidence from Statewide K-12 Tests

References

• Backes, B., & Cowan, J. (2019). Is the pen mightier than the keyboard? The effect of online testing on measured student achievement. Economics of Education Review, 68, 89–103. doi:10.1016/j.econedurev.2018.12.007
   Web of Science ®Google Scholar
• Backes, B., Cowan, J., Goldhaber, D., Koedel, C., Miller, L. C., & Xu, Z. (2018). The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance? Economics of Education Review, 62, 48–65. doi:10.1016/j.econedurev.2017.10.004
   Web of Science ®Google Scholar
• Bennett, R. E., Braswell, J., Oranje, A., Sandene, B., Kaplan, B., & Yan, F. (2008). Does it matter if I take my mathematics test on computer? A second empirical study of mode effects in NAEP. Journal of Technology, Learning, and Assessment, 6(9).
   Google Scholar
• Brody, L., & Koh, Y. (2020, Nov 21). Student test scores drop in math since covid-19 pandemic. The Wall Street Journal. https://www.wsj.com/articles/student-test-scores-drop-in-math-since-covid-19-pandemic-11605974400.
   Google Scholar
• Brown, T., Chen, J., Ali, U., Costanzo, K., Chun, S., & Ling, G. (2015). Mode comparability study based on spring 2014 field test data. Partnership for Assessment of Readiness for College and Careers.
   Google Scholar
• Educational Testing Service, Pearson, & Measured Progress. (2016). Final technical report for 2015 administration.
   Google Scholar
• Goldberg, A. L., & Pedulla, J. J. (2002). Performance differences according to test mode and computer familiarity on a practice graduate record exam. Educational and Psychological Measurement, 62(6), 1053–1067. doi:10.1177/0013164402238092
   Web of Science ®Google Scholar
• Gordanier, J., Ozturk, O., & Zhan, C. (2023). Pencils down? Computerized testing and student achievement. Education Finance and Policy, 18(2), 232–252.
   Web of Science ®Google Scholar
• Horkay, N., Bennett, R. E., Allen, N., Kaplan, B., & Yan, F. (2006). Does it matter if I take my writing test on computer? An empirical study of mode effects in NAEP. Journal of Technology, Learning, and Assessment, 5(2). Retrieved July 29, 2007, from http://escholarship.bc.edu/jtla/vol5/2/
   Google Scholar
• Jacob, B., & Rothstein, J. (2016). The measurement of student ability in modern assessment systems. Journal of Economic Perspectives, 30(3), 85–108. doi:10.1257/jep.30.3.85
   Web of Science ®Google Scholar
• Kuhfeld, M., Tarasawa, B., Johnson, A., Ruzek, E., & Lewis, K. (2020). Learning during COVID-19: Initial findings on students’ reading and math achievement and growth. Portland, OR: NWEA Research.
   Google Scholar
• Massachusetts Department of Elementary and Secondary Education. (2016). Representative samples and PARCC to MCAS concordance studies. Malden, MA: Massachusetts Department of Elementary and Secondary Education.
   Google Scholar
• Massachusetts Department of Elementary and Secondary Education. (2017). 2017 next-generation MCAS and MCAS-Alt technical report. Malden, MA: Massachusetts Department of Elementary and Secondary Education.
   Google Scholar
• Massachusetts Department of Elementary and Secondary Education. (2018). 2018 next-generation MCAS and MCAS-Alt technical report. Malden, MA: Massachusetts Department of Elementary and Secondary Education.
   Google Scholar
• Olson, L. (2019). The new testing landscape. FutureEd. https://www.future-ed.org/wp-content/uploads/2019/09/FutureEdTestingLandscapeReport.pdf
   Google Scholar
• Pearson. (2017). Final technical report for 2016 administration.
   Google Scholar
• Powers, D. E. (1999). Test anxiety and test performance: Comparing paper‐based and computer‐adaptive versions of the GRE general test. ETS Research Report Series, 1999(2), i–32. doi:10.1002/j.2333-8504.1999.tb01813.x
   Google Scholar
• Sandene, B., Horkay, N., Bennett, R. E., Allen, N., Braswell, J., Kaplan, B., & Oranje, A. (2005). Online assessment in mathematics and writing: Reports from the NAEP technology-based assessment project, research and development series. (NCES 2005–457). National Center for Education Statistics.
   Google Scholar
• Schaeffer, G. A., Bridgeman, B., Golub‐Smith, M. L., Lewis, C., Potenza, M. T., & Steffen, M. (1998). Comparability of paper‐and‐pencil and computer adaptive test scores on the GRE® general test. ETS Research Report Series, 1998(2), i–25. doi:10.1002/j.2333-8504.1998.tb01787.x
   Google Scholar
• StataCorp. (2021). Stata statistical software: Release 17. College Station, TX: StataCorp LLC.
   Google Scholar
• Steedle, J., Pashley, P., & Cho, Y. (2020). Three studies of comparability between paper-based and computer-based testing for the ACT. ACT Research & Policy. Research Report. ACT, Inc.
   Google Scholar
• Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2007). A meta-analysis of testing mode effects in grade K-12 mathematics tests. Educational and Psychological Measurement, 67(2), 219–238. doi:10.1177/0013164406288166
   Web of Science ®Google Scholar
• White, S., Kim, Y. Y., Chen, J., & Liu, F. (2015). Performance of fourth-grade students in the 2012 NAEP computer-based writing pilot assessment: Scores, text length, and use of editing tools. Working Paper Series. NCES 2015–119. National Center for Education Statistics.
   Google Scholar