An Evaluation of the Forecasting Accuracy of Two Trend Estimation Methods from Two Computer Adaptive Tests

References

• Ardoin, S. P., Christ, T. J., Morena, L. S., Cormier, D. C., & Klingbeil, D. A. (2013). A systematic review and summarization of the recommendations and research surrounding curriculum-based measurement of oral reading fluency (CBM-R) decision rules. Journal of School Psychology, 51(1), 1–18. doi:10.1016/j.jsp.2012.09.004.
   PubMed Web of Science ®Google Scholar
• Bakeman, R. (2005). Recommended effect size statistics for repeated measures designs. Behavior Research Methods, 37(3), 379–384. doi:10.3758/bf03192707
   PubMed Web of Science ®Google Scholar
• Ball, C. R., & Christ, T. J. (2012). Supporting valid decision making: Uses and misuses of assessment data within the context of RTI. Psychology in the Schools, 49(3), 231–244. doi:10.1002/pits.21592
   Web of Science ®Google Scholar
• Berkeley, S., Bender, W. N., Gregg Peaster, L., & Saunders, L. (2009). Implementation of response to intervention: A snapshot of progress. Journal of Learning Disabilities, 42(1), 85–95. doi:10.1177/0022219408326214
   PubMed Web of Science ®Google Scholar
• Bulut, O., & Cormier, D. C. (2018). Validity evidence for progress monitoring with star reading: Slope estimates, administration frequency, and number of data points. Frontiers in Education, 3(68), 1–12. doi:10.3389/feduc.2018.00068
   Google Scholar
• Christ, T. J., & Hintze, J. M. (2007). Psychometric considerations when evaluating response to intervention. In S.R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook of Response to Intervention (pp. 93–105). Boston, MA: Springer.
   Google Scholar
• Core Team (2021). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. https://www.R-project.org/.
   Google Scholar
• Cronbach, L. J., & Gleser, G. C. (1964). The signal/noise ratio in the comparison of reliability coefficients. Educational and Psychological Measurement, 24(3), 467–480. doi:10.1177/001316446402400303
   Web of Science ®Google Scholar
• Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52(3), 219–232. doi:10.1177/001440298505200303
   PubMed Web of Science ®Google Scholar
• Ferguson, R. J., Robinson, A. B., & Splaine, M. (2002). Use of the reliable change index to evaluate clinical significance in SF-36 outcomes. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 11(6), 509–516. doi:10.1023/a:1016350431190
   PubMed Web of Science ®Google Scholar
• Fuchs, L. S., & Fuchs, D. (2011). Using CBM for progress monitoring in reading. Washington, DC: National Center on Student Progress Monitoring.
   Google Scholar
• Fuchs, L. S., Fuchs, D., & Malone, A. S. (2017). The taxonomy of intervention intensity. Teaching Exceptional Children, 50(1), 35–43.
   Web of Science ®Google Scholar
• Gelman, A., & Hill, J. (2007). Data analysis using regression and hierarchical/multileve models. New York: Cambridge University Press. doi:10.1017/CBO9780511790942
   Google Scholar
• Good, R. H., & Shinn, M. R. (1990). Forecasting accuracy of slope estimates for reading curriculum-based measurement: Empirical evidence. Behavioral Assessment, 12, 179–193.
   Google Scholar
• Hintze, J. M., Wells, C. S., Marcotte, A. M., & Solomon, B. G. (2018). Decision-making accuracy of CBM progress-monitoring data. Journal of Psychoeducational Assessment, 36(1), 74–81. doi:10.1177/0734282917729263
   Web of Science ®Google Scholar
• Honaker, J., King, G., & Blackwell, M. (2011). Amelia II: A program for missing data. Journal [Database] of Statistical Software, 45(7), 1–47. doi:10.18637/jss.v045.i07
   Web of Science ®Google Scholar
• January, S. A. A., Van Norman, E. R., Christ, T. J., Ardoin, S. P., Eckert, T. L., & White, M. J. (2018). Progress monitoring in reading: Comparison of weekly, bimonthly, and monthly assessments for students at risk for reading difficulties in grades 2–4. School Psychology Review, 47(1), 83–94. doi:10.17105/SPR-2017-0009.V47-1
   Web of Science ®Google Scholar
• Jenkins, J. R., Hudson, R. F., & Johnson, E. S. (2007). Screening for at-risk readers in a response to intervention framework. School Psychology Review, 36(4), 582–600. doi:10.1080/02796015.2007.12087919
   Web of Science ®Google Scholar
• Kim-Kang, G., & Weiss, D. J. (2008). Adaptive measurement of individual change. Zeitschrift Für Psychologie/Journal of Psychology, 216(1), 49–58. doi:10.1027/0044-3409.216.1.49
   Web of Science ®Google Scholar
• Kowarik, A., & Templ, M. (2016). Imputation with the R package VIM. Journal of Statistical Software, 74(7), 1–16. doi:10.18637/jss.v074.i07
   Web of Science ®Google Scholar
• Marston, D., & Deno, S. L. (1990). All charting of student performance is not precision teaching: A response to Koorland and Nelson. Exceptional Children, 57(1), 73–75. doi:10.1177/001440299005700110
   Web of Science ®Google Scholar
• Mercer, S. H., Lyons, A. F., Johnston, L. E., & Millhoff, C. L. (2015). Robust regression for slope estimation in curriculum-based measurement progress monitoring. Assessment for Effective Intervention, 40(3), 176–183. doi:10.1177/1534508414555705
   Web of Science ®Google Scholar
• Newell, K. W., & Christ, T. J. (2017). Novice interpretations of progress monitoring graphs: Extreme values and graphical aids. Assessment for Effective Intervention, 42(4), 224–236. doi:10.1177/1534508417694855
   Web of Science ®Google Scholar
• Renaissance Learning. (2021a). Star assessments™ for math technical manual. https://help.renaissance.com/US/PDF/SM/SMRPTechnicalManual.pdf.
   Google Scholar
• Renaissance Learning. (2021b). Star assessments™ for reading technical manual. https://help.renaissance.com/US/PDF/SR/SRRPTechnicalManual.pdf.
   Google Scholar
• Sen, P. K. (1968). Estimate of the regression coefficient based on Kendall’s tau. Journal of the American Statistical Association, 63(324), 1379–1389. doi:10.1080/01621459.1968.10480934
   Web of Science ®Google Scholar
• Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill improvement. In A. Thomas, & J. Grimes (Eds.), Best practices in school psychology (pp. 141–157). Bethesda, MD: National Association of School Psychologists.
   Google Scholar
• Shinn, M. R., Good, R. H., III., & Stein, S. (1989). Summarizing trend in student achievement: A comparison of methods. School Psychology Review, 18(3), 356–370. doi:10.1080/02796015.1989.12085432
   Web of Science ®Google Scholar
• Silberglitt, B., Parker, D., & Muyskens, P. (2016). Assessment: Periodic assessment to monitor progress. In Jimerson, S. R., Burns, M. K., & VanDerHeyden, A. M. (Eds.), Handbook of response to intervention (pp. 271–291). Springer.
   Google Scholar
• Stecker, P. M., Fuchs, D., & Fuchs, L. S. (2008). Progress monitoring as essential practice within response to intervention. Rural Special Education Quarterly, 27(4), 10–17. doi:10.1177/875687050802700403
   Google Scholar
• Van Norman, E. R., & Forcht, E. R. (2022). Accuracy of common progress monitoring decision rules to evaluate response to instruction with two computer adaptive tests [Unpublished manuscript].
   Google Scholar
• Van Norman, E. R., & Nelson, P. M. (2019). The influence of trend estimation method on forecasting curriculum-based measurement of reading performance. Journal of School Psychology, 74, 44–57. doi:10.1016/j.jsp.2019.04.001
   PubMed Web of Science ®Google Scholar
• Van Norman, E. R., & Ysseldyke, J. E. (2020). The impact of data collection frequency and trend estimation method on the consistency of growth estimates from two computer-adaptive tests. School Psychology Review, 49(1), 20–30. doi:10.1080/2372966X.2020.1716634
   Web of Science ®Google Scholar
• Vannest, K. J., Parker, R. I., Davis, J. L., Soares, D. A., & Smith, S. L. (2012). The Theil–Sen slope for high-stakes decisions from progress monitoring. Behavioral Disorders, 37(4), 271–280. doi:10.1177/019874291203700406
   Web of Science ®Google Scholar
• Weiss, D. J., & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21(4), 361–375. doi:10.1111/j.1745-3984.1984.tb01040.x
   Web of Science ®Google Scholar
• Wilcox, R. R. (2001). Fundamentals of modern statistical methods: Substantially improving power and accuracy. New York, NY: Springer.
   Google Scholar
• Wilcox, R. R. (2004). Some results on extensions and modifications of the Theil-Sen regression estimator. The British Journal of Mathematical and Statistical Psychology, 57(Pt 2), 265–280. doi:10.1348/0007110042307230
   PubMedGoogle Scholar
• Zuidema, S. U., Buursema, A. L., Gerritsen, M. G., Oosterwal, K. C., Smits, M. M., Koopmans, R. T., & de Jonghe, J. F. (2011). Assessing neuropsychiatric symptoms in nursing home patients with dementia: Reliability and reliable change index of the neuropsychiatric inventory and the cohen‐mansfield agitation inventory. International Journal of Geriatric Psychiatry, 26(2), 127–134. doi:10.1002/gps.2499
   PubMed Web of Science ®Google Scholar