Comparing the influence of various measurement error presentations in test score reports on educational decision-making

References

• American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME) . (2014). Standards for educational and psychological testing . Washington, DC: American Psychological Association.
   Google Scholar
• Auspurg, K. , & Hinz, T. (2014). Factorial survey experiments. Applications for the Social Sciences (Vol. 175). Thousand Oaks, CA: Sage Publications.
   Google Scholar
• Bannert, M. , & Mengelkamp, C. (2008). Assessment of metacognitive skills by means of instruction to think aloud and reflect when prompted. Does the verbalisation method affect learning? Metacognition and Learning , 3 , 39–58. doi:10.1007/s11409-007-9009-6
   Web of Science ®Google Scholar
• Bates, D. , Maechler, M. , Bolkers, B. , Walker, S. , Christensen, R. H. B. , Singman, H. , … Green, P. (2017). The lme4 package . Retrieved from http://r-forge.r-project.org/projects/lme4/
   Google Scholar
• Bradshaw, J. , & Wheater, R. (2009). National foundation for educational research: International survey of results reporting (OFQUAL 10/4705) . London: Office of Qualifications and Examinations.
   Google Scholar
• Brodlie, K. W. , Osoria, R. A. , & Lopes, A. (2012). A review of uncertainty in data visualization. In J. Dill , R. Earnshaw , D. Kasik , J. Vince , & P. C. Wong (Eds.), Expanding the frontiers of visual analytics and visualization (pp. 81–109). London: Springer.10.1007/978-1-4471-2804-5
   Google Scholar
• Brookhart, S. M. , & Nitko, A. J. (2008). Assessment and grading in classrooms . Upper Saddle River, NJ: Pearson Education.
   Google Scholar
• Correll, M. , & Gleicher, M. (2014). Error bars considered harmful: Exploring alternate encodings for mean and error. IEEE Transactions on Visualization and Computer Graphics , 20 , 2142–2151. doi:10.1109/TVCG.2014.2346298
   PubMed Web of Science ®Google Scholar
• Epp, C. D. , & Bull, S. (2015). Uncertainty representation in visualizations of learning analytics for learners: Current approaches and opportunities. IEEE Transactions on Learning Technologies , 8 , 242–260.
   Web of Science ®Google Scholar
• Feldt, L. S. , & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105–146). New York, NY : American Council on Education and Macmillan.
   Google Scholar
• Gardner, J. (2013). The public understanding of error in educational assessment. Oxford Review of Education , 39 , 72–92. doi:10.1080/03054985.2012.760290
   Web of Science ®Google Scholar
• Gershon, N. (1998). Visualization of an imperfect world. IEEE Computer Graphics and Applications , 18 , 43–45. doi:10.1109/38.689662
   Web of Science ®Google Scholar
• Goodman, D. P. , & Hambleton, R. K. (2004). Student test score reports and interpretive guides: Review of current practices and suggestions for future research. Applied Measurement in Education , 17 , 145–220. doi:10.1207/s15324818ame1702
   Web of Science ®Google Scholar
• Hullman, J. , Rhodes, R. , Rodriguez, F. , & Shah, P. (2011). Research on graph comprehension and data interpretation: Implications for score reporting (ETS RR-11-45) . Paper presented at the ETS Score Reporting conference, Princeton, NJ.
   Google Scholar
• Impara, J. C. , Divine, K. P. , Bruce, F. A. , Liverman, M. R. , & Gay, A. (1991). Does interpretive test score information help teachers? Educational Measurement: Issues and Practice , 10 (4), 16–18.10.1111/emip.1991.10.issue-4
   Google Scholar
• Johnson, C. R. , & Sanderson, A. R. (2003). A next step: Visualizing errors and uncertainty. IEEE Computer Graphics and Applications , 23 (5), 6–10.10.1109/MCG.2003.1231171
   Web of Science ®Google Scholar
• Kinkeldey, C. , MacEachren, A. M. , Riveiro, M. , & Schiewe, J. (2015). Evaluating the effect of visually represented geodata uncertainty on decision-making: Systematic review, lessons learned, and recommendations. Cartography and Geographic Information Science , 44 , 1–21. doi:10.1080/15230406.2015.1089792
   Web of Science ®Google Scholar
• Kinkeldey, C. , MacEachren, A. M. , & Schiewe, J. (2014). How to assess visual communication of uncertainty? A systematic review of geospatial uncertainty visualisation user studies. The Cartographic Journal , 51 , 372–386. doi:10.1179/1743277414Y.0000000099
   Web of Science ®Google Scholar
• Leitner, M. , & Buttenfield, B. P. (2000). Guidelines for the display of attribute certainty guidelines for the display of attribute certainty. Cartography and Geographic Information Science , 27 , 3–14. doi:10.1559/152304000783548037
   Google Scholar
• MacEachren, A. M. , Robinson, A. , Hopper, S. , Gardner, S. , Murray, R. , Gahegan, M. , & Hetzler, E. (2005). Visualizing geospatial information uncertainty: What we know and what we need to know. Cartography and Geographic Information Science , 32 , 139–160. doi:10.1559/1523040054738936
   Google Scholar
• MacEachren, A. M. , Roth, R. E. , O’Brien, J. , Li, B. , Swingley, D. , & Gahegan, M. (2012). Visual semiotics & uncertainty visualization: An empirical study. IEEE Transactions on Visualization & Computer Graphics , 18 , 2496–2505.10.1109/TVCG.2012.279
   PubMed Web of Science ®Google Scholar
• Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist , 47 (2), 71–85. doi:10.1080/00461520.2012.667064
   Web of Science ®Google Scholar
• Newby, P. (2010). Research methods for education . Harlow: Pearson Education Limited.
   Google Scholar
• Newton, P. E. (2005). The public understanding of measurement inaccuracy. British Educational Research Journal , 31 , 419–442. doi:10.1080/01411920500148648
   Web of Science ®Google Scholar
• Pang, A. T. , Wittenbrink, C. M. , & Lodha, S. K. (1997). Approaches to uncertainty visualization. The Visual Computer , 13 , 370–390. doi:10.1007/s003710050111
   Web of Science ®Google Scholar
• Phelps, R. P. , Zenisky, A. , Hambleton, R. K. , & Sireci, S. G. (2010). On the reporting of measurement uncertainty and reliability for U.S. educational and licensure tests (OFQUAL 10/4759). London: Office of Qualifications and Examinations.
   Google Scholar
• Sanyal, J. , Zhang, S. , Bhattacharya, G. , Amburn, P. , & Moorhead, R. J. (2009). A user study to compare four uncertainty visualization methods for 1D and 2D datasets. IEEE Transactions on Visualization and Computer Graphics , 15 , 1209–1218. doi:10.1109/TVCG.2009.114
   PubMed Web of Science ®Google Scholar
• Shepard, L. A. (2006). Classroom assessment. In R. L. Brennan (Ed.), Educational measurment (4th ed., pp. 623–646). Westport: American Council on Education and Praeger.
   Google Scholar
• Wainer, H. (1995). Depicting error (Technical Report No 95-2). Princeton, NJ: Educational Testing Service.
   Google Scholar
• Wainer, H. , Hambleton, R. K. , & Meara, K. (1999). Alternative displays for communicating NAEP results: A redesign and validity study. Journal of Educational Measurement , 36 , 301–335. doi:10.1111/j.1745-3984.1999.tb00559.x
   Web of Science ®Google Scholar
• Zapata-Rivera, D. , Zwick, R. , & Vezzu, M. (2016). Exploring the effectiveness of a measurement error tutorial in helping teachers understand score report results. Educational Assessment , 21 , 215–229. doi:10.1080/10627197.2016.1202110
   Web of Science ®Google Scholar
• Zwick, R. , Zapata-Rivera, D. , & Hegarty, M. (2014). Comparing graphical and verbal representations of measurement error in test score reports. Educational Assessment , 19 , 116–138. doi:10.1080/10627197.2014.903653
   Google Scholar