Argument-Based Validation in Practice: Examples From Mathematics Education

References

• American Educational Research Association, American Psychological Association, & National Council for Measurement in Education. (2014). Standards for educational and psychological testing. Washington, D.C.: American Educational Research Association.
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
   Google Scholar
• Argument. 2018. In Merriam-Webster.com. Retrieved June 4, 2018, From https://www.merriamwebster.com/dictionary/argument
   Google Scholar
• Argument. (n.d.). Retrieved from https://www.merriam-webster.com/dictionary/argument
   Google Scholar
• Baxter, G., & Mislevy, R. J. (2005). The case for an integrated design framework for assessing science inquiry. SRI International Retrieved from http://padi.sri.com
   Google Scholar
• Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17(2–3), 62–87. doi:10.1080/10627197.2012.715014
   Google Scholar
• Bostic, J. (2017). Moving forward: Instruments and opportunities for aligning current practices with testing standards. Investigations in Mathematics Learning, 9(3), 109–110. doi:10.1080/19477503.2017.1325662
   Google Scholar
• Bostic, J., Krupa, E., Carney, M., & Shih, J. (in press). Reflecting on the past and thinking ahead in the measurement of students’ outcomes. In J. Bostic, E. Krupa, & J. Shih (Eds.), Quantitative measures of mathematical knowledge. New York, NY: Routledge.
   Google Scholar
• Bostic, J., Sondergeld, T., Folger, T., & Kruse, L. (2017). PSM7 and PSM8: Validating two problem-solving measures. Journal of Applied Measurement, 18(2), 151–162.
   PubMedGoogle Scholar
• Bostic, J. D., & Sondergeld, T. A. (2015). Measuring sixth-grade students’ problem solving: Validating an instrument addressing the mathematics common core. School Science and Mathematics, 115(6), 281–291. doi:10.1111/ssm.2015.115.issue-6
   Web of Science ®Google Scholar
• Carney, M. B., Cavey, L., & Hughes, G. (2017). Assessing teacher attentiveness to student mathematical thinking: Validity claims and evidence. The Elementary School Journal, 118(2), 281–309. doi:10.1086/694269
   Web of Science ®Google Scholar
• Center on Standards and Assessment Implementation. (2016). Overview of major assessment types in standards-based instruction. San Francisco, CA: WestEd. Retrieved from http://www.csai-online.org/sites/default/files/resources/6257/CSAI_AssessmentTypes.pdf
   Google Scholar
• Cizek, G. J., Rosenberg, S. L., & Koons, H. H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68(3), 397–412. doi:10.1177/0013164407310130
   Web of Science ®Google Scholar
• Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (Vol. 2nd, pp. 443–507). Washington, D.C.: American Council on Education.
   Google Scholar
• Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281. doi:10.1037/h0040957
   PubMed Web of Science ®Google Scholar
• Cureton, E. E. (1951). Validity. In E. F. Lindquist (Ed.), Educational Measurement (pp. 621–694). Washington, D.C.: American Council on Education.
   Google Scholar
• Ferrara, S., Lai, E., Reilly, A., & Nichols, P. D. (2017). Principled approaches to assessment design, development, and implementation. In A. A. Rupp & J. P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and applications (pp. 41–74). Chichester, UK: John Wiley and Sons.
   Google Scholar
• Gleason, J., Livers, S., & Zelkowski, J. (2017). Mathematics Classroom Observation Protocol for Practices (MCOP2): A validation study. Investigations in Mathematics Learning, 9(3), 111–129. doi:10.1080/19477503.2017.1308697
   Google Scholar
• Haertel, E. H. (1999). Validity arguments for high-stakes testing: In search of the evidence. Educational Measurement: Issues and Practice, 18(4), 5–9. doi:10.1111/j.1745-3992.1999.tb00276.x
   Google Scholar
• Haertel, E. H., & Lorié, W. A. (2004). Rejoiner to Commentary. Measurement: Interdisciplinary Research and Perspectives, 2(2), 129–133.
   Google Scholar
• Hill, H. C., & Shih, J. C. (2009). Examining the quality of statistical mathematics education research. Journal for Research in Mathematics Education, 40(3), 241–250.
   Web of Science ®Google Scholar
• Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527. doi:10.1037/0033-2909.112.3.527
   Web of Science ®Google Scholar
• Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319–342. doi:10.1111/jedm.2001.38.issue-4
   Web of Science ®Google Scholar
• Kane, M. T. (2002). Validating high-stakes testing programs. Educational Measurement: Issues and Practice, 21(1), 31–41. doi:10.1111/j.1745-3992.2002.tb00083.x
   Google Scholar
• Kane, M. T. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research and Perspectives, 2(3), 135–170.
   Google Scholar
• Kane, M. T. (2006). Valildation. In R. L. Brennan. National Council on Measurement in Education, & American Council on Education (Ed.), Educational Measurement (4th ed., pp. 17–64). Westport, CT: Praeger Publishers.
   Google Scholar
• Kane, M. T. (2007). Validating measures of mathematical knowledge for teaching. Measurement: Interdisciplinary Research and Perspectives, 5(2–3), 180–187.
   Google Scholar
• Kane, M. T. (2012). Validating score interpretations and uses. Language Testing, 29(1), 3–17. doi:10.1177/0265532211417210
   Web of Science ®Google Scholar
• Kane, M. T. (2013). The argument-based approach to validation. School Psychology Review, 42(4), 448.
   Web of Science ®Google Scholar
• Kane, M. T. (2016). Validation strategies: Delineating and validating proposed interpretations and uses of test scores. In S. Lane, M. Raymond, & T. M. Haladyna (Eds.), Handbook of Test Development (Vol. 2nd, pp. 64–80). New York, NY: Routledge.
   Google Scholar
• Marion, S. F., & Pellegrino, J. W. (2006). A validity framework for evaluating the technical quality of alternate assessments. Educational Measurement: Issues and Practice, 25(4), 47–57. doi:10.1111/j.1745-3992.2006.00078.x
   Google Scholar
• Markus, K. A., & Borsboom, D. (2013). Frontiers of test validity theory: Measurement, causation, and meaning. New York, NY: Routledge.
   Google Scholar
• Mehrens, W. A. (1997). The consequences of consequential validity. Educational Measurement: Issues and Practice, 16(2), 16–18. doi:10.1111/j.1745-3992.1997.tb00588.x
   Google Scholar
• Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (pp. 13–103). New York, NY: MacMillan.
   Google Scholar
• Mislevy, R. J. (1996). Test theory reconceived. Journal of Educational Measurement, 33, 379–416. doi:10.1111/jedm.1996.33.issue-4
   Web of Science ®Google Scholar
• Mislevy, R. J. (2006). Cognitive psychology and educational assessment. Educational Measurement, 4, 257–305.
   Google Scholar
• Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Research Report Series, 2003(1). doi:10.1002/j.2333-8504.2003.tb01908.x
   Google Scholar
• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). Focus article: On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3–62.
   Google Scholar
• National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: The National Academies Press.
   Google Scholar
• National Research Council. (2014). Developing assessments for the next generation science standards. Washington, DC: The National Academies Press.
   Google Scholar
• Newton, P. E. (2012). Clarifying the consensus definition of validity. Measurement: Interdisciplinary Research & Perspective, 10(1–2), 1–29.
   Google Scholar
• Newton, P. E., & Baird, J.-A. (2016). The great validity debate. Assessment in Education: Principles, Policy & Practice, 23(2), 173–177. doi:10.1080/0969594X.2016.1172871
   Web of Science ®Google Scholar
• Oliveri, M. E., Lawless, R., & Young, J. W. (2015). A validity framework for the use and development of exported assessment. Princeton, NJ: Educational Testing Service.
   Google Scholar
• Pellegrino, J. W., DiBello, L. V., & Goldman, S. R. (2016). A framework for conceptualizing and evaluating the validity of instructionally relevant assessments. Educational Psychologist, 51(1), 59–81. doi:10.1080/00461520.2016.1145550
   Web of Science ®Google Scholar
• Perie, M., & Marion, S. (2008). Developing a validity argument for a state alternate assessment (AA-AAS) system: A guide for states. Retrieved July 15, 2008 from http://www.naacpartners.org/projects/valdityGSEG/expertPanel.aspx.
   Google Scholar
• Schilling, S. G. (2004). Conceptualizing the validity argument: An alternative approach. Measurement: Interdisciplinary Research and Perspectives, 2(3), 178–182.
   PubMedGoogle Scholar
• Schilling, S. G. (2007). The role of psychometric modeling in test validation: An application of multidimensional item response theory. Measurement: Interdisciplinary Research and Perspectives, 5(2–3), 93–106.
   Google Scholar
• Schilling, S. G., Blunk, M., & Hill, H. C. (2007). Test validation and the MKT Measures: Generalizations and conclusions. Measurement: Interdisciplinary Research and Perspectives, 5(2–3), 118–128.
   Google Scholar
• Schilling, S. G., & Hill, H. C. (2007). Assessing measures of mathematical knowledge for teaching: A validity argument approach. Measurement: Interdisciplinary Research and Perspectives, 5(2–3), 70–80.
   Google Scholar
• Shaw, S., Crisp, V., & Johnson, N. (2012). A framework for evidencing assessment validity in large-scale, high-stakes international examinations. Assessment in Education: Principles, Policy & Practice. doi:10.1080/0969594X.2011.563356
   Google Scholar
• Shear, B. R., & Zumbo, B. D. (2014). What counts as evidence: A review of validity studies in educational and psychological measurement. In B. D. Zumbo & E. K. H. Chan (Eds.), Validity and validation in social, behavioral, and health sciences (pp. 91–111). Cham, Switzerland: Springer.
   Google Scholar
• Sireci, S. G. (2012). Smarter balanced assessment consortium: Comprehensive research Agenda. Retrieved from https://portal.smarterbalanced.org/library/en/comprehensive-research-agenda.pdf
   Google Scholar
• Toulmin, S. E. (1958). The uses of argument. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Wiliam, D. (2014). Principled assessment design. London, UK: SSAT.
   Google Scholar
• Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Erlbaum.
   Google Scholar
• Wolming, S., & Wikström, C. (2010). The concept of validity in theory and practice. Assessment in Education: Principles, Policy & Practice, 17(2), 117–132. doi:10.1080/09695941003693856
   Google Scholar