Innovations in Setting Performance Standards for K–12 Test-Based Accountability

REFERENCES

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education . 1999 . Standards for educational and psychological testing , Washington, DC : American Educational Research Association .
   Google Scholar
• Cizek , G. J. 2010 . Handbook of formative assessment , Edited by: Andrade , H. New York : Taylor and Francis .
   Google Scholar
• Bejar , I. I. , Braun , H. I. and Tannenbaum , R. J. 2007 . “ A prospective, progressive, and predictive approach to standard setting ” . In Assessing and modeling cognitive development in school , Edited by: Lissitz , R. 1 – 30 . Maple Grove, MN : JAM Press .
   Google Scholar
• Bennet , R. E. 2009 . A critical look at the meaning and basis of formative assessment (ETS Research Memorandum -09-06 Educational Testing Service) Princeton, NJ
   Google Scholar
• Black , P. and Wiliam , D. 1998 . Assessment and classroom learning . Assessment in Education , 5 ( 1 ) : 7 – 74 .
   Google Scholar
• Camilli , M. J. , Cizek , G. J. and Lugg , C. A. 2001 . “ Psychometric theory and the validation of performance standards: History and future perspective ” . In Setting performance standards: Concepts, methods and perspectives , Edited by: Cizek , G. J. 445 – 476 . Mahwah, NJ : Lawrence Erlbaum .
   Google Scholar
• Cizek , G. J. , ed. 2001 . Setting performance standards: Concepts, methods, and perspectives , Hillsdale, NJ : Erlbaum .
   Google Scholar
• Cizek , G. J. 1993 . Reconsidering standards and criteria . Journal of Educational Measurement , 30 : 93 – 106 .
   Web of Science ®Google Scholar
• Cizek , G. J. and Bunch , M. B. 2007 . Standard Setting: A guide to establishing and evaluating performance standards on tests , Thousand Oaks, CA : Sage .
   Google Scholar
• Corcoran , T. , Mosher , F. A. and Rogat , A. 2009 . Learning progressions in science: An evidence-based approach to reform (CPRE Research Report # RR-63) , Philadelphia, PA : Columbia University .
   Google Scholar
• Egan , K. L. , Schneider , M. C. and Ferrara , S. in press . “ A validity framework for defining proficient performance and setting cut scores for accessible tests ” . In Accessible tests of student achievement: issues, innovations, and applications , Edited by: Elliot , S. , Kettler , R. , Beddow , P. and Kurz , A. New York : Springer .
   Google Scholar
• Ewing , M. , Packman , S. , Hamen , C. and Clark Thurber , A. in press . Representing targets of measurement using ECD . Applied Measurement in Education. ,
   Google Scholar
• Ferrara , S. and DeMauro , G. E. 2006 . “ Standardized assessment of individual achievement in K-12 ” . In Educational measurement , 4th , Edited by: Brennan , R. L. 579 – 621 . Westport, CT : American Council on Education/Praeger .
   Google Scholar
• Ferrara , S. , Dubravka , S. , Skucha , S. and Murphy , A. Test development with standard setting in mind . Paper presented at the annual meeting of National Council on Measurement in Education (NCME) in San Diego . CA. April .
   Google Scholar
• Ferrara , S. , Huff , K. and Lopez , E. Targeting cognition in item design to enhance valid interpretations of performance and to guide instruction . Paper presented at the annual meeting of the American Educational Research Association . Denver, CO.
   Google Scholar
• Ferrara, S., Perie, M., & Johnson, E. (2008). Matching the judgmental task with standard setting panelist expertise: The item-descriptor (ID) matching method. Journal of Applied Testing Technology, 9(1). http://www.testpublishers.org/Documents/JATT2008_Ferrara%20et%20al.%20IDM.pdf (http://www.testpublishers.org/Documents/JATT2008_Ferrara%20et%20al.%20IDM.pdf) (Accessed: 31 May 2009 ).
   Google Scholar
• Feuer , M. J. Externalities of testing: Lessons from the blizzard of 2010 . Measurement: Interdisciplinary Research and Perspectives . Vol. 8 , pp. 59 – 69 .
   Google Scholar
• Gorin , J. S. 2005 . Manipulation of processing difficulty on reading comprehension test questions: The feasibility of verbal item generation . Journal of Educational Measurement , 42 : 351 – 373 .
   Web of Science ®Google Scholar
• Gorin , J. S. and Embretson , S. E. 2006 . Item difficulty modeling of paragraph comprehension items . Applied Psychological Measurement , 30 ( 5 ) : 394 – 411 .
   Web of Science ®Google Scholar
• Haertel , E. H. and Lorie , W. A. 2004 . Validating standards-based test score interpretations . Measurement: Interdisciplinary Research and Perspectives , 2 ( 2 ) : 61 – 103 .
   Google Scholar
• Hambleton , R. K. 2001 . “ Setting performance standards on educational assessments and criteria for evaluating the process ” . In Setting performance standards: Concepts, methods, and perspectives , Edited by: Cizek , G. J. Mahwah, NJ : Lawrence Erlbaum .
   Google Scholar
• Hambleton , R. K. and Pitoniak , M. 2006 . “ Setting performance standards ” . In Educational measurement , 4th , Edited by: Brennan , R. L. 433 – 470 . Westport, CT : American Council on Education/Praeger .
   Google Scholar
• Hattie , J. and Timperley , H. 2007 . The power of feedback . Review of Education Research , 77 ( 1 ) : 81 – 112 .
   Web of Science ®Google Scholar
• Hendrickson , A. , Huff , K. and Luecht , R. in press . Claims, evidence and achievement level descriptors as a foundation for item design and test specifications . Applied Measurement in Education. ,
   Google Scholar
• Heritage , M. , Kim , J. , Vendlinski , T. and Herman , J. 2009 . From evidence to action: A seamless process in formative assessment? . Educational Measurement: Issues and Practice , 28 ( 3 ) : 24 – 31 .
   Google Scholar
• Huff, K. (2003). An item modeling approach to descriptive score reports. Electronic Doctoral Dissertations, University of Massachusetts at Amherst, Paper AAI3096285. http://scholarworks.umass.edu/dissertations/AAI3096285 (http://scholarworks.umass.edu/dissertations/AAI3096285)
   Google Scholar
• Jaeger , R. M. 1989 . “ Certification of student competence ” . In Educational measurement , 3rd , Edited by: Linn , R. L. 485 – 514 . New York : American Council on Education and Macmillan .
   Google Scholar
• Kane , M. T. 2001 . “ So much remains the same: Conception and status of validation in setting standards ” . In Setting performance standards: Concepts, methods, and perspectives , Edited by: Cizek , G. J. Mahwah, NJ : Lawrence Erlbaum .
   Google Scholar
• Kane , M. T. 1994 . Validating the performance standards associated with passing scores . Review of Educational Research , 64 ( 3 ) : 425 – 461 .
   Web of Science ®Google Scholar
• Kluger , A. N. and DeNisi , A. 1996 . The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory . Psychological Bulletin , 119 : 254 – 284 .
   Web of Science ®Google Scholar
• Linn, R. (2005). Conflicting demands of No Child Left Behind and state systems: Mixed messages about school performance. Educational Policy Analysis Archives, 13(33). http://epaa.asu.edu/epaa/v11n31/ (http://epaa.asu.edu/epaa/v11n31/) (Accessed: 1 December 2009 ).
   Google Scholar
• Linn, R. L. (2003, September 1). Performance standards: Utility for different uses of assessments. Education Policy Analysis Archives, 11(31) http://epaa.asu.edu/epaa/v11n31/ (http://epaa.asu.edu/epaa/v11n31/) (Accessed: 1 December 2009 ).
   Google Scholar
• Loomis , S. C. and Bourque , M. L. 2001 . “ From tradition to innovation: Standard setting on the National Assessment of Educational Progress ” . In Standard setting: Concepts, methods, and perspectives , Edited by: Cizek , G. J. 175 – 217 . Mahwah, NJ : Erlbaum .
   Google Scholar
• Mills , C. N. and Jaeger , R. M. 1998 . “ Creating description of desired student achievement when setting performance standards ” . In Meeting the requirements of Title 1: Handbook for the development of performance standards , Edited by: Hansche , L. N. 73 – 85 . Washington, DC : US Department of Education .
   Google Scholar
• Mislevy , R. J. , Steinberg , L. S. and Almond , R. G. 2003 . On the structure of educational assessments . Measurement: Interdisciplinary Research and Perspectives , 1 : 3 – 67 .
   Google Scholar
• Mislevy , R. J. , Steinberg , L. S. and Almond , R. G. 2002 . “ On the roles of task model variables in assessment design ” . In Item generation for test development , Edited by: Irvine , S. and Kyllonen , P. 97 – 128 . Mahwah, NJ : Erlbaum .
   Google Scholar
• National Research Council . 2001 . Knowing what students know: The science and design of educational assessment , Washington, DC : National Academy Press .
   Google Scholar
• Nichols , P. , Twing , J. , Mueller , C. D. and O'Malley , K. 2010 . Standard-setting methods as measurement processes . Educational Measurement: Issues and Practice , 29 ( 1 ) : 14 – 24 .
   Google Scholar
• Nitko , A. J. 1980 . Distinguishing the many varieties of criterion-referenced tests . Review of Educational Research , 50 : 461 – 485 .
   Web of Science ®Google Scholar
• Pellegrino , J. W. , Jones , L. R. and Mitchell , K. J. 1999 . Grading the nation's report card: Evaluating NAEP and transforming the assessment of educational progress , Washington, DC : National Academy Press .
   Google Scholar
• Perie , M. 2008 . A guide to understanding and developing performance-level descriptors . Educational Measurement: Issues and Practice , 27 : 15 – 29 .
   Google Scholar
• Plake , B. , Huff , K. and Reshetar , R. in press . Evidence centered design as a foundation for ALD development . Applied Measurement in Education. ,
   Google Scholar
• Popham , J. 2008 . Transformative assessment , Alexandria, VA : Association for Supervision and Curriculum Development .
   Google Scholar
• Schmeiser , C. B. and Welch , C. J. 2006 . “ Test development ” . In Educational measurement , 4th , Edited by: Brennan , R. L. 307 – 353 . Washington, DC : National Council on Measurement in Education & American Council on Education .
   Google Scholar
• Schneider , M. C. , Egan , K. L. , Siskind , T. , Brailsford , A. and Jones , E. Concurrence of target student descriptors and mapped item demands in achievement levels across time . Paper presented at the annual meeting of the National Council on Measurement in Education . April , San Diego, CA.
   Google Scholar
• Schneider , C. , Huff , K. , Egan , K. , Tully , M. and Ferrara , S. Aligning achievement level descriptors to mapped item demands across achievement levels to enhance valid interpretation of scale scores and inform item development . Paper presented at the annual meeting of the American Educational Research Association . Denver, CO.
   Google Scholar
• Schulz , E. M. , Kolen , M. J. and Nicewander , W. A. 1999 . A rationale for defining achievement levels using IRT-estimated domain scores . Applied Psychological Measurement , 23 ( 4 ) : 347 – 362 .
   Web of Science ®Google Scholar
• Schulz , E. M. , Lee , W. and Mullen , K. 2005 . A domain level approach to describing growth in achievement . Journal of Educational Measurement , 42 : 1 – 26 .
   Web of Science ®Google Scholar
• Sireci , S. G. , Hauger , J. B. , Wells , C. S. , Shea , C. and Zenisky , A. L. 2009 . Evaluation of the standard setting on the 2005 grade 12 national assessment of educational progress mathematics test . Applied Measurement in Education , 22 ( 4 ) : 339 – 358 .
   Web of Science ®Google Scholar
• Thompson , M. and Wiliam , D. 2008 . “ Tight but loose: A conceptual framework for scaling up school reforms ” . In Tight but loose: Scaling up teacher professional development in diverse contexts (Research Report RR-08-29) , Edited by: Wylie , E. C. Princeton, NJ : Educational Testing Service .
   Google Scholar
• Wiliam , D. 1996 . Meanings and consequences in standard setting . Assessment in Education: Principles, Policy and Practice , 3 ( 3 ) : 287
   Google Scholar
• Zieky , M. , Perie , M. and Livingston , S. 2008 . Cut scores: A manual for setting performance standards on educational and occupational tests , Princeton, NJ : Educational Testing Service .
   Google Scholar