Situating Standard Setting within Argument-Based Validity

References

• Alderson, J. C. (2007). The CEFR and the need for more research. The Modern Language Journal, 91(4), 659–663. doi:10.1111/modl.2007.91.issue-4
   Web of Science ®Google Scholar
• Alderson, J. C., Figueras, N., Kuijper, H., Nold, G., Takala, S., & Tardieu, C. (2006). Analysing tests of reading and listening in relation to the Common European Framework of Reference: The experience of the Dutch CEFR construct project. Language Assessment Quarterly, 3(1), 3–30. doi:10.1207/s15434311laq0301_2
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
   Google Scholar
• Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2(1), 1–34. doi:10.1207/s15434311laq0201_1
   Google Scholar
• Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford, UK: Oxford University Press.
   Google Scholar
• Bejar, I. I., Braun, H., & Tannenbaum, R. J. (2007). A prospective, predictive and progressive approach to standard setting. In R. W. Lissitz (Ed.), Assessing and modeling cognitive development in school: Intellectual growth and standard setting (pp. 1–30). Maple Grove, MN: Jam Press.
   Google Scholar
• Chapelle, C. A. (2008). The TOEFL validity argument. In C. A. Chapelle, M. K. Enright, & J. M. Jamieson (Eds.), Building a validity argument for the Test of English as a Foreign Language (pp. 319–352). London, UK: Routledge.
   Google Scholar
• Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (Eds.). (2008). Building a validity argument for the Test of English as a Foreign Language. New York, NY: Routledge.
   Google Scholar
• Cizek, G. J. (Ed.). (2001). Setting performance standards: Concepts, methods, and perspectives. Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Cizek, G. J. (2012). Defining and distinguishing validity: Interpretations of score meaning and justifications of test use. Psychological Methods, 17(1), 31–43. doi 10.1037/a0026975.
   PubMed Web of Science ®Google Scholar
• Cizek, G. J., & Bunch, M. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. London, UK: Sage Publications.
   Google Scholar
• Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Council of Europe. (2009). Relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment. A manual. Retrieved February 15, 2009, from http://www.coe.int/t/dg4/linguistic/manuel1_en.asp
   Google Scholar
• Davidson, F., & Fulcher, G. (2007). The Common European Framework of Reference (CEFR) and the design of language tests: A matter of effect. Language Teaching, 40(3), 231–241. doi:10.1017/S0261444807004351
   Google Scholar
• Educational Testing Service. (2005). Setting the final cut score. Retrieved November 29, 2013, from http://www.ets.org/Media/Tests/TOEFL/pdf/setting_final_scores.pdf
   Google Scholar
• Egan, K. L., Schneider, M. C., & Ferrara, S. (2013). Performance level descriptors: History, practice, and a proposed framework. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 79–106). New York, NY: Routledge.
   Google Scholar
• Figueras, N., North, B., Takala, S., Verhelst, N., & Van Avermaet, P. (2005). Relating examinations to the Common European Framework: A manual. Language Testing, 22(3), 261–279. doi:10.1191/0265532205lt308oa
   Google Scholar
• Fulcher, G. (2004). Deluded by artifices? The Common European Framework and Harmonization. Language Assessment Quarterly, 1(4), 253–266. doi:10.1207/s15434311laq0104_4
   Google Scholar
• Fulcher, G., & Davidson, F. (2009). Test architecture, test retrofit. Language Testing, 26(1), 123–144. doi:10.1177/0265532208097339
   Web of Science ®Google Scholar
• Geisinger, K. F., & McCormick, C. M. (2010). Adopting cut scores: Post-standard-setting panel considerations for decision makers. Educational Measurement: Issues and Practice, 29(1), 38–44. doi:10.1111/j.1745-3992.2009.00168.x
   Google Scholar
• Glass, G. V. (1978). Standards and criteria. Journal of Educational Measurement, 15(4), 237–261. doi:10.1111/jedm.1978.15.issue-4
   Web of Science ®Google Scholar
• Hambleton, R. K. (1978). Use of cut-off scores. Journal of Educational Measurement, 15(4), 277–294. doi:10.1111/j.1745-3984.1978.tb00075.x
   Web of Science ®Google Scholar
• Hambleton, R. K. (2001). Setting performance standards on educational assessments and criteria for evaluating the process. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 89–116). Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Hambleton, R. K., & Pitoniak, M. J. (2006). Setting performance standards. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 433–470). Westport, CT: Praeger Publishers.
   Google Scholar
• Hasselgreen, A. (2005). Assessing the language of young learners. Language Testing, 22(3), 337–354. doi:10.1191/0265532205lt312oa
   Google Scholar
• Hasselgreen, A. (2012). Adapting the CEFR for the classroom assessment of young learners’ writing. Canadian Modern Language Review, 69(4), 415–435. doi:10.3138/cmlr.1705.415
   Web of Science ®Google Scholar
• Ilc, G., & Stopar, A. (2015). Validating the Slovenian national alignment to CEFR: The case of the B2 reading comprehension examination in English. Language Testing, 32(4), 443–462. doi:10.1177/0265532214562098
   Web of Science ®Google Scholar
• Jones, N., & Saville, N. (2009). European language policy: Assessment, learning, and the CEFR. Annual Review of Applied Linguistics, 29, 51–63. doi:10.1017/S0267190509090059
   Web of Science ®Google Scholar
• Kaftandjieva, F. (2004). Standard setting. Section B of the reference supplement to the preliminary version of the manual for relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment. Strasbourg, France: Council of Europe.
   Google Scholar
• Kaftandjieva, F. (2010). Methods for setting cut scores in criterion-referenced achievement tests. Arnhem, Netherlands: Cito.
   Google Scholar
• Kaftandjieva, F., & Takala, S. (2002). Council of Europe scales of language proficiency: A validation study. In J. C. Alderson (Ed.), Common European Framework of Reference for Languages: Learning, teaching, assessment. Case studies (pp. 106–129). Strasbourg, France: Council of Europe.
   Google Scholar
• Kane, M. T. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64(3), 425–461. doi:10.3102/00346543064003425
   Web of Science ®Google Scholar
• Kane, M. T. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research and Perspectives, 2, 135–170.
   Google Scholar
• Kane, M. T. (2006). Validity. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: Praeger Publishers.
   Google Scholar
• Kane, M. T. (2012). Validating score interpretations and uses. Language Testing, 29(1), 3–17. doi:10.1177/0265532211417210
   Web of Science ®Google Scholar
• Kane, M. T. (2013). Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement, 50(1), 1–73. doi:10.1111/jedm.2013.50.issue-1
   Web of Science ®Google Scholar
• Kane, M. T., Crooks, T., & Cohen, A. S. (1999). Validating measures of performance. Educational Measurement: Issues and Practice, 18(2), 5–17. doi:10.1111/j.1745-3992.1999.tb00010.x
   Google Scholar
• Kenyon, D. M. (2012). Using Bachman’s assessment use argument as a tool in conceptualizing the issues surrounding linking ACTFL and CEFR. In E. Tschirner (Ed.), Aligning frameworks of reference in language testing: The ACTFL proficiency guidelines and the Common European Framework of Reference for Languages. Tübingen, Germany: Stauffenburg Verlag.
   Google Scholar
• Kenyon, D. M., & Römhild, A. (2013). Standard setting in language testing. In A. J. Kunnan (Ed.), The companion to language assessment (pp. 944–961). Malden, MA: John Wiley & Sons, Inc.
   Google Scholar
• Little, D. (2007). The Common European Framework of Reference for Languages: Perspectives on the making of supranational language education policy. The Modern Language Journal, 91(4), 645–655.
   Web of Science ®Google Scholar
• McClarty, K. L., Way, W. D., Porter, A. C., Beimers, J. N., & Miles, J. A. (2013). Evidence-based standard setting: Establishing a validity framework for cut scores. Educational Researcher, 42, 78–88. doi:10.3102/0013189X12470855
   Web of Science ®Google Scholar
• McNamara, T. (2006). Validity in language testing: The challenge of Sam Messick’s legacy. Language Assessment Quarterly, 3(1), 31–51. doi:10.1207/s15434311laq0301_3
   Google Scholar
• McNamara, T., & Roever, C. (2006). Language testing: The social dimension. Oxford, UK: Blackwell.
   Google Scholar
• Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan.
   Google Scholar
• Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(4), 241–256. doi:10.1177/026553229601300302
   Google Scholar
• Milanovic, M., & Weir, C. J. (2010). Series editors’ note. In W. Martyniuk (Ed.), Relating language examinations to the Common European Framework of Reference for Languages: Case studies and reflections on the use of the Council of Europe’s Draft Manual (pp. viii-xx). Cambridge, UK: Cambridge University Press.
   Google Scholar
• Norcini, J. J., Lipner, R. S., Langdon, L. O., & Strecker, C. A. (1987). A comparison of three variations on a standard-setting method. Journal of Educational Measurement, 24(1), 56–64. doi:10.1111/jedm.1987.24.issue-1
   Web of Science ®Google Scholar
• North, B. (2000). The development of a common framework scale of language proficiency. New York, NY: Peter Lang.
   Google Scholar
• North, B. (2002). A CEF-based self-assessment tool for university entrance. In J. C. Alderson (Ed.), Common European Framework of Reference for Languages: Learning, teaching, assessment. Case studies (pp. 146–166). Strasbourg, France: Council of Europe.
   Google Scholar
• North, B. (2007). The CEFR Illustrative Descriptor Scales. The Modern Language Journal, 91(4), 656–659. doi:10.1111/j.1540-4781.2007.00627_3.x
   Web of Science ®Google Scholar
• North, B. (2014). The CEFR in practice. Cambridge, UK: Cambridge University Press.
   Google Scholar
• North, B., & Schneider, G. (1998). Scaling descriptors for language proficiency scales. Language Testing, 15(2), 217–262. doi:10.1177/026553229801500204
   Google Scholar
• Pant, H. A., Rupp, A. A., Tiffin-Richards, S. P., & Köller, O. (2009). Validity issues in standard-setting studies. Studies in Educational Evaluation, 35(2–3), 95–101. doi:10.1016/j.stueduc.2009.10.008
   Google Scholar
• Papageorgiou, S. (2010). Investigating the decision-making process of standard setting participants. Language Testing, 27(2), 261–282. doi:10.1177/0265532209349472
   Web of Science ®Google Scholar
• Papageorgiou, S., & Cho, Y. (2014). An investigation of the use of TOEFL® Junior™ Standard scores for ESL placement decisions in secondary education. Language Testing, 31(2), 223–239. doi:10.1177/0265532213499750
   Web of Science ®Google Scholar
• Plake, B. S. (2008). Standard setters: Stand up and take a stand! Educational Measurement: Issues and Practice, 27(1), 3–9. doi:10.1111/j.1745-3992.2008.00110.x
   Google Scholar
• Plake, B. S., Huff, K., & Reshetar, R. (2010). Evidence-centered assessment design as a foundation for achievement-level descriptor development and for standard setting. Applied Measurement in Education, 23(4), 342–357. doi:10.1080/08957347.2010.510964
   Web of Science ®Google Scholar
• Popham, W. J. (1978). As always, provocative. Journal of Educational Measurement, 15(4), 297–300. doi:10.1111/jedm.1978.15.issue-4
   Web of Science ®Google Scholar
• Raymond, M. R., & Reid, J. B. (2001). Who made thee a judge? Selecting and training participants for standard setting. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 119–158). Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Shohamy, E., & McNamara, T. (2009). Language tests for citizenship, immigration, and asylum. Language Assessment Quarterly, 6(1), 1–5. doi:10.1080/15434300802606440
   Web of Science ®Google Scholar
• Subkoviak, M. J. (1988). A practitioner’s guide to computation and interpretation of reliability indices for mastery tests. Journal of Educational Measurement, 25(1), 47–55. doi:10.1111/jedm.1988.25.issue-1
   Web of Science ®Google Scholar
• Tannenbaum, R. J., & Cho, Y. (2014). Criteria for evaluating standard-setting approaches to map English language test scores to frameworks of English language proficiency. Language Assessment Quarterly, 11(3), 233–249. doi:10.1080/15434303.2013.869815
   Web of Science ®Google Scholar
• Tannenbaum, R. J., & Kannan, P. (2015). Consistency of Angoff-based standard-setting judgments: Are item judgments and passing scores replicable across different panels of experts? Educational Assessment, 20(1), 66–78. doi:10.1080/10627197.2015.997619
   Web of Science ®Google Scholar
• Tannenbaum, R. J., & Katz, I. R. (2013). Standard setting. In K. F. Geisinger (Ed.), APA handbook of testing and assessment in psychology: Vol 3. Testing and assessment in school psychology and education (pp. 455–477). Washington, DC: American Psychological Association.
   Google Scholar
• Toulmin, S. E. (2003). The uses of argument. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Wall, D. (2000). The impact of high-stakes testing on teaching and learning: Can this be predicted or controlled? System, 28(4), 499–510. doi:10.1016/S0346-251X(00)00035-X
   Google Scholar
• Wall, D. (2005). The impact of high-stakes testing on classroom teaching: A case study using insights from testing and innovation theory. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Wall, D., & Alderson, J. C. (1993). Examining washback: The Sri Lankan impact study. Language Testing, 10(1), 41–69. doi:10.1177/026553229301000103
   Google Scholar
• Weir, C. J. (2005). Limitations of the Common European Framework of Reference for Languages (CEFR) for developing comparable examinations and tests. Language Testing, 22(3), 281–300. doi:10.1191/0265532205lt309oa
   Google Scholar
• Wiliam, D. (1996). Meanings and consequences in standard setting. Assessment in Education: Principles, Policy & Practice, 3(3), 287–308. doi:10.1080/0969594960030303
   Google Scholar
• Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cutscores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.
   Google Scholar