Two Versions of the Contrasting-Groups Standard-Setting Method: A Review

REFERENCES

• Angoff, W. H. (1971). Scales, norms and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). Washington, DC: American Council on Education.
   Google Scholar
• Bacharach, S. B., & Lawler, E. J. (1980). Power and politics in organizations. San Francisco: Jossey-Bass.
   Google Scholar
• Baron, J. B., Rindone, D. A., & Prowda, P. (1981, April). Setting statewide proficiency standards: A multi-method multi-fate analysis. Paper presented at the meeting of the National Council on Measurement in Education, Los Angeles.
   Google Scholar
• Berk, R. A. (1986). A consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56, 137–172.
   Web of Science ®Google Scholar
• Brandon, P. R. (2002). A review of the literature on the steps of the Angoff standard-setting method. Manuscript submitted for publication.
   Google Scholar
• Cascio, W. F., Alexander, R. A., & Barrett, G. V. (1988). Setting cutoff scores: Legal, psychometric, and professional issues and guidelines. Personnel Psychology, 41, 1–24.
   Web of Science ®Google Scholar
• Cizek, G. J., & Husband, T. H. (1997, March). A Monte-Carlo investigation of the contrasting groups standard-setting method. Paper presented at the meeting of the American Educational Research Association, Chicago.
   Google Scholar
• Clauser, B. E., & Clyman, S. G. (1994). A contrasting groups approach to standard setting for performance assessments of clinical skills. Academic Medicine, 69(RIME supplement), 542–544.
   Web of Science ®Google Scholar
• Clauser, B. E., & Leucht, R. M. (1998, April). An evaluation of factors influencing the accuracy of contrasting groups standard setting. Paper presented at the meeting of the American Educational Research Association, San Diego, CA.
   Google Scholar
• Cohen, A. S., Kane, M. T., & Crooks, T. J. (1999). A generalized examinee-centered method for setting standards on achievement tests. Applied Measurement in Education, 12, 343–366.
   Web of Science ®Google Scholar
• Fitzpatrick, A. R. (1989). Social influences in standard setting: The effects of social interaction on group judgments. Review of Educational Research, 59, 315–328.
   Web of Science ®Google Scholar
• Hambleton, R. K. (1980). Test score validity and standard setting methods. In R. A. Berk (Ed.), Criterion referenced measurement: State of the art (pp. 80–123). Baltimore: Johns Hopkins University.
   Google Scholar
• Jaeger, R. M. (1989). Certification of student competence. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 485–514). New York: American Council on Education/Macmillan.
   Google Scholar
• Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64, 425–461.
   Web of Science ®Google Scholar
• Kane, M. (1998a). Choosing between examinee-centered and test-centered standard-setting methods. Educational Assessment, 5, 129–145.
   Google Scholar
• Kane, M. (1998b). Criterion bias in examinee-centered standard setting: Some thought experiments. Educational Measurement: Issues and Practice, 17(1), 23–30.
   Google Scholar
• Koffler, S. L. (1980). A comparison of approaches for standard setting. Journal of Educational Measurement, 17, 167–178.
   Web of Science ®Google Scholar
• Livingston, S. A., & Zieky, M. J. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.
   Google Scholar
• Livingston, S. A., & Zieky, M. J. (1989). A comparative study of standard-setting methods. Applied Measurement in Education, 2, 121–141.
   Google Scholar
• McLaughlin, D. H. (1993). Validity of the 1992 NAEP achievement-level-setting process. In L. Shepard, R. Glaser, R. Linn, & G. Bohrnstedt (Eds.), Setting performance standards for student achievement: Background studies (pp. 81–122). Palo Alto, CA: National Academy of Education.
   Google Scholar
• McLaughlin, D. H., Dubois, P. A., Eaton, M. S., Erlich, D. E., O'Donnell, C. A., Stancavage, F. B., et al. (1993). Comparison of teachers' and researchers' ratings of students' performance in mathematics and reading with NAEP measurement of achievement levels. In L. Shepard, R. Glaser, R. Linn, & G. Bohrnstedt (Eds.), Setting performance standards for student achievement: Background studies (pp. 283–362). Palo Alto, CA: National Academy of Education.
   Google Scholar
• Mills, C. N. (1983). A comparison of three methods of establishing cutoff scores on criterion-referenced tests. Journal of Educational Measurement, 20, 283–292.
   Web of Science ®Google Scholar
• Mills, C. N., & Melican, G. J. (1988). Estimating and adjusting cutoff scores: Features of selected methods. Applied Measurement in Education, 1, 261–275.
   Google Scholar
• Mills, C. N., Melican, G. J., & Ahluwalia, N. T. (1991). Defining minimal competence. Educational Measurement: Issues and Practice, 10(2), 7–10.
   Google Scholar
• Moore, C. M. (1987). Group techniques for idea building. Newbury Park, CA: Sage.
   Google Scholar
• Norcini, J. (1994). Research on standards for professional licensure and certification examinations. Evaluation and the Health Professions, 17, 160–177.
   Web of Science ®Google Scholar
• Plake, B. S. (1998). Setting performance standards for professional licensure and certification. Applied Measurement in Education, 11, 65–80.
   Web of Science ®Google Scholar
• Plake, B. S., Melican, G. J., & Mills, C. N. (1991). Factors influencing intrajudge consistency during standard setting. Educational Measurement: Issues and Practice, 10(2), 15–26.
   Google Scholar
• Poggio, J. P., Glasnapp, D. R., & Eros, D. S. (1981, April). An empirical investigation of the Angoff, Ebel, and Nedelsky standard-setting methods. Paper presented at the meeting of American Educational Research Association, Los Angeles.
   Google Scholar
• Poggio, J. P., Glasnapp, D. R., & Eros, D. S. (1982, March). An evaluation of contrasting groups methods for setting standards. Paper presented at the meeting of the American Educational Research Association, New York.
   Google Scholar
• Raymond, M. R., & Reid, J. B. (2001). Who made thee a judge? Selecting and training participants for standard setting. In G. C. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 119–157). Mahwah, NJ: Erlbaum.
   Google Scholar
• Reckase, M. (1994, June). Standard-setting on performance assessments: A comparison of the paper selection method and contrasting groups method. Paper presented at the National Conference on Large-Scale Assessment, Albuquerque, NM.
   Google Scholar
• Shepard, L. A. (1979). Setting standards. In M. A. Bunda & J. R. Sanders (Eds.), Practices and problems in competency-based education (pp. 59–71). Washington, DC: National Council on Measurement in Education.
   Google Scholar
• Shepard, L. (1980). Technical issues in minimum competency testing. In D. C. Berliner (Ed.), Review of research in education: Vol. 8 (pp. 30–82). Washington, DC: American Educational Research Association.
   Google Scholar
• Shepard, L., Glaser, R., Linn, R., & Bohrnstedt, G. (1993). Setting performance standards for student achievement tests. Stanford, CA: National Academy of Education.
   Google Scholar
• Webb, M. W. II, & Miller, E. R. (1995, April). A comparison of the paper selection method and contrasting groups method for setting standards in constructed response items. Paper presented at the meeting of the National Council on Measurement in Education, San Francisco.
   Google Scholar
• Woehr, D. J., Arthur, W. Jr., & Fehrmann, M. L. (1991). An empirical comparison of cutoff score methods for content-related and criterion-related validity settings. Educational and Psychological Measurement, 51, 1029–1039.
   Web of Science ®Google Scholar
• Zieky, M. J., & Livingston, S. A. (1977). Manual for setting standards on the basic skills assessment tests. Princeton, NJ: Educational Testing Service.
   Google Scholar