Thinking on the edge: the influence of discussion and statistical data on awarders’ perceptions of borderline candidates in an Angoff awarding meeting

References

• Angoff, W 1971, Scales, norms and equivalent scores, American Council on Education, Washington, DC.
   Google Scholar
• Asch, S E 1951, ‘Effects of group pressure upon the modification and distortion of judgments’, in Groups, Leadership and Men, ed H Guetzkow, Carnegie Press, Pittsburgh.
   Google Scholar
• Berk, R1996, ‘Standard setting: the next generation (where few psychometricians have gone before!)’, Applied Measurement in Education, vol 9, pp.215–235.
   Google Scholar
• Boursicot, K & Roberts, T 2006, ‘Setting standards in a professional higher education course: Defining the concept of the minimally competent student in performance based assessment at the level of graduation from medical school’, Higher Education Quarterly, vol 60, pp.74–90.
   Google Scholar
• Busch, J & Jaeger, R1990, ‘Influence of type of judge, normative information, and discussion on standards recommended for the National Teachers Examinations’, Journal of Educational Measurement, vol 27, no.2, pp.145–163.
   Google Scholar
• Carrwright, D & Zander, A 1960, Group dynamics (2nd ed), Row, Peterson and Company, Evanston IL.
   Google Scholar
• Cizek, G 1996, ‘Setting passing scores’, Educational Measurement: Issues and Practice, vol 15, no.2, pp.20–31.
   Google Scholar
• Ferdous, A, Nering, M & Plake, B 2006, ‘Factors that influence judges’ decisions in an Angoff standard setting study’, Paper presented at the 2006 American Educational Research Association Annual Meeting, San Francisco, CA.
   Google Scholar
• Fitzpatrick, A 1984, ‘Social influences in standard-setting: The effect of group interaction on individuals’ judgement’, Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
   Google Scholar
• Giraud, G, Impara, J & Plake, B 2005, ‘Teachers’ conceptions of the target examinee in Angoff standard setting’, Applied Measurement in Education, vol 18, no 3, pp223–232.
   Google Scholar
• Goodwin, L 1999, ‘Relations between observed item difficulty levels and Angoff minimum passing levels for a group of borderline examinees’, Applied Measurement in Education, vol 12, pp.13–28.
   Google Scholar
• Laming, D 2004, Human judgment: The eye of the beholder, Thomson, London.
   Google Scholar
• Hambleton, R K, Brennan, R L, Brown, W, Dodd, Forsyth, R A, Mehrens, W A, Nellhaus, J, Reckase, M, Rindone, D, van der Linden, W J, & Zwick, R 2000 ‘A response to “Setting reasonable and useful performance standards” in the National Academy of Sciences’ Grading the Nation’s Report Card’, Educational Measurement: Issues and Practice, vol 19, no.2, pp.5–14.
   Google Scholar
• Hayes, M 2001, ‘The role of the Angoff procedures in the level setting processes for the end of Key Stage tests in mathematics’, AGRAQ paper, Qualifications and Curriculum Authority internal report.
   Google Scholar
• Impara J & Plake, B 1997, ‘Standard setting: an alternative approach’, Journal of Educational Measurement, vol 34, no.4, pp.353–366.
   Google Scholar
• Impara, J & Plake, B 1998, ‘Teachers’ ability to estimate item difficulty: a test of the assumptions in the Angoff standard setting method’, Journal of Educational Measurement, vol 35, no.l, pp.69–81.
   Google Scholar
• Johnson, M 2007, ‘Does the anticipation of a grade motivate vocational test takers?’, Research in Post Compulsory Education, vol 12, no.2, pp.159–179.
   Google Scholar
• Murphy, R, Burke, P, Cotton, T, Hancock, J, Partington, J, Robinson, C, Tolley, H, Wilmut, J, Gower, R 1995, ‘The dynamics of GCSE awarding’, Report of a project conducted for the School Curriculum and Assessment Authority’, School of Education, University of Nottingham.
   Google Scholar
• Norcini, J & Shea, J 1997, ‘The credibility and comparability of standards’, Applied Measurement in Education, vol 10, no.1, pp.39–59.
   Google Scholar
• Novakovic, N 2008, ‘The influence of performance data on awarders’ estimates in an Angoff awarding meetings’, Research Matters: A Cambridge Assessment Publication, vol 5, pp.15–19.
   Google Scholar
• Plake, B & Impara, J 2001, ‘Ability of panelists to estimate item performance for a target group of candidates: an issue in judgmental standard setting’, Educational Assessment, vol 7, no.2, pp.87–97.
   Google Scholar
• Ricker, K 2006, ‘Setting cut-scores: a critical review of the Angoff and modified Angoff methods’, The Alberta Journal ofEducational Research, vol 52, no.1, pp.53–64.
   Google Scholar
• Shepard, L A 1995, ‘Implications for standard setting of the National Academy of Education evaluation of National Assessment of Educational Progress achievement levels’, Proceedings of the Joint Conference on Standard Setting for Large-Scale Assessments, U.S. Government Printing Office, Washington DC.
   Google Scholar
• Sherif, M 1935, ‘A study of some social factors in perception’, Archives of Psychology, vol 22, no.187.
   Google Scholar
• Sizmur, S 1997, ‘Look back in Angoff: a cautionary tale’, British Educational Research Journal, vol 23, no.1, pp.3–13.
   Google Scholar
• Skorupski, W & Hambleton, R 2005, ‘What are panelists thinking when they participate in standard-setting studies?’, Applied Measurement in Education, vol 18, no.3, pp.233–356.
   Google Scholar