Versatility in multiple mini-interview implementation: Rater background does not significantly influence assessment scoring

References

• Anderson N, Shackleton V. 1990. Decision making in the graduate selection interview: a field study. J Occup Psychol. 63(1):63–76.
   Web of Science ®Google Scholar
• Bernabeo E, Iobst W, Holmboe E. 2011. Opening the black box of clinical skills assessment via observation: a conceptual model. Med Educ. 45(10):1048–1060.
   PubMed Web of Science ®Google Scholar
• Bibler Zaidi NL, Santen SA, Purkiss JA, Teener CA, Gay SE. 2016. A hybrid interview model for medical school interviews: combining traditional and multisampling formats. Acad Med. 91(11):1526–1529.
   PubMed Web of Science ®Google Scholar
• Chong L, Taylor S, Haywood M, Adelstein B-A, Shulruf B. 2017. The sights and insights of examiners in objective structured clinical examinations. J Educ Eval Health Prof. 14:34.
   PubMed Web of Science ®Google Scholar
• Christensen MK, Lykkegaard E, Lund O, O’Neill LD. 2018. Qualitative analysis of MMI raters’ scorings of medical school candidates: a matter of taste? Adv Health Sci Educ. 23(2):289–310.
   PubMed Web of Science ®Google Scholar
• Downing SM, Haladyna TM. 2004. Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ. 38(3):327–333.
   PubMed Web of Science ®Google Scholar
• Eva KW, Reiter HI, Rosenfeld J, Norman GR. 2004. The relationship between interviewers’ characteristics and ratings assigned during a multiple mini-interview. Acad Med. 79(6):602.
   PubMed Web of Science ®Google Scholar
• Feldman M, Lazzara EH, Vanderbilt AA, DiazGranados D. 2012. Rater training to support high-stakes simulation-based assessments. J Contin Educ Health Prof. 32(4):279–286.
   PubMed Web of Science ®Google Scholar
• Gingerich A, Kogan J, Yeates P, Govaerts M, Holmboe E. 2014. Seeing the ‘black box’ differently: assessor cognition from three research perspectives. Med Educ. 48(11):1055–1068.
   PubMed Web of Science ®Google Scholar
• Gingerich A, Regehr G, Eva KW. 2011. Rater-based assessments as social judgments: rethinking the etiology of rater errors. Acad Med. 86:S1.
   PubMed Web of Science ®Google Scholar
• Glazer G, Startsman LF, Bankston K, Michaels J, Danek JC, Fair M. 2016. How many schools adopt interviews during the student admission process across the health professions in the United States of America? J Educ Eval Health Prof. 13:12.
   PubMed Web of Science ®Google Scholar
• Griffin BN, Wilson IG. 2010. Interviewer bias in medical student selection. Med J Aust. 193(6):343–346.
   PubMed Web of Science ®Google Scholar
• Harris S, Owen C. 2007. Discerning quality: using the multiple mini-interview in student selection for the Australian National University Medical School. Med Educ. 41(3):234–241.
   PubMed Web of Science ®Google Scholar
• Hecker K, Donnon T, Fuentealba C, Hall D, Illanes O, Morck DW, Muelling C. 2009. Assessment of applicants to the veterinary curriculum using a multiple mini-interview method. J Vet Med Educ. 36(2):166–173.
   PubMed Web of Science ®Google Scholar
• Holt RW, Hansberger JT, Boehm-Davis DA. 2002. Improving rater calibration in aviation: a case study. Int J Aviat Psychol. 12(3):305–330.
   Web of Science ®Google Scholar
• Kogan JR, Hess BJ, Conforti LN, Holmboe ES. 2010. What drives faculty ratings of residents’ clinical skills? The impact of faculty’s own clinical skills. Acad Med. 85:S25.
   PubMed Web of Science ®Google Scholar
• Leduc J-M, Rioux R, Gagnon R, Bourdy C, Dennis A. 2017. Impact of sociodemographic characteristics of applicants in multiple mini-interviews. Med Teach. 39(3):285–294.
   PubMed Web of Science ®Google Scholar
• Murphy KR, Anhalt RL. 1992. Is halo error a property of the rater, ratees, or the specific behaviors observed? J Appl Psychol. 77(4):494–500.
   Web of Science ®Google Scholar
• O’Brien A, Harvey J, Shannon M, Lewis K, Valencia O. 2011. A comparison of multiple mini-interviews and structured interviews in a UK setting. Med Teach. 33:397–402.
   PubMed Web of Science ®Google Scholar
• Patterson F, Rowett E, Hale R, Grant M, Roberts C, Cousans F, Martin S. 2016. The predictive validity of a situational judgement test and multiple-mini interview for entry into postgraduate training in Australia. BCM Med Educ. 16:87.
   Web of Science ®Google Scholar
• Rees EL, Hawarden AW, Dent G, Hays R, Bates J, Hassell AB. 2016. Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: a BEME systematic review: BEME Guide No. 37. Med Teach. 38(5):443–455.
   PubMed Web of Science ®Google Scholar
• Reiter HI, Lockyer J, Ziola B, Courneya C-A, Eva K, Alliance (CaMMIRA), for the C.M.M.-I.R. 2012. Should efforts in favor of medical student diversity be focused during admissions or farther upstream? Acad Med. 87(4):443.
   PubMed Web of Science ®Google Scholar
• Roberts WL, Pugliano G, Langenau E, Boulet JR. 2012. Modeling relationships between traditional preadmission measures and clinical skills performance on a medical licensure examination. Adv Health Sci Educ. 17(3):403–417.
   PubMed Web of Science ®Google Scholar
• Roberts C, Rothnie I, Zoanetti N, Crossley J. 2010. Should candidate scores be adjusted for interviewer stringency or leniency in the multiple mini-interview? Med Educ. 44(7):690–698.
   PubMed Web of Science ®Google Scholar
• Ross M, Walker I, Cooke L, Raman M, Ravani P, Coderre S, McLaughlin K. 2017. Are female applicants rated higher than males on the multiple mini-interview? Findings from the University of Calgary. Acad Med. 92(6):841–846.
   PubMed Web of Science ®Google Scholar
• Schwartzstein RM. 2015. Getting the right medical students — nature versus nurture. N Engl J Med. 372(17):1586–1587.
   PubMed Web of Science ®Google Scholar
• Sebok SS, Syer MD. 2015. Seeing things differently or seeing different things? Exploring raters’ associations of noncognitive attributes. Acad Med. 90:S50.
   PubMed Web of Science ®Google Scholar
• Uijtdehaage S, Doyle L, Parker N. 2011. Enhancing the reliability of the multiple mini-interview for selecting prospective health care leaders. Acad Med. 86(8):1032.
   PubMed Web of Science ®Google Scholar
• Weber DE, Roth W-M, Mavin TJ, Dekker SW. 2013. Should we pursue inter-rater reliability or diversity? An empirical study of pilot performance assessment. Aviation in focus. J Aeronaut Sci. 4(2):34–58.
   Google Scholar
• Wexley KN, Youtz MA. 1985. Rater beliefs about others: their effects on rating errors and rater accuracy. J Occup Psychol. 58(4):265–275.
   Web of Science ®Google Scholar
• Yule S, Rowley D, Flin R, Maran N, Youngson G, Duncan J, Paterson-Brown S. 2009. Experience matters: comparing novice and expert ratings of non-technical skills using the NOTSS system. ANZ J Surg. 79(3):154–160.
   PubMed Web of Science ®Google Scholar