Numerical versus narrative: A comparison between methods to measure medical student performance during clinical clerkships

References

• Battistone MJ, Milne C, Sande MA, Pangaro LN, Hemmer PA, Samuel Shomaker T. 2002. The feasibility and acceptability of implementing formal evaluation sessions and using descriptive vocabulary to assess student performance on a clinical clerkship. Teach Learn Med. 14:5–10.
   PubMed Web of Science ®Google Scholar
• Battistone MJ, Pendleton B, Milne C, Battistone ML, Sande MA, Hemmer PA, Shomaker TS. 2001. Global descriptive evaluations are more responsive than global numeric ratings in detecting students' progress during the inpatient portion of an internal medicine clerkship. Acad Med. 76:S107.
   Web of Science ®Google Scholar
• Carifio J, Perla R. 2008. Resolving the 50-year debate around using and misusing Likert scales. Med Educ. 42:1150–1152.
   PubMed Web of Science ®Google Scholar
• Charmaz K. Constructing grounded theory. London: Sage; 2014.
   Google Scholar
• Cohen GS, Blumberg P, Ryan NC, Sullivan PL. 1993. Do final grades reflect written qualitative evaluations of student performance? Teach Learn Med. 5:10–15.
   Google Scholar
• Epstein RM. 2007. Assessment in medical education. N Engl J Med. 356:387–396.
   PubMed Web of Science ®Google Scholar
• Epstein RM, Hundert EM. 2002. Defining and assessing professional competence. JAMA. 287:226–235.
   PubMed Web of Science ®Google Scholar
• Eva KW, Hodges BD. 2012. Scylla or Charybdis? Can we navigate between objectification and judgement in assessment? Med Educ. 46:914–919.
   PubMed Web of Science ®Google Scholar
• Gingerich A, Regehr G, Eva KW. 2011. Rater-based assessments as social judgments: rethinking the etiology of rater errors. Acad Med. 86:S7.
   Web of Science ®Google Scholar
• Ginsburg S, van der Vleuten CP, Eva KW. 2017. The hidden value of narrative comments for assessment: a quantitative reliability analysis of qualitative data. Acad Med. [accessed 2017 Apr 11]. DOI:10.1097/ACM.0000000000001669
   Web of Science ®Google Scholar
• Ginsburg S. 2011. Respecting the expertise of clinician assessors: construct alignment is one good answer. Med Educ. 45:546–548.
   PubMed Web of Science ®Google Scholar
• Ginsburg S, Eva K, Regehr G. 2013. Do in-training evaluation reports deserve their bad reputations? a study of the reliability and predictive ability of ITER scores and narrative comments. Acad Med. 88:1539–1544.
   PubMed Web of Science ®Google Scholar
• Ginsburg S, Gold W, Cavalcanti RB, Kurabi B, McDonald-Blumer H. 2011. Competencies plus: the nature of written comments on internal medicine residents' evaluation forms. Acad Med. 86:S34.
   PubMed Web of Science ®Google Scholar
• Ginsburg S, McIlroy J, Oulanova O, Eva K, Regehr G. 2010. Toward authentic clinical evaluation: pitfalls in the pursuit of competency. Acad Med. 85:780–786.
   PubMed Web of Science ®Google Scholar
• Ginsburg S, Regehr G, Lingard L, Eva KW. 2015. Reading between the lines: faculty interpretations of narrative evaluation comments. Med Educ. 49:296–306.
   PubMed Web of Science ®Google Scholar
• Govaerts MJB, Van der Vleuten CPM, Schuwirth LWT, Muijtjens AMM. 2007. Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Adv Health Sci Educ Theory Pract Educ. 12:239–260.
   PubMed Web of Science ®Google Scholar
• Govaerts M, Vleuten CPM. 2013. Validity in work‐based assessment: expanding our horizons. Med Educ. 47:1164–1174.
   PubMed Web of Science ®Google Scholar
• Guerrasio J, Cumbler E, Trosterman A, Wald H, Brandenburg S, Aagaard E. 2012. Determining need for remediation through postrotation evaluations. J Grad Med Educ. 4:47–51.
   PubMedGoogle Scholar
• Hanson JL, Rosenberg AA, Lane JL. 2013. Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United States. Front Psychol. 4:668.
   PubMed Web of Science ®Google Scholar
• Hatala R, Sawatsky AP, Dudek N, Ginsburg S, Cook DA. 2017. Using in-training evaluation report (ITER) qualitative comments to assess medical students and residents: a systematic review. Acad Med. 92:868–879.
   PubMed Web of Science ®Google Scholar
• Hodges B. 2013. Assessment in the post-psychometric era: learning to love the subjective and collective. Med Teach. 35:564–568.
   PubMed Web of Science ®Google Scholar
• Hoffman K, Hosokawa M, Donaldson J. 2009. What criteria do faculty use when rating students as potential house officers? Med Teacher. 31:e417.
   Web of Science ®Google Scholar
• Kuper A, Reeves S, Albert M, David Hodges B. 2007. Assessment: do we need to broaden our methodological horizons? Med Educ. 41:1121–1123.
   PubMed Web of Science ®Google Scholar
• Lurie SJ, Mooney CJ, Lyness JM. 2009. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Acad Med. 84:301–309.
   PubMed Web of Science ®Google Scholar
• Nasca TJ, Philibert I, Brigham T, Flynn TC. 2012. The next GME accreditation system-rationale and benefits. N Engl J Med. 366:1051–1056.
   PubMed Web of Science ®Google Scholar
• Pangaro LN. 2000. Investing in descriptive evaluation: a vision for the future of assessment. Med Teacher. 22:478–481.
   PubMed Web of Science ®Google Scholar
• Pangaro L. 1999. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med. 74:1203–1207.
   PubMed Web of Science ®Google Scholar
• Regehr G, Ginsburg S, Herold J, Hatala R, Eva K, Oulanova O. 2012. Using standardized narratives to explore new ways to represent faculty opinions of resident performance. Acad Med. 87:419–427.
   PubMed Web of Science ®Google Scholar
• Schuwirth L, Van Der Vleuten C. 2004. Merging views on assessment. Med Educ. 38:1208–1210.
   PubMed Web of Science ®Google Scholar
• Schuwirth LWT, Vleuten CPM. 2006. A Plea for new psychometric models in educational assessment. Med Educ. 40:296–300.
   PubMed Web of Science ®Google Scholar
• Schwind CJ, Williams RG, Boehler ML, Dunnington GL. 2004. Do individual attendings' post-rotation performance ratings detect residents' clinical performance deficiencies? Acad Med. 79:453–457.
   PubMed Web of Science ®Google Scholar
• Tamblyn R, Abrahamowicz M, Dale Dauphinee W, Hanley JA, Norcini J, Girard N, Grand'Maison P, Brailovsky C. 2002. Association between licensure examination scores and practice in primary care. JAMA. 288:3019–3026.
   PubMed Web of Science ®Google Scholar
• Tamblyn R, Abrahamowicz M, Dauphinee D, Wenghofer E, Jacques A, Klass D, Smee S, Blackmore D, Winslade N, Girard N. 2007. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA. 298:993–1001.
   PubMed Web of Science ®Google Scholar
• Van der Vleuten CPM, Schuwirth LWT, Scheele F, Driessen EW, Hodges B. 2010. The assessment of professional competence: building blocks for theory development. Best Prac Res Clin Obstet Gynaecol. 24:703–719.
   PubMed Web of Science ®Google Scholar