Twelve tips for implementing tools for direct observation of medical trainees’ clinical skills during patient encounters

References

• Bing-You RG, Trowbridge RL. Why medical educators may be failing at feedback. JAMA 2009; 302: 1330–1331
   PubMed Web of Science ®Google Scholar
• Braddock III CH, III, Edwards KA, Hasenberg NM, Laidley TL, Levinson W. Informed decision making in outpatient practice: Time to get back to basics. JAMA 1999; 282: 2313–2320
   PubMedGoogle Scholar
• Burdick WP, Schoffstall J. Observation of emergency medicine residents at the bedside: How often does it happen?. Acad Emerg Med 1995; 2: 909–913
   PubMed Web of Science ®Google Scholar
• Cook DA, Dupras DM, Beckman TJ, Thomas KG, Pankratz VS. Effect of rater training on reliability and accuracy of mini-CEX scores: A randomized, controlled trial. J Gen Intern Med 2009; 24: 74–79
   PubMed Web of Science ®Google Scholar
• Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA 2006; 296: 1094–1102
   PubMed Web of Science ®Google Scholar
• de Haes JC, Oort F, Oosterveld P, Ten Cate O. Assessment of medical students’ communicative behaviour and attitudes: Estimating the reliability of the use of the Amsterdam attitudes and communication scale through generalisability coefficients. Patient Educ Couns 2001; 45: 35–42
   PubMed Web of Science ®Google Scholar
• Downing SM. Validity: On meaningful interpretation of assessment data. Med Educ 2003; 37: 830–837
   PubMed Web of Science ®Google Scholar
• Duffy FD, Gordon GH, Whelan G, Cole-Kelly K, Frankel R, Buffone N, Lofton S, Wallace M, Goode L, Langdon L. Assessing competence in communication and interpersonal skills: The Kalamazoo II report. Acad Med 2004; 79: 495–507
   PubMed Web of Science ®Google Scholar
• Ende J. Feedback in clinical medical education. JAMA 1983; 250: 777–781
   PubMed Web of Science ®Google Scholar
• Ericsson K, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993; 100: 363–406
   Web of Science ®Google Scholar
• Eva KW, Regehr G. “I’ll never play professional football” and other fallacies of self-assessment. J Contin Educ Health Prof 2008; 28: 14–19
   PubMed Web of Science ®Google Scholar
• Govaerts MJ, van der Vleuten CP, Schuwirth LW, Muijtjens AM. The use of observational diaries in in-training evaluation: Student perceptions. Adv Health Sci Educ Theory Pract 2005; 10: 171–188
   PubMed Web of Science ®Google Scholar
• Graber ML. Taking steps towards a safer future: Measures to promote timely and accurate medical diagnosis. Am J Med 2008; 121: S43–S46
   PubMedGoogle Scholar
• Graber M, Franklin N, Gordon R, Power D, Shanafelt T. Diagnostic error in internal medicine. Arch Intern Med 2005; 165: 1493–1499
   PubMed Web of Science ®Google Scholar
• Han PK, Keranen LB, Lescisin DA, Arnold RM. The palliative care clinical evaluation exercise (CEX): An experience-based intervention for teaching end-of-life communication skills. Acad Med 2005; 80: 669–676
   PubMed Web of Science ®Google Scholar
• Hasnain M, Bordage G, Connell K, Sinacore J. History-taking behaviors associated with diagnostic competence of clerks: An exploratory study. Acad Med 2001; 76: S14–S17
   PubMed Web of Science ®Google Scholar
• Hattie JA, Timperly H, Rogers HJ. The power of feedback. Rev Res Educ 2007; 77: 81–112
   Google Scholar
• Hemmer PA, Pangaro L. Using formal evaluation sessions for case-based faculty development during clinical clerkships. Acad Med 2000; 75: 1216–1221
   PubMed Web of Science ®Google Scholar
• Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents’ clinical competence: A randomized trial. Ann Intern Med 2004a; 140: 874–881
   PubMed Web of Science ®Google Scholar
• Holmboe ES, Yepes M, Williams F, Huot SJ. Feedback and the mini clinical evaluation exercise. J Gen Intern Med 2004b; 19: 558–561
   PubMed Web of Science ®Google Scholar
• Howley LD, Wilson WG. Direct observation of students during clerkship rotations: A multiyear descriptive study. Acad Med 2004; 79: 276–280
   PubMed Web of Science ®Google Scholar
• Kilminster S, Jolly B, van der Vleuten CP. A framework for effective training for supervisors. Med Teach 2002; 24: 385–389
   PubMed Web of Science ®Google Scholar
• Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review. JAMA 2009; 302: 1316–1326
   PubMed Web of Science ®Google Scholar
• Kotter J. Leading change: Why transformation efforts fail. Harv Bus Rev 1995; 73: 59–67
   Web of Science ®Google Scholar
• Kuo AK, Irby DI, Loeser H. Does direct observation improve medical students’ clerkship experiences?. Med Educ 2005; 39: 518
   PubMed Web of Science ®Google Scholar
• Lane JL, Gottlieb RP. Structured clinical observations: A method to teach clinical skills with limited time and financial resources. Pediatrics 2000; 105: 973–977
   PubMed Web of Science ®Google Scholar
• Malhotra S, Hatala R, Courneya CA. Internal medicine residents’ perceptions of the Mini-Clinical Evaluation Exercise. Med Teach 2008; 30: 414–419
   PubMed Web of Science ®Google Scholar
• Mangione S, Burdick WP, Peitzman SJ. Physical diagnosis skills of physicians in training: A focused assessment. Acad Emerg Med 1995; 2: 622–629
   PubMed Web of Science ®Google Scholar
• Mckinley RK, Fraser RC, van der Vleuten C, Hastings AM. Formative assessment of the consultation performance of medical students in the setting of general practice using a modified version of the leicester assessment package. Med Educ 2000; 34: 573–579
   PubMed Web of Science ®Google Scholar
• Messick S. Standards of validity and the validity of standards in performance assessment. Educ Meas 1995; 14: 5–8
   Google Scholar
• Mitka M. Many physician practices fall short on accurate blood pressure measurement. JAMA 2008; 299: 2842–2844
   PubMedGoogle Scholar
• Murphy KR, Cleveland JN, Skattebo AL, Kinney TB. Raters who pursue different goals give different ratings. J Appl Psychol 2004; 89: 158–164
   PubMed Web of Science ®Google Scholar
• Noel GL, Herbers Jr JE, Jr, Caplow MP, Cooper GS, Pangaro LN, Harvey J. How well do internal medicine faculty members evaluate the clinical skills of residents?. Ann Intern Med 1992; 117: 757–765
   PubMed Web of Science ®Google Scholar
• Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: A method for assessing clinical skills. Ann Intern Med 2003; 138: 476–481
   PubMed Web of Science ®Google Scholar
• PMETB [Internet]. 2008. Postgraduate Medical Education Training Board. Standards for curriculum and assessment systems. United Kingdom. [Published 2009 November 20]. Available from: http://www.pmetb.org.uk/fileadmin/user/Standards_Requirements/PMETB_Scas_July2008_Final.pdf
   Google Scholar
• Ross R. A clinical-performance biopsy instrument. Acad Med 2002; 77: 268
   PubMedGoogle Scholar
• Shumway JM, Harden RM. AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician. Med Teach 2003; 25: 569–584
   PubMed Web of Science ®Google Scholar
• Simpson M, Buckman R, Stewart M, Maguire P, Lipkin M, Novack D, Till J. Doctor-patient communication: The Toronto consensus statement. BMJ 1991; 303: 1385–1387
   PubMed Web of Science ®Google Scholar
• Steinert Y, Cruess RL, Cruess SR, Boudreau JD, Fuks A. Faculty development as an instrument of change: A case study on teaching professionalism. Acad Med 2007; 82: 1057–1064
   PubMed Web of Science ®Google Scholar
• Stewart MA. Effective physician-patient communication and health outcomes: A review. CMAJ 1995; 152: 1423–1433
   PubMed Web of Science ®Google Scholar
• Torre DM, Sebastian JL, Simpson DE. A PDA-based instructional tool to monitor students’ cardiac auscultation during a medicine clerkship. Med Teach 2005; 27: 559–561
   PubMed Web of Science ®Google Scholar
• Torre DM, Simpson DE, Elnicki DM, Sebastian JL, Holmboe ES. Feasibility, reliability and user satisfaction with a PDA-based mini-CEX to evaluate the clinical skills of third-year medical students. Teach Learn Med 2007; 19(3): 271–277
   Google Scholar
• van de Ridder JM, Stokking KM, Mcgaghie WC, Ten Cate OT. What is feedback in clinical education?. Med Educ 2008; 42: 189–197
   PubMed Web of Science ®Google Scholar
• van der Vleuten CP, Schuwirth LW. Assessing professional competence: From methods to programmes. Med Educ 2005; 39: 309–317
   PubMed Web of Science ®Google Scholar
• Vukanovic-Criley JM, Criley S, Warde CM, Boker JR, Guevara-Matheus L, Churchill WH, Nelson WP, Criley JM. Competency in cardiac examination skills in medical students, trainees, physicians, and faculty: A multicenter study. Arch Intern Med 2006; 166: 610–616
   PubMed Web of Science ®Google Scholar
• Westberg J. Helping learners become reflective practitioners. Educ Health (Abingdon) 2001; 14: 313–321
   PubMedGoogle Scholar
• Williams RG, Klamen DA, McGaghie WC. Cognitive, social and environmental sources of bias in clinical performance ratings. Teach Learn Med 2003; 15: 270–292
   PubMed Web of Science ®Google Scholar