What works best in a general practice specific OSCE for medical students: Mini-CEX or content-related checklists?

Abstract

Aim: To develop and pilot a General Practice (GPr) OSCE assessing medical students dealing with patient encounters, which are typical for GPr and to compare different measurement instruments (global ratings, content-specific checklists).

Methods: A blueprint based on Entrusted Professional Activities was used to develop prototypical OSCE stations. Four stations were tested with voluntary medical students. Students were videotaped and assessed with self-developed content-specific checklists, a global rating for communication skills, and mini-CEX. Results were compared according to students’ phases of studies.

Results: All three measurements were able to discriminate between clinical and pre-clinical students. Clearest results were achieved by using mini-CEX. Content-specific checklists were not able to differentiate between those groups for the more difficult stations. Inter-station reliability for the global ratings was sufficient for high-stakes exams. Students enjoyed the OSCE-setting simulating GPr consultation hours. They would prefer feedback from GPs after the OSCE and from simulated patients after each encounter.

Discussion and conclusion: Although the OSCE was short, results indicate advantages for using a global rating instead of checklists. Further research should include validating these results with a larger group of students and to find the threshold during the phases of education for switching from checklists to global ratings.

Keywords:

• Ambulatory medicine
• OSCE
• sychometrics
• undergraduate
• medicine

Acknowledgements

We thank all medical doctors (namely Benita Mangold, Rolf Stegemann, Anne Simmenroth, Jürgen in der Schmitten), students, simulated patients, and colleagues who helped us to conduct our study. We also thank our outstanding research assistants Katharina Schaefer, Katharina Blum, Claire Vogel, and Insa Wessels.

Ethical approval

The study protocol received approval by the Ethical Committee of the Medical Faculty of LMU Munich, Germany, proposal code 133-14.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Funding

The project was funded by the Dr. Hildegard-Hampp Foundation 2014.

Notes on contributors

Patrick Giemsa

Patrick Giemsa, MSc, is a Psychologist and Psychotherapist with his own practice. His area of research are social cognition and metacognition in patients with schizophrenia. He has also been lecturing statistics and research methods at MHB Neuruppin and SFU Berlin since 2014.

Clara Wübbolding

Clara Wübbolding, Dr. Med., studied medicine at the Ludwig-Maximilian-University until 2016. Central aspects of her dissertation related to different methods of assessing students especially in General Practice and Family Medicine. Since May 2017, she works as a resident physician at the ‘Israelitisches Krankenhaus’ in Hamburg.

Martin R. Fischer

Martin R. Fischer, MD, MME, FAMEE, is an internist and medical educator. He is a Professor of Medicine and the Director of the Institute for Medical Education at LMU Munich. He also serves as the Assoc. Dean for Clinical Studies at the Medical Faculty of LMU Munich.

Tanja Graupe

Tanja Graupe (née Pander), Dipl. Päd., is research assistant at the Institute for Medical Education at the University Hospital, LMU Munich since 2012. Her main research topics are communication, emotion handling skills and empathy as well as scientific and clinical career paths and mentoring.

Anja Härtl

Anja Härtl, MD, is responsible for the faculty development at Department for Medical Education and Educational Research, Faculty of Medicine, University of Augsburg. Her main research interest is the development and integration of the role as an educator for health professions.

Christine Lenz

Christine Lenz, Dr. Med., is working as a General Practitioner in her own practice since 2001. She is specialized in Internal Medicine and General Practice. She lectures General Medicine at LMU Munich since 2004.

Linda Sanftenberg

Linda Sanftenberg, Dr. rer. nat., is a research associate at the Institute of General Practice and Family Medicine, University Hospital, LMU, Munich, since 2014. The main topics of her studies are health promotion, preventive medicine, vaccination and immunization as well as travel medicine in General Practice.

Jörg Schelling

Jörg Schelling, Dr. Med, MD, is working as a General Practitioner in a family-owned practice since 2006. He is specialized in Internal Medicine and General Practice and an Honorary Professor at LMU Munich. From 2014 to 2016 he was chair and founding director of the Institute for General Practice and Family Medicine of LMU.

Katrin Schüttpelz-Brauns

Katrin Schüttpelz-Brauns, Dr. rer. nat., is the head of the Educational Research Team at the Department of Undergraduate Education and Educational Development, Medical Faculty Mannheim at Heidelberg University since 2012. Her main research topics are competency-based learning, teaching and assessing as well as formative assessments in medical education.

Claudia Kiessling

Claudia Kiessling, Dr. med., MPH, is a Professor in the Department of Health at the University of Witten/Herdecke and directs the curriculum for personal and professional development in medical education. Her research interests are clinical communication, patient-centred medicine, professional identity formation, and assessment.