The association between Situational Judgement Test (SJT) scores and professionalism concerns in undergraduate medical education

Abstract

Introduction: Situational judgement tests (SJTs) are widely used in selecting medical students and doctors. Emerging evidence suggests SJTs are capable of testing an individual’s ability to respond to role-relevant professionalism scenarios, however, evidence is lacking for their use in identifying students with concerning professional behaviours.

This study aimed to determine the association between medical student professionalism-based SJT scores and recorded professionalism concerns during training.

Methods: A cross-sectional study was conducted utilising SJT scores from second-year medical students and occurrences of student professionalism concerns. Concerns were reviewed and mapped to General Medical Council standards. Multivariate logistic regression was used to determine associations between SJT scores and professionalism concerns.

Results: 247 students were included in the study. For every point increase in SJT score, students were 10% less likely to have multiple professionalism concerns [OR (95% CI) 0.90 (0.83–0.97); p = .007].

Students scoring below 1 and 2 standard deviations from the mean score were 4 and 11-times more likely to have multiple concerns [OR (95% CI) 4.52 (1.12–18.25); p = .034] and [OR (95% CI) 11.45 (1.72–76.15); p = .012].

Conclusion: Lower SJT scores were significantly associated with an increased risk of professionalism concerns. These findings support the potential for SJT exams to identify medical students that may require closer supervision and remediation during undergraduate education.

Keywords:

• Situational Judgement Test
• SJT
• professionalism assessment
• professionalism concerns

Acknowledgments

Thanks to Dr. Edward Tyrrell for assistance in statistical analysis of the datasets.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Glossary

Glossary

Situational judgement tests (SJTs): Are a type of assessment modality. Their use developed from general and social intelligence testing making their way into medical education in the 1990s (Norcini and Shea 2016; Patterson et al. 2016). SJT questions are designed to test an individual’s ability to respond to role-relevant scenarios, often dealing with ethical issues in a plausible real job situation. They can be used in formative or summative examinations and modern medical situational judgement tests tend to be multiple-choice machine-marked examinations (Lievens et al. 2005; Patterson et al. 2016; Dore et al. 2017).

Dore KL, Reiter HI, Kreuger S, Norman GR. 2017. CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores. Advances in health sciences education: theory and practice. 22(2):327–336.

Lievens F, Buyse T, Sackett PR. 2005. The operational validity of a video-based situational judgment test for medical college admissions: illustrating the importance of matching predictor and criterion construct domains. The Journal of applied psychology. 90(3):442–452.

Norcini J, Shea J. 2016. Chapter 11, Assessment of professionalism in the development of a professional identity. In: Cruess RL CS, Steinert Y, editor. Teaching Medical Professionalism. Second ed. Cambridge: Cambridge University Press; p. 155- 168.

Patterson F, Zibarras L, Ashworth V. 2016. Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. Medical Teacher. 38(1):3–17.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Notes on contributors

Gurvinder S. Sahota

Dr. Gurvinder S. Sahota, MMedSci (merit) Medical Education, is a clinical academic and educator, responsible for first and second year medical students’ clinical and professional development at Nottingham Medical School.

Jaspal S. Taggar

Dr. Jaspal S. Taggar, MSc Applied Epidemiology, PhD, is a Clinical Associate Professor and educator, who is Head of the Primary Care Education Unit at Nottingham University. He is responsible for strategic oversight and quality assurance of clinical placements and undergraduate primary care education at Nottingham Medical School.