Deanery placement and the Situational Judgement Test

Dear editor

Singagireson et al¹ explored the contentious issue of allocating Foundation Programme posts using the Situational Judgement Test (SJT), a system that has been criticized on the basis of fairness.^2,3 The authors found evidence⁴ that suggested academic performance at medical school did not correlate with SJT scores, and suggested that relative success on the SJT could artificially inflate academic scores attained at medical school. They concluded that the SJT was a fair means of assessing other skills required in the role of a physician. Whilst we are in agreement, the argument they present fails to examine the reasoning behind using the SJT.

The reliability of SJTs in general is fairly good,^5–7 and it is a system used in general practice national training allocation in the UK and abroad.⁸ However, the statement “logic suggests if you have done well throughout medical school, you would have gained the knowledge and aptitude to perform well in the SJT” is a straw-man argument. The evidence the authors use to refute this did not show a statistically significant correlation between academic performance and SJT score. The data set was limited by its sample size (n=239), and there was risk of selection bias with inclusion of data from 36% of all UK medical schools (n=12), in addition to reporting bias.⁴ However, this is irrelevant, as the SJT is not something you can revise for. Indeed, the SJT does not require aptitude or knowledge, nor should it be expected to assess academic rigor. It is an interdisciplinary psychometric test used to assess a candidate’s behavioral tendencies in different workplace situations. Within this context, it tests commitment to professionalism, coping under pressure, effective communication, patient focus, and effective teamwork.⁹ These are skills that form gradually, and cannot be learned from a textbook. Becoming a good doctor goes beyond academic rating.

Moreover, in stark contrast to their claim that an “off day” during the SJT would detrimentally affect deanery allocation, statistics from this year identified that 90.5% of medical students were allocated one of their top five deanery choices (n=8119).¹⁰ They also postulated on the “selective advantage” when an academically gifted student enrolls at a less competitive university, altogether insinuating that somehow medical school applicants strategically calculate which university to attend on the basis of academic competition. One would imagine that according to their article, the SJT would have a large spread in scores to be able to mitigate for the variability in academic ranking. This year, average SJT scores were between 37.4 and 40.9 (out of 50) in UK medical schools. This is less than a 9% variation; significant, but not sweeping.¹⁰

Finally, we would like to call into question the importance of there being interuniversity variability in quality of graduates. Perhaps the authors were alluding to the degree of preparedness a medical student felt in being able to practise confidently as a Foundation Year 1 (F1) doctor. A report compiled by the regulating authority, the General Medical Council (GMC), stated that overall medical graduates are good F1s, despite wide variation in the proportion of medical students feeling prepared for Foundation practice.¹¹ The GMC looked at past research¹² and commissioned a rapid review by Monrouxe et al.¹³ Preparedness among newly qualified F1s is improving: 70% felt prepared for their new posts in 2014 compared to 51% in 2012. Changes in question wording could have been a factor in this; however, graduate preparedness has been steadily increasing since 2009.¹¹ More importantly, the number of doctors in difficulty has declined from 4.6% in 2010 to 2.6% in 2013 for F1s.¹¹

Daily working life requires interpersonal skills and emotional durability that ought to be assessed. Greater engagement from medical students in the realities of ward work can be a challenge, and any pressure to shift the focus away from theoretical knowledge is welcome. The context-dependent syllabus of the SJT could incentivize students to spend time shadowing on the wards. Medicine is a vocation; preparedness or quality of F1s is dependent on experience. Surely the goal of the SJT is to motivate better comprehension of the GMC Code of Conduct through application, not study.

Disclosure

The authors report no conflicts of interest in this communication.