Refresh of a Clinical Skills Assessment for Physician Trainees

Abstract

Purpose

The Royal Australasian College of Physicians (RACP) oversees physician training across Australia and Aotearoa New Zealand. Success in a written examination and clinical skills assessment (known as the clinical examination) at the mid-point of training is a requirement to progress from basic to advanced training. The clinical examination had evolved over many years without a review process. This paper describes the approach taken, the changes made and the evaluation undertaken as part of a formal review.

Methods

A working party that included education experts and examiners experienced in the assessment of clinical skills was established. The purpose of the clinical examination and competencies being assessed were clarified and were linked to learning objectives. Significant changes to the marking and scoring approaches resulted in a more holistic approach to the assessment of candidate performance with greater transparency of standards. Evaluation over a 2-year period was undertaken before the adoption of the new approach in 2019.

Results

In 2017 testing of a new marking rubric occurred during the annual examination cycle which confirmed feasibility and acceptability. The following year an extensive trial utilising the new marking rubric and a new scoring approach took place involving 1142 examiners, 880 candidates and 5280 scoresheets which led to some minor modifications to the scoring system. The final marking and scoring approaches resulted in unchanged pass rates and improved inter-rater reliability. Feedback from examiners confirmed that the new marking and scoring approaches were easier to use and enabled better feedback on performance for candidates.

Conclusion

The refresh of the RACP clinical examination has resulted in an assessment that has clarity of purpose, is linked to learning objectives, has greater transparency of expected standards, has improved inter-rater reliability, is well accepted by examiners and enables feedback on examination performance to candidates.

Keywords:

• physician assessment
• reliability
• clinical assessment
• long case
• short case

Ethics Approval

As this was a quality improvement activity related to RACP clinical examinations, no institutional review or ethics approval was sought. However, the manuscript was reviewed by the Executive General Manager, Education, Learning and Assessment, RACP and the Chair of the Education Committee, RACP who both provided input into the final manuscript.

Acknowledgments

The authors wish to acknowledge the clinical examination review working group, the Royal Australasian College of Physicians Examination Unit and the National Examiner Panel for their contribution to the refresh of the clinical examination.

Disclosure

The authors report no conflicts of interest in this work.