Extended score interval in the assessment of basic surgical skills

Abstract

Introduction

The Basic Surgical Skills course uses an assessment score interval of 0–3. An extended score interval, 1–6, was proposed by the Swedish steering committee of the course. The aim of this study was to analyze the trainee scores in the current 0–3 scored version compared to a proposed 1–6 scored version.

Methods

Sixteen participants, seven females and nine males, were evaluated in the current and proposed assessment forms by instructors, observers, and learners themselves during the first and second day. In each assessment form, 17 tasks were assessed. The inter-rater reliability between the current and the proposed score sheets were evaluated with intraclass correlation (ICC) with 95% confidence intervals (CI).

Results

The distribution of scores for ‘knot tying’ at the last time point and ‘bowel anastomosis side to side’ given by the instructors in the current assessment form showed that the highest score was given in 31 and 62%, respectively. No ceiling effects were found in the proposed assessment form. The overall ICC between the current and proposed score sheets after assessment by the instructors increased from 0.38 (95% CI 0.77–0.78) on Day 1 to 0.83 (95% CI 0.51–0.94) on Day 2.

Discussion

A clear ceiling effect of scores was demonstrated in the current assessment form, questioning its validity. The proposed score sheet provides more accurate scores and seems to be a better feedback instrument for learning technical surgical skills in the Basic Surgical Skills course.

• surgical education
• basic surgical skills
• residents
• score interval
• assessment

Acknowledgements

We thank the members of the Swedish steering committee of the Basic Surgical Skills course, the Swedish Surgical Society, for participating in the design of the study. We also thank all the participating instructors in the Basic Surgical Skills course at Practicum, Lund. Members : Dan Sevonius and Antoni Zawadzki, Lund/Malmö; Anders Rosemar, Gothemburg; Jonas Leo, Stockholm; Arne Eklund, Västerås/Uppsala; and Yucel Cengiz and Maria Nordström, Sundsvall. Instructors: Dan Sevonius, Antoni Zawadzki, Stefan Acosta, Peter Mangell, Peter Månsson, Karolin Isaksson, Henrik Jutesten, Mikael Ekelund, Per Törnqvist, Jan Holst, Johan Magnusson.