Perspectives on simulation-based assessment of operative skill in surgical training

Abstract

Introduction

The perspectives of the wider surgical community toward simulation-based assessment (SBA) in training is a gap in the literature. This study aims to explore the factors associated with the acceptable use of SBA in surgical training, through the perceptions and experiences of a broad range of stakeholder representatives, building on findings from a review of the published literature.

Materials and methods

Ten semi-structured interviews were conducted, using a sequential transformative qualitative methods approach, with representatives from identified key stakeholder groups; executive management, risk management, a practicing surgeon, an anaesthesiologist, a theatre-nursing representative, a representative from simulation industry, a patient, a medical student, a junior surgical trainee, and a senior surgical trainee. Interview transcripts underwent reflexive thematic analysis using an inductive and constructivist framework (NVIVO software, NVIVO 12, QSR International).

Results

Four themes emerged: the ‘need’ for SBA, the concept of a ‘minimum standard’, the ‘optimum design’ of an SBA framework, and ‘fairness’. SBA is a potential solution to challenges in the current training environment. It emerged that it should not replace trainer judgement, but could ensure that trainees meet a minimum operative competency standard. SBA should be used to identify underperforming trainees early in training to provide targeted remediation. The application of SBA in high-stakes settings such as trainee selection, autonomy granting, and end-of training certification has perceived benefits over current assessment methods.

Conclusions

This study builds on findings from prior research to explore factors regarding the acceptable use of simulation as an assessment method in surgical training, including perspectives from a broad range of stakeholder representatives. Findings can inform the development of simulation-based assessment curricula in surgical training.

Keywords:

• Surgery
• assessment
• simulation
• competency-based education
• performance of procedures

Ethical approval

Ethical approval for this study was provided by the ethics committee of the University of Medicine and Health Sciences at the Royal College of Surgeons in Ireland.

Author contributions

C Toale: Data collection, data analysis, draft authorship.

M Morris: Study design/methodology, draft review.

D O Kavanagh: Study design/methodology, recruitment, draft review.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Glossary

Simulation-Based Assessment: Assessment of and/or for learning in a simulated, and not workplace-based, environment.

Additional information

Funding

This work was supported by the Hermitage Medical Clinic/Royal College of Surgeons in Ireland Strategic Academic Recruitment (StAR MD) program.

Notes on contributors

Conor Toale

Conor Toale, MSc, MRCSI, is a specialist trainee in General Surgery conducting a post-graduate doctoral degree with the Royal College of Surgeons in Ireland, focussing on the use of simulation-technology in the assessment of operative competence.

Marie Morris

Marie Morris, MSc, PGDipHEd, PhD, is a Senior Lecturer in Post-graduate Surgical Education at the Royal College of Surgeons in Ireland, and programme lead for RCSI’s Master’s in Surgical Science and Practice degree.

Dara O. Kavanagh

Dara O. Kavanagh, MCh FRCSI, is a Clinical Lecturer in Surgery and Head of Surgical Research at the Department of Surgical affairs, Royal College of Surgeons in Ireland.