Value and face validity of objective structured assessment of technical skills (OSATS) for work based assessment of surgical skills in obstetrics and gynaecology

Abstract

Background: Changes in UK Junior Doctor working patterns, reduced training hours and increased patient expectations have resulted in less exposure of Juniors to surgical procedures. Validated methods of assessment are therefore required to assure the surgical competency of future Consultants. Objective Structured Assessment of Technical Skills (OSATS) forms are one possible tool.

Aims: To investigate trainee and trainer perceptions of the value and validity of OSATS in the operating theatre.

Method: OSATS forms and feedback questionnaires.

Results: 38 trainees and 16 trainers provided feedback on the OSATS forms used during 60 procedures, returning 119 questionnaires in total. Value : 85% of trainee and 76% of trainer responses agreed/strongly agreed that OSATS would improve trainees’ surgical skills. Face validity: 80% of trainee and 76% of trainer responses agreed/strongly agreed with OSATS ability to assess surgical skills. Application: 76% of trainee and trainer responses agreed/strongly agreed that OSATS should become part of the annual assessment process for trainees.

Conclusions: Trainees and trainers perceive OSATS to be a valuable and valid tool for the objective assessment of surgical skills in the work place. More research is required to assess their criterion-related, content, construct and predictive validity as well as their reliability in the workplace.

Additional information

Notes on contributors

J. F. Bodle

J. F. BODLE MD MRCOG, Lecturer in Obstetrics and Gynaecology with extensive experience in co-ordination and delivery of undergraduate and post graduate education. Research interests include minimal access surgery, surgical skills training and assessment and medical education.

S. J. Kaufmann

S. J. KAUFMANN FRCOG, Consultant Gynaecologist. Obstetrics and Gynaecology Postgraduate Training Programme Director Yorkshire Deanery, Royal College of Obstetricians and Gynaecologists Postgraduate Tutor.

D. Bisson

D. BISSON FRCOG, Consultant Obstetrician and Gynaecologist. Obstetrics and Gynaecology Postgraduate Training Programme Director and Chair of Southwest Deanery Specialist Training Committee, Royal College of Obstetricians and Gynaecologists Postgraduate Tutor.

B. Nathanson

B. NATHANSON MPhil, Senior Lecturer in Physiology, and Education Development at the Royal College of Surgeons, England. Workforce Development Manager and New Deal for Junior Doctors, Hillingdon Hospital. Publications on the expectations and experiences of overseas doctors training in the UK. Head of Education Development RCOG.

D. M. Binney

D. M. BINNEY MMedSci, Senior Lecturer in Research Methods Sheffield Hallam University. Reviewer for Journal of Sports Science.