The feasibility and acceptability of integrating regular centralised laboratory-based skills training into a surgical training programme

Abstract

Background: Traditional laboratory-based skills training provides mass training that does not match clinical experience and is not tailored to individual needs. This compromises the transfer and retention of skills into clinical practice.

Aim: To demonstrate the feasibility of integrating a centralised programme of laboratory-based surgical skills training into a higher surgical training programme and to evaluate its effectiveness and acceptability to trainees.

Methods: Laboratory-based skills training was provided at a central site, delivered by consultants and tailored to the trainees’ level of clinical experience. Each trainee was expected to attend one session a month for 11 months a year. Evaluation was conducted through attendance records, structured evaluations by participants, independent qualitative questionnaires and web interviews.

Results: Forty-two specialist surgical trainees in the North West London higher surgical training programme participated in laboratory-based skills sessions delivered by 19 consultants over a period of two years. The average attendance was 70.5% for trainees and 100% for trainers. All sessions were rated by trainees as well-organised and useful with an average score of more than 4 out of 5. Trainees felt that the Skills Programme can complement surgical training by allowing practice under expert supervision in a safe environment.

Conclusions: Centralising laboratory-based skills training and integrating it into a clinical programme is feasible and acceptable and represents a paradigm shift in surgical training. Involvement of trainees in designing the content is valuable.

Introduction

Learning in the operating theatre (OT) alone can no longer meet the increasing demands of surgical training and competency development as a result of cumulative changes in the training and work regulations of surgeons (Kneebone et al. 2004; Temple 2010). Laboratory-based skills programmes have been developed to increase training opportunities and to enhance the transfer of skills from the laboratory to clinical practice. Several studies have shown significant improvement in operative performance in the OT when trainees had received prior laboratory training (Poenaru et al. 1998; Reznick & MacRae 2006; Roberts et al. 2006).

After two decades of development of laboratory-based skills training, the strategy of training in those programmes needs to be reviewed for several reasons. First, the recent expansion in laparoscopic surgery has increased the exposure of almost all trainees to laparoscopic techniques and, therefore, short familiarisation courses are no longer needed. The objectives of laboratory training have changed from the introduction of new techniques to competency-based curriculum in order to achieve proficiency levels. Second, the acquisition of surgical skills should include skills in open surgery as well as laparoscopic techniques. It is no longer acceptable to rely on ‘ad hoc’ skill acquisition in open surgery during ‘on-the-job’ training. Third, traditional laboratory-based skill courses provide mass training concentrated in a few hours or days. The published literature suggests that this is inferior to skills training distributed at regular intervals during a clinical training programme (Mackay et al. 2002; Grantcharov et al. 2004; Sedlack & Kolars 2004; Moulton et al. 2006; Larsen et al. 2009). Fourth, the lack of coordination between laboratory and clinical training compromises skills transfer and retention in clinical practice. Last, traditional laboratory-based skill training is not tailored to individual's needs, and, therefore, does not compensate for differences in innate abilities and enhance potential talents. Those challenges to traditional laboratory-based skills training provide the rationale for integrating such training into clinical programmes (Dhariwal et al. 2007; Singh et al. 2008). Centralising laboratory-based training for all trainees in a programme onto a single site makes a more systematic and standardised approach possible, provides a larger pool of experts to draw on as trainers and offers economies of scale.

This article introduces the Skills Programme and presents the results of a study evaluating its feasibility and acceptability through an in-depth exploration of the views of participants on the usefulness of the Programme and how it could be improved.

Table

Practice points

Centralised distributed surgical training is feasible and can be effectively integrated within a higher specialist registrar programme. Surgical trainees who participated in the LD Skills Programme supported that simulation-based training is essential as long as the curriculum and training models are level-appropriate.

Method

Participants

Forty-two Specialist Registrars from the North West London Deanery attended the Skills Programme. The numbers across year groups were as follows: Year 1: n = 13; Year 2: n = 12; Year 3: n = 17.

The educational intervention

The London Integrated General Surgical Skills Programme was developed to provide systematic structured hands-on experience of a number of index procedures for specialist registrars. The Programme was designed to match the trainees’ training stage in clinical practice, tailored to each individual's needs and abilities.

The programme was established with the coordinated efforts of clinicians and academics with a special interest in surgical education, including a programme director and an oversight board consisting of senior academic surgeons, representatives from the School of Surgery and educationalists. For the day-to-day running of the programme, a full-time technician and a part-time programme manager were employed.

All Specialist Registrars in their first three years of the North West London Deanery Higher Surgical Training Programme were required to attend 11 monthly skills training sessions per year. The session timetable for the entire year was provided for each trainee at the outset. In addition, each trainee received a personal reminder a week before their scheduled training session. The training was consultant-led, and involved hands-on experience with synthetic and non-live animal models on the bench and laparoscopic box-trainers.

A set of technical skills and procedures in open and laparoscopic surgeries were covered in each year (Table 1). Technical skills were categorised to match clinical surgery that the trainee was involved with, in order to maximise the transfer of skills between laboratory training and clinical practice.

Table 1.  Procedures covered by the Programme

Year 1

Repair of inguinal hernia with mesh (six–eight models in two sessions)

Primary varicose vein surgery (six–eight models in two sessions)

Knot tying in difficult angles and narrow fields (one session)

Small bowel anastomosis (six–eight models in two sessions)

Ergonomics of the setup for laparoscopic surgery (one session)

Diagnostic laparoscopy (one session)

Basic laparoscopic dissection techniques (one session)

Year 2

Large bowel anastomosis [with unmatched ends] (six models in two2 sessions)

Gastro-jejunostomy and cholecysto-jejunostomy (six models in two sessions)

Basic vascular anastomosis (six models in two2 sessions)

Laparoscopic cholecystectomy (eight–ten models in four sessions)

Year 3

Oesophageal anastomosis (six–eight models in three sessions)

Anal anastomosis (six–eight models in three sessions)

Laparoscopic suturing and intra and extra-corporeal knot tying (three sessions)

Advanced laparoscopic dissection techniques (one sessions)

Each session was attended by a maximum of 12 trainees. The trainee to trainer ratio was one consultant for a maximum number of 12 trainees. The relevant trusts were informed about each trainee's participation well in advance so as to minimise disruption to clinical services. The training sessions were delivered in the mornings from 09.00 to 12.30. Where possible, the day of the week in which the training session was delivered advanced by one day each month, to ensure that a trainee's absence from their training hospital was uniformly spread throughout the week over the course of the training year. The laboratory ran a total of five sessions a week, during 33 weeks of the year. Each session was supervised by a consultant who also carried out an Objective Structured Assessment of Technical Skills (OSATS; Martin et al. 1997; Reznick et al. 1997) assessment on each trainee at the end of the session. Each hospital trainer received a letter after each session, indicating the skills gained and highlighting areas where the trainee needed more practice.

Methods of evaluation

Feasibility was assessed by the number and frequency of sessions delivered, the number of consultants who participated and their attendance and the proportion of monthly sessions trainees were able to attend. The evaluation of trainees was performed by the trainer at the end of each training session using OSATS.

Acceptability to trainees was evaluated quantitatively through a structured questionnaire completed by the trainees at the end of each session. Questions related to the organisation of the session, the effectiveness of the teaching method and the tutors, the materials/models, and session duration and the level of difficulty. Responses were on a 5-point Likert scale. A further qualitative evaluation, at the end of the first and second years of the Programme, was aimed at eliciting trainees’ opinions and attitudes. Open-ended questions were asked through a questionnaire and web-based interviews, exploring issues relating to the perceived advantages and disadvantages of laboratory-based skills training; views on the best teaching methods of skill acquisition; factors affecting attendance; trainees’ comments on the curriculum; suggestions for changes to the format or content of the course and possible impact of skills training on trainees’ clinical practice. The development of the questionnaire was based on a focus group discussion. The administration and analysis of the questionnaire were conducted by an independent research company in order to secure impartiality and to reduce interpretation biases. All participants were informed of the anonymity and confidentiality of their responses.

Results

Feasibility

The programme succeeded in delivering 22 monthly sessions for each trainee over a two-year period. Five sessions a week were delivered by 19 consultants to 42 specialist registrars, during a total of 33 weeks per year. Consultants attended 100% of the sessions they were booked to supervise. Trainees’ average attendance was 70.5% overall, 75% in 2006/7 and 66% in 2007/8 (Figure 1). A number of different reasons for absence were cited with the most common reason being on leave or being unable to find a person to cover for them. Overall, we found that clinical responsibilities, work pressure, staff shortages, Trusts’ unwillingness to cooperate and the unpredictability of the clinical environment made 100% attendance difficult to achieve even with plenty of notice.

Figure 1. Trainee attendance record across year group for 2006–2007 and 2007–2008.

Acceptability

Quantitative data

The structured evaluation questionnaire was completed at the end of every session (100% response rate). Average scores for each item ranged between 4 and 5, indicating a high degree of satisfaction with each session (Figure 2). No significant differences appeared between the three training levels, or over the two years of the Programme.

Figure 2. Trainee session evaluations across the three SpR year groups for 2006–2007.

Qualitative data

One year into the Programme (Phase 1), 24/42 (57%) trainees filled in the qualitative questionnaire either after one of the sessions or by response to an e-mail. A second questionnaire was forwarded to the trainees toward the end of the second year (Phase 2). This included the same questions and format as the first with the addition of one question on the clinical impact of the Programme. The response rate to the follow-up questionnaire or to an anonymous web-based interview using the same questions was 17/42 (40%). The following themes emerged during data analysis.

General attitude to Programme

Participants felt that the Programme could fill an important gap in training. The themes that emerged regarding the advantages of the Programme were: laboratory skills training provided a stress free environment to practise, good teacher-student ratios, opportunities to ask questions, accelerates the learning curve and provides opportunities to share experiences with peers. In some cases, trainees were more specific and commented on individual sessions as well. For example, one trainee stated, ‘Useful in laparoscopic experience to make up for the lack of clinical opportunities’’ Another trainee mentioned, ‘Generally, the teaching is of a very high standard. You are working with very respected people; in some cases the people who invented the actual procedures!’.

The senior trainees were more inclined to identify some minor disadvantages relating to the level of difficulty and the teaching quality of some of the sessions. ‘I do value being able to practise, and it is good to learn a technique away from a patient – but then you have to actually apply it on a patient’. A minority of senior trainees expressed reservations about some of the models; specifically, the ability of some models to mimic reality and the need to practise skills in a more realistic setting. A major concern among trainees seemed to be the possibility of missing practice in the OT while attending the skills training.

Attendance

During Phase 1 trainees were unsure whether the Programme was compulsory, but by Phase 2 there seemed to be a clear view that attendance was compulsory and would be taken into consideration in annual reviews. Some trainees, especially senior SpRs, were frustrated because they were obliged to attend. One trainee stated, ‘I was concerned; it is awfully disruptive, it takes ages for me to travel back for the afternoon clinics’. The two issues that emerged as main factors for non-attendance related to the lack of support from their hospitals and consultants and their own reluctance to miss training in the OT. Some trainees suggested that the Programme should be more flexible so that they do not miss OT opportunities and that they are able to use their judgement and come to the sessions that will benefit them the most. However, another view was expressed as, ‘I think you have to make it compulsory, because between the pressure from work and pressure from the consultants, people would end up not coming’.

Attitudes towards the learning method

Supervised practice in theatre was the preferred learning method, followed by unsupervised practice in theatre. Laboratory-based skills training as delivered by the Programme came next. Unsupervised practice on their own time, videos and CD ROMS, reading books and more didactic approaches were considered less effective methods of learning compared to hands-on training. Table 2 shows average rankings for each method of learning (lower scores indicate higher level of preference).

Table 2.  Participants’ rankings of preferred learning methods

Teaching method	Average Rank
Supervised practice in theatre	1.0
Unsupervised practice in theatre	3.1
Structured training in laboratory setting	3.6
Watching consultants	3.7
Unsupervised practice in own time	5.1
Using videos/CD ROMs, etc	5.9
Lectures	6.3
Reading books	6.5

Attitudes towards the Programme content and structure

The content of the Programme was deemed useful and appropriate by most, but some trainees were reluctant to practise skills; they felt they would not be using in their specialisation and preferred to receive training that was suited to their chosen specialty. Senior trainees thought the content was more useful for junior trainees. Other suggestions related to the tailoring and timing of the training. One trainee noted, ‘There needs to be a better match between the training in the laboratory and the training opportunities in theatre’. Another trainee stated, ‘Full-day sessions are preferable to reduce the impact on clinical duties’. However, trainees were, ‘Very grateful to be involved’.

Reflections on clinical impact of the training

Mixed views were expressed with most of the respondents applauding the idea, for example, ‘The main impact is in the skills that are related to my current post and procedures I am currently practising’ and ‘Easy to translate to clinical practice. The course has been excellent’. A small number of trainees thought that the clinical impact of attending the skills laboratory was minimal unless the skills learned during the sessions were subsequently reinforced by clinical exposure.

Suggestions for improvement

A number of suggestions were made on how to improve the content of the Programme, encourage attendance, and improve assessment. Our response to these is considered in the Section ‘Discussion’.

Assessment

OSATS

The consultant-trainers assessed trainees’ generic surgical skills after each session using a global rating scale. The majority of trainees received scores between 20 and 28 with the maximum score being 35. The range of the data was limited indicating reduced variability; therefore, further statistical analyses with these data were not performed.

Discussion

In this article, we have set out to present and evaluate the feasibility and perceived usefulness of a centralised distributed skills programme embedded within clinical training. The pilot phase of the programme was successfully completed and the participants felt that laboratory skills training could complement current training opportunities. The trainees highlighted that the Skills Programme allows for practice in a stress-free environment with targeted consultant-led teaching. More senior trainees were contented with the programme but they were more critical about the realism of the simulators.

The evaluation study provided good evidence for the feasibility and acceptability of this training approach. It also provided good ideas for improving the Programme, which has now been extended to encompass approximately 190 of London's surgical trainees in the first four years of their higher specialty training. In response to suggestions made during the evaluation, trainees are now given more flexibility to change sessions, if their circumstances do not allow them to attend. All trainees are now told they have to achieve at least 70% attendance and this information is included in their annual reviews. In order to encourage attendance, the Programme content is constantly revised and advanced procedures and more realistic models are added. Plans are in progress to train senior trainees on specialised procedures using cadaveric models. In addition, the evaluation revealed a need for more opportunities for practice and augmentation of skills that have a prolonged learning curve, and where there is limited opportunity to develop within clinical practice, such as advanced laparoscopic techniques (Larsen et al. 2009). For this reason, more sessions were added to the Programme content, such as laparoscopic appendicectomy, fundoplication and colectomy while laparoscopic procedures are now introduced at an early stage for junior trainees. Where simulation units are available locally, trainees are encouraged to practise their learned surgical skills within a simulated clinical environment.

The effectiveness of the training was more difficult to establish by objective means. The consultant assessment of trainees was carried out using the OSATS forms. We found that this scoring system was not discriminative and reduced the value of these scores in assessment. This was attributed to the structure of the training and the assessment methodology. First, with respect to the structure of the training, most of the consultant's time was spent demonstrating and guiding trainees with little time left for assessment. Second, a scale assessing generic skills may have not been well-suited to laboratory-based training. Methods need to reflect the support required during training as well. Such approach has been successfully implemented in the English National Training Programme in Laparoscopic Colorectal Surgery (Miskovic et al. 2011). Third, some trainers believed that assessment puts additional stress on trainees and creates a tense learning environment. For this reason, a number of trainers may have been too lenient and gave favourable scores to junior trainees. It is noteworthy that the majority of the trainers believed that a reliable and valid assessment methodology would provide useful information to trainees and their supervisors, regarding their progress. Fourth, it is difficult to ensure inter-assessor reliability with large numbers of assessors.

Current efforts aim at developing and employing an assessment methodology that is feasible, reliable, and valid. Many benefits would emerge from a reliable indicator of trainee progress, which could be fed back to supervisors and trainees. In the long term, this information could be used to audit the quality of the Programme and its clinical impact. It would also be possible to define a set of surgical competencies with exit criteria based on competence and educational objectives, rather than simply a ‘time-served’ criterion. This requires a data bank of the level of competencies expected from trainees at different stages of training. This would allow identification of poor performers at an early stage with provision of further targeted training. Further work is required to provide robust means of assessing the effectiveness of the Programme through the assessment of trainees.

There is also a need to address other aspects of surgery, which relate to psychological qualities and the acquisition of non-clinical skills needed for competent performance in the OT. Decision making and stress management in the OT can be practised in a simulated environment. It may be possible to create teaching opportunities and training in non-clinical skills within the scope of this Programme. These options are currently under consideration.

Conclusion

A centralised integrated laboratory-based skills training Programme proved to be feasible and acceptable to trainees. Trainees who attended the Programme felt that in light of the shrinking training opportunities for surgical training, centralised simulation training programmes are necessary and should become compulsory elements of the training path.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

This work was funded by the London Deanery.