Simulated patient programmes in Europe: Collegiality or separate development?

Abstract

Background: Simulated patients (SPs) are widely used in medical education yet little is known about how individual schools recruit, develop, use, evaluate and maintain SPs. Opportunities for sharing SP development expertise and materials among institutions are not often utilised.

Aims: In order for different SP programmes to learn from each other, there needs to be some basis for establishing meaningful comparisons.

Method: In 2006, the Association of Standardized Patient Educators (ASPE) piloted a survey instrument that would facilitate comparisons of SP educational practices in different institutions. Four European countries at varying stages of SP programme development were selected as representative of the spread of SP experience in Europe (Belgium, Ireland, Scotland and the Netherlands). Key SP contacts were identified in each medical school. Contacts were asked to complete a 49-item questionnaire developed collaboratively between ASPE and the authors. The overall response rate was 86%.

Results: There were considerable differences between countries in terms of their approach to developing SPs and quality assuring their performance. Whilst SP education was regarded as an expensive enterprise, there was little evidence of resource sharing between different centres in the same country.

Conclusions: There is a clear need to facilitate closer collaboration between centres in developing and quality assuring SPs.

Introduction

Four decades following Barrows’ (1968) original description of simulated patients (SPs), most medical schools now work with SPs to support the teaching and evaluation of communication skills as well as a variety of other purposes (Olive et al. 1997). However, despite the fact that SPs are employed extensively throughout the developed world, there is only one descriptive comparative study of their use in different centres (Stillman et al. 1990). Further, there is only one transnational SP educator organisation, the Association for Standardized Patient Educators (ASPE) that facilitates teachers who work with SPs to share ideas or compare best practices, largely among North American institutions. The culture within SP education has, therefore, been essentially one of separate development rather than one of mutuality. Yet the extensive literature on the use of SPs in health care education describe many different approaches to developing and maintaining SPs that could and should be more widely shared and disseminated (Barrows 1993; Lane & Rollnick 2007).

In order for different centres of education to learn from and about each other, there needs to be some basis for establishing meaningful comparisons. In 2006, ASPE set out to develop a survey instrument that would facilitate comparisons of SP educational practices in different institutions as well as inform the organisation about the educational needs of SP educators. Europe was selected as the most appropriate place to pilot the questionnaire, because in 2006, ASPE was seeking to expand its membership from its North American base. The purposes of this article therefore are: (1) to pilot an SP survey instrument, (2) to describe how different European medical schools work with SPs and (3) to elucidate some of the educational and development needs of institutions in relation to working with SPs. This article will present some of the more important findings from the 2006 to 2007 ASPE survey.

Table

Practice points

European medical schools tend to employ amateur actors and volunteer patients as SPs. There is little sharing of expertise, ideas and cases between centres in the same country. There are no consistent approaches to quality assurance in terms of case portrayal and feedback to students for European SPs. There is clear interest in establishing a European SP education organisation to facilitate exchange between different centres.

Methods

Sample

Four European countries at varying stages of SP programme development were selected as representative of the spread of SP experience in Europe. SP-based education has been established in Scotland for 15 years and in the Netherlands for over 25 years, whereas it is a more recent introduction in the Republic of Ireland (9 years) and Belgium (11 years). All of the medical schools in each of the four countries were included in the sample (in Belgium only Flemish speaking schools were included). Key SP contacts were identified in each medical school. Key contacts were defined as persons who had overall responsibility for SP recruitment, training and employment in their respective institutions. Accuracy in identifying key players was enhanced by the fact that there was at least one author from each participating country.

Instrument

A 49-item questionnaire was developed collaboratively between ASPE and the authors. The authors set out to create a comprehensive institutional survey instrument that would examine SP recruitment, selection, training, usage, maintenance, quality assurance and funding at undergraduate and postgraduate levels in different countries. The themes and items used in the questionnaire were developed by seeking expert SP opinion at all stages of the questionnaire development. Drafts were circulated, amended and re-circulated until consensus was reached on the content of a final draft. The final questionnaire draft was subsequently field tested for both comprehensiveness and comprehensibility. (The version of the questionnaire used in this study is available in Appendix 1.) The questionnaire was largely quantitative with some qualitative response options. The survey was delivered to key SP contacts (i.e. the respondents) electronically via an online survey website (SurveyMonkey.com, Portland, Oregon, USA, Author/Owner: R. Finley) in 2006 with follow-up reminders within 2 weeks and 4 weeks of the original mailing. Numerical data were analysed using Statview for Windows, version 5.01.1, SAS Institute Inc., 1998 and SSPS (SPSS for Windows version 11, SPSS Inc., 2007). The limited written data were analysed qualitatively using framework analysis (Carter et al. 1999). This article will present the quantitative survey findings only as the qualitative findings are mostly contextual descriptions and qualifiers of quantitative responses.

Results

There were 22 medical schools between the four countries at the time of the survey, five in Ireland, five in Scotland, four in the Flemish speaking region of Belgium and eight in the Netherlands. Completed questionnaires were received from 19 out of the 22 medical schools approached, a response rate of 86%. Whilst the response rate from Ireland (5/5), Belgium (4/4) and the Netherlands were high (7/8), the response from the Scottish institutions was lower at 3/5. Thus, it can be assumed that the survey findings are more representative of Dutch, Irish and Belgian institutions than the findings that relate to the Scottish schools. The data presented below represents aggregated data from all four countries unless otherwise specified.

SP demographics

Amongst surveyed institutions, females comprised a majority of their SP pool (58%) and 70.5% of their SPs are over 40 years of age. There were a great variety of different SP backgrounds amongst the SPs used in different centres. The majority of SPs in all four countries fell into three categories: lay persons with minimal amateur actor experience (28%), actors with professional experience (18%) and volunteer patients (23%). The remaining significant categories included teaching staff (11%) and medical students (10%). Whilst there were no major differences between the different countries’ use of actors as SPs, there were some interesting patterns. For example, programmes in Belgium and Ireland are more likely to work with unpaid volunteer patients (8/9 schools) than programmes in Scotland and the Netherlands (4/10 schools).

SP programmes were categorised as being school or discipline based. School-based programmes were free standing, independently funded and often associated with a skills lab. On the other hand, discipline-based programmes (of which there could be several in one school) were operated and funded within and between disciplines, e.g. general practice. We found that whilst discipline and school-based programmes could exist in the same centre, the newest country to the concept of working with SPs (Ireland) used a discipline- (or department-) based configuration (4/5 schools), whereas the remaining three countries with larger medical schools and more established programmes tended to use school-based configurations (10/14 schools).

SP recruitment and training and quality assurance

SPs were usually recruited by word of mouth through medical school and personal social networks in 15/19 schools. Interestingly, most schools (15/19) also used the SPs’ own social networks to recruit more SPs. More standard (and perhaps more expensive) forms of advertising, e.g. clinic/institutional noticeboards (9/19) and newspaper adverts (6/19), were used less often.

The quality assurance of SP training and performance was quite variable. For example, only 7/19 institutions used a standardised approach to developing and training SPs (by standardised approach we mean that there was a uniformly applied school protocol for developing SPs). This means that a majority of institutions (13/19) in this sample trained their SPs in a non-uniform manner. There were also differences in actual training methods between the programmes. For example, in Scotland and the Netherlands, schools were more likely to use video recording and multi-source feedback of SP performance during SP and case development (6/10 schools) when compared with programmes in Belgium and Ireland (2/9).

Whilst a majority (18/19) of institutions evaluated their SPs’ performances, this was usually done by the tutors (14/19) with fewer institutions using student evaluations (9/19) or SP peer evaluations (4/19). Only 2/19 institutions used a standardised rating form to ensure consistency of SP ratings.

SP case development

All of the respondents created their own cases for SP teaching. Cases were developed from real patient presentations brought by clinician teachers or represented modifications of patients’ own stories where volunteer patients were being used as SPs. Despite the fact that so many cases are being developed in different institutions, only 6/19 institutions share cases with other schools. The best case-sharing performance was in Scotland where all of the medical schools share cases through an inter-institutional skills training network.

How SPs are used

SPs are used for teaching and formatively assessing communication skills in all of the participating institutions and for physical skills training in 14/19 schools. Of the schools that use SPs to teach physical examination skills, 13/19 schools said that they integrate physical examination and communication skills within their SP roles and the subsequent student assessments. SPs are used to provide medical students with authentic learning experiences in all of the participating schools and are encouraged to provide direct feedback to students on their performance in 15/19 schools. (The term ‘direct’ feedback means feedback that is given to students directly by the SP themselves.) Interestingly, in Ireland SPs provide direct feedback to students in only 1/5 schools.

Infrastructure and funding

Some institutions were not in a position to base their SP teaching in a dedicated skills training laboratory. For example, only 2/5 of schools in Ireland indicated that they had access to a skills lab for SP teaching, whereas all of the institutions in the Netherlands and Scotland based their SP teaching in dedicated skills training facilities.

Of the institutions who responded, two-thirds had dedicated budgets to pay their SPs; however, only one-third of the schools had additional budgetary provision for the operational expenses of running SP programmes (e.g., travel costs, subsistence, equipment, etc.).

Needs assessment and inter-school collaboration

A large majority of schools (18/19) indicated that they would like to both share and acquire new ideas and approaches regarding the use of SPs in medical education through interaction with other schools. Seventeen schools indicated they would welcome the establishment of a common forum that would make it possible for them to meet with other SP trainers, teachers and users. Sixteen schools replied that they would like to exchange best practices in SP training and the maintenance of SP banks. Respondents said that they would welcome the establishment of an SP organisation in Europe to address common learning needs such as SP training methods (17/19), the opportunity to form research collaborations (13/19) and gaining access to alternative performance rating scales (13/19).

Discussion

This study set out to survey SP use in four European countries at different stages of SP programme development. Whilst the findings cannot be regarded as representative of all schools in the four countries, given the incomplete response rate in Scotland and the limitation of the survey to the Flemish speaking institutions of Belgium, they do demonstrate important similarities and differences between SP employment in the four countries. The survey has also revealed important areas for potential collaboration and development between institutions in different parts of Europe. There was almost unanimous agreement that a formal collaborative structure to support communication and exchange between institutions should be established. Thus, one of the primary goals that follow from this survey is to establish a European and global forum for the sharing of information and best practice in SP education.

The key similarities between different centres included their approaches to recruiting SPs, the demography of SPs used and the employment of SPs for teaching communication skills. Centres diverged more in their use of SPs for physical examination skills and the integration of communication skills with physical examination skills in the design of teaching case scenarios. Given the importance of ensuring authenticity in medical education simulations (Regehr & Norman 1996) and the emergence of widely accepted integrated frameworks for teaching communication skills (Kneebone et al. 2002; Kurtz et al. 2003), the use of integrated communication and physical examination skills cases is now the accepted best practice (Kurtz et al. 2003). Thus, the separation of communication skills training from physical examination skills teaching that was apparent in some of the surveyed schools represents a practice which could be ameliorated through better information exchange about evidence-based practice in SP education.

Case development and SP training are generally done within each institution in Europe. Yet, case development and SP training are labour intensive and expensive endeavours (King et al. 1994; Colliver & Swartz 1997). Only in Scotland, and to a lesser extent in the Netherlands, was there a demonstrable sharing of expertise and resources between different centres. The value of sharing cases, standardised approaches to case design and SP training are likely to be manifold in terms of sharing best practice as well as reducing production costs for participants. This is an important consideration for well-established, independently funded programmes as well as for newer programmes working to gain a foothold in medical education institutions. Sharing cases within countries is always likely to be easier than sharing cases between separate countries given the linguistic and cultural differences that apply. However, the value of gaining access to new case ideas, designs and content is likely to outweigh the translational and contextual challenges.

The quality assurance of SP training and performance is likely to be highly variable between different countries given the lack of standardised approaches to case development, SP training and SP performance evaluation evident in many of the participating institutions. Countries with less established programmes may be understandably less rigorous in their approach to case development and SP quality assurance. However, variance in case development and SP training are likely to lead to inconsistencies in SP case portrayal. Consistency of performance is a vital factor in ensuring the reliability of SPs in the summative assessment of medical students (Gorter et al. 2000).

Newer forms of evaluating SP performance were not as prevalent in countries with less established programmes, e.g. student SP evaluation. Wider dissemination of standard SP rating instruments, such as the MASP (Wind et al. 2004), would help to ensure a broader multi-source approach to SP evaluation. Similarly, SP programmes in Belgium and Ireland were less likely to use SPs for student feedback despite good evidence to show the considerable educational benefits of SP-provided student feedback (Vessey & Huss 2002).

Whilst SP programmes are expensive to run (Colliver & Swartz 1997), a lack of a dedicated budget for the operational costs of an SP programme can only serve to undermine the autonomy of the programme to develop as it should. It is hard to say whether establishing larger, free-standing SP programmes within institutions is preferable to facilitating smaller programmes to develop within departments or disciplines. However, by combining smaller programmes into one larger programme there may be economies of scale as well as a wider sharing of interdisciplinary expertise.

The European SP survey covered only four countries in its pilot phase. Clearly, a wider survey is required as the differences between the four participant countries indicate the likelihood of even greater difference in a more extensive sample. However, the lessons learnt are likely to be of relevance to all European SP programme directors and their associated schools despite differences in language, culture and health system.

The data are presented in terms of descriptive patterns rather than inferential certainties. A larger survey will allow sufficient numbers to detect statistically significant differences in approaches to SP training and teaching. However, it is clear that there are considerable advantages in trying to establish a more collegial approach to case development, SP training and SP evaluation between institutions in the same country as well as between different European countries. In the Netherlands, for example, the circulation of this questionnaire led directly to the formal launch of a Dutch SP organisation that now meets three times a year with active exchange on wide-ranging SP teaching issues such as SP feedback training and SP payments. Furthermore, the strongly expressed wish of most respondents to establish an SP education networking organisation in Europe led to the establishment of a formal 5-year agreement between ASPE and Association for Medical Education in Europe (AMEE) to dedicate a full pre-conference SP teachers day, led by ASPE at the annual AMEE conference. These exciting initiatives and collaborations have arisen from a limited pilot survey and they provide hope that even closer and better-structured international collaboration and information exchange in SP education will occur in the future.

Acknowledgements

We would particularly like to thank the participating SP key contacts from medical schools in Belgium, Ireland, the Netherlands and Scotland. This research was supported by a research grant from ASPE.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

Peter Cantillon

PETER CANTILLON is a professor of primary care. His research interests include the influence of teachers on students’ clinical education experiences and the relationship between teachers’ pedagogical knowledge/beliefs and their teaching behaviour.

Brian Stewart

BRIAN STEWART is a recently qualified doctor, currently practicing in hospital medicine. His interests include medical education. He is currently practicing hospital medicine and is doing a postgraduate degree in medical education.

Karolien Haeck

KAROLIEN HAECK is a general practitioner and responsible for the simulated patients program at Faculty of Medicine and Health Sciences, Ghent University, Belgium.

James Bills

JAMES BILLS is an assistant professor of medical education and administration whose research interests include defining and measuring medical education programmatic outcomes as well as diversity issues.

Jean Ker

JEAN KER is the Director of the Institute of Health Skills and Education and is the national lead for the clinical skills network in Scotland. She is interested in the use of simulated patients in re-creating authentic clinical environments for learning non-technical skills such as prioritisation, decision making and time management.

Jan-Joost Rethans

JAN-JOOST RETHANS is a general practitioner. He is responsible for the simulated and standardised patient programme, the CORE programme and the skills programme of the master-phase at Maastricht Medical School. He is a world-renowned expert in simulated patient research and development.