The introduction of a standardised national licensing exam as a driver of change in medical education: A qualitative study from Switzerland

Abstract

Introduction

Only a few studies have described the impacts, strengths and needs for further development of national licensing exams (NLE). To gain such insights regarding the Swiss NLE, which includes a multiple-choice and a standardised clinical skills exam, we explored the perceptions of involved experts and stakeholders.

Methods

We explored participants’ perceptions in four focus group discussions. The interviews were recorded, transcribed verbatim and qualitatively analysed using a thematic analysis approach.

Results

The analysis resulted in five perceived impacts, two strengths and two needs for further developments of the NLE. Perceived impacts were (1) steering students’ learning behaviour, (2) supporting teachers and assessors to align teaching to competencies, (3) elevating the importance of the Swiss Catalogue of Learning Objectives, (4) setting incentives for the further development of curricula, and (5) fostering the collaboration between the faculties of medicine. Perceived strengths were the blend of assessment formats, including their competency-based orientation, and the collaborative development approach. Perceived needs lay in the NLE’s further development to sustain its fit for purpose and in incentives for people involved.

Conclusion

According to our study, this NLE had, and has, notable impacts on medical education in Switzerland. Our insights can be useful for others planning a similar undertaking.

Keywords:

• Assessment
• OSCE
• written
• undergraduate

Introduction

An increasing number of countries are considering the introduction of National Licensing Exams (NLE), which include a clinical skills part (Archer et al. 2016; Swanson and Roberts 2016). A recent example is the United Kingdom, which plans to introduce such an exam including both a knowledge and clinical skills exam in 2022 (Rutter-Locher and Menzies-Wilson 2018).

Practice points

• According to the participants of this study, the Swiss National Licensing Exam (NLE) has notable impacts on medical education in Switzerland and can therefore be seen as a driver of change in medical education.

• Overall, the NLE is perceived as appropriate for its intended purpose and considered as a move towards competency-based assessment.

• To keep this exam fit for purpose, constant further development is necessary.

• This study can inform others planning a similar undertaking.

In Switzerland, a NLE was introduced in 2011. The Federal law on medical professions, which came into force in 2008, deliberately provided for the possibility for individual universities to shape their own profile and the resulting differences in medical programs. However, with a NLE, the legislator wanted to assure that the educational level is comparable between the different medical programs and that all graduates possess a minimum level of competence.

In order to take a closer look at assessments such as NLE, the following criteria for good assessment from a recent international consensus statement might be helpful: (1) validity or coherence, (2) reproducibility or consistency, (3) equivalence, (4) feasibility, (5) educational effect, (6) catalytic effect, and (7) acceptability (Norcini et al. 2018). It is undisputed that high-stakes licensing exams place high demands on validity, reproducibility, equivalence and feasibility. However, various studies have demonstrated that these requirements can be met in national clinical skills exams, including the Swiss NLE (Harik et al. 2006; Lillis et al. 2012; Guttormsen et al. 2013; Archer et al. 2016). It appears to be of particular interest to examine the impact of such an exam, e.g. with regard to the educational effect (criterion 5), which is defined as the assessment motivates those who take it to prepare in a fashion that has educational benefit (Norcini et al. 2018), and the catalytic effect (criterion 6), which is characterised as the assessment provides results and feedback in a fashion that creates, enhances, and supports education (Norcini et al. 2018). There is some literature available on the potential impact of such exams in this regard. For instance, Gilliland and colleagues conducted a study exploring whether curriculum changes had occurred following the introduction of the Step 2 Clinical Skills component of the United States Medical Licensing Examination (USMLE). Based on a survey of medical educators in US medical schools, over one third of respondents reported changes to the ‘objectives, content, and/or emphasis of their curriculum’ (Gilliland et al. 2008), providing some evidence for the catalytic effect of such an exam. In a qualitative interview study with leaders of medical school assessment, Hauer and colleagues demonstrated an impact of the same exam (USMLE Step 2 Clinical Skills) on how medical students prepared for the exam (educational effect) and on the curricula, including exams (Hauer et al. 2006). However, according to Archer and colleagues, who performed a systematic review on the impact of NLEs, the literature has yet to yield a clear picture regarding the impact of NLEs on medical school curricula (Archer et al. 2016). Furthermore, there are almost no studies on the needs for further developments of such national licensing exams.

Taken together, it seems necessary to take a closer look at the impact and needs for further development of such a national licensing exam. We therefore designed a study to explore the Swiss NLE with regard to its impacts, strengths and needs for further development 2 years after its implementation, from the perspective of experts and stakeholders who are in charge of this national exam.

Gaining more insight into these themes should be informative not only for others who wish to introduce such an exam, but also for other persons who collaborate in high-stakes exam development.

Methods

Setting

The Swiss National Licensing Examination has now been successfully implemented nine times. It consists of a written part (MCQ, type A, single best answer and 20% Kprim, overall 300 questions) and a clinical skills part (12 station OSCE, 13 min per station). This NLE is still unique in Europe, where no other country currently uses a nationally standardised clinical skills exam as part of a national exam to date. A more detailed description of the exam can be found in Guttormsen et al. 2013 and Berendonk et al. 2015.

Design

Only a limited number of persons are heavily involved in developing, conducting, and further developing the Swiss NLE. Therefore, qualitative methods like focus group studies are especially suitable as they are able to generate relevant results also in smaller groups and are particularly apt for addressing ‘why questions’ (Barbour 2005).

A constructivist thematic analysis approach was employed in the present study. Thematic analysis (TA) is a pragmatic approach to qualitative analysis, which involves searching for themes across a dataset (Braun and Clarke 2006). While TA draws on some of the techniques of grounded theory (Kennedy and Lingard 2006), it remains theoretically flexible and can be adapted to suit the specific affordances of a given study. Due to the constructivist approach, it was possible to draw on the existing relevant literature (e.g. Norcini et al. 2018 on criteria for good assessment and Hauer et al. 2006 as well as Gilliland et al. 2008) on the impact of NLE to inform the development of our research question. Moreover, this also provided sensitising concepts that primed the data analysis.

Researchers

Since the researchers play an active role in data collection and analysis, it is important to provide information about them. The study group comprised seven researchers. Five of them (SH, CS, PH, GB, CB) are physicians who are highly engaged in medical education. All of them are also engaged in the Swiss NLE as active members of working groups of the Swiss FLE (clinical skills, budget group). A further two are non-physicians, with backgrounds in psychology (SG) and economics (DJ), with SG also being a medical educator who is involved in the FLE (exam board). The first author additionally has vast experience in qualitative research and focus group moderation (SH).

Participants

Involved experts and stakeholders (n = 28) received an invitation to participate by email. Of the invited persons, 25 (89%) volunteered to participate in one of the four focus group interviews (4–9 participants).

In total, four focus group discussions were held: Focus group 1 with the working group ‘Clinical Skills’ (six members) and one further expert in the NLE; Focus group 2 with the working group ‘Multiple Choice’ (six members), two site responsibles and one further expert in the Swiss NLE; Focus group 3 with Deans of Studies (three) and the chair of the exam board; Focus group 4 with three stakeholders (two from the Federal Office for Public Health (FOPH) and one from the board which chairs the board of examiners and reports directly to the Federal Council (MEBEKO)), plus two further experts involved in the development process of the NLE. As some of the working group members are at the same time site responsibles, all five site responsibles were also involved.

Data collection

In interviews of peer groups, the imbalance of power between researcher and subjects is reduced and in-depth discussion is promoted, thus ensuring a relatively safe environment. The moderator encouraged all of the participants to contribute, highlighted the value of differing opinions, and asked the participants to note down their thoughts on a given topic prior to the group discussion, in order to counteract the potential drawback of social pressure from peers or moderators inhibiting the open expression of opinions. Four two-hour focus group sessions were held on different days, moderated by one of the authors (SH), who is an experienced moderator of small groups, including focus groups (Huwendiek et al. 2009, 2017; Huwendiek, Dunker, et al. 2013). Consistency across group interviews was promoted by a questioning route (Krueger and Casey 2000) (Supplemental Appendix 1). The moderator (SH) and assistant moderators (CB, DJ) took notes and one of the assistant moderators (DJ) video-recorded the sessions.

Data analysis

The recordings were transcribed literally, and the transcripts were primarily analysed by three of the authors (DJ, SH, CB). First, they read all of the transcripts while identifying and highlighting preliminary themes, in line with guidelines for thematic analysis (Braun and Clarke 2006). Next, they established themes using an iterative process. In this process, the whole research team discussed coded themes, and the discussions were in turn used to inform the coding process. This continued until the whole research team reached consensus.

Results

The analysis resulted in nine themes which reflect the experts’ and stakeholders’ perceptions with regard to the perceived impacts (5), strengths (2) and needs for further developments (2) of the Swiss NLE. The main results are also visualised in Figures 1 and 2. In Supplemental Table 1, representative citations from the interviews are provided for each theme, with numbers in parentheses indicating the focus group and the participant.

Figure 1. Main strengths, needs and impacts of the Swiss NLE.

Figure 2. More detailed display of the impacts of the Swiss NLE.

Perceived impact

Theme I1: the NLE steers students’ learning behaviour towards practical and communication skills

There was strong agreement among the participants that the Swiss NLE drives the learning behaviour of the students. The introduction of a CS part was reported to have exerted a considerable pre-assessment effect on student learning behaviour. This is in contrast to earlier years, when there was no CS examination and the students prepared themselves for the examinations primarily in a theoretical manner. Now, the participants reported that students were much more engaged in practical and communication skills (e.g. in skills labs, using role plays) when preparing for the CS exam, as the students knew that this would help them for the exam. However, while the impact of the NLE was overall very positively perceived, it was also discussed by the focus group participants that the modification of the exams could also have some negative side effects like encouraging students to act like ‘simulated doctors’.

Theme I2: the NLE supports teachers and assessors in aligning their teaching to clinical competencies

According to the participants, the teaching staff perceives a higher pressure to teach adequately to meet the needs of the NLE, since the clinical lecturers are well aware that the students’ performance on the exam is also related to the quality of their teaching. In this respect, it was felt that the quality of the teaching with regard to skills has been improved. Another reason why the teaching of skills has improved might lie in the fact that the assessors in the NLE directly experience the aspects and topics with which the students can easily cope, and where they still have deficits. This informs their own teaching. However, participants reported that some, especially older examiners, were not comfortable with the OSCE examination format and regretted the lack of individual conversation with the candidates.

Theme I3: the NLE increases the importance and use of the Swiss Catalogue of Learning Objectives (SCLO) by both students and teachers

Since 2001, a national catalogue of learning objectives has existed in Switzerland. While this catalogue played a rather minor role prior to the introduction of this licensing exam, it has become an important instrument to define the content of the exam. In this sense, as the basis for the exam content, the SCLO took on a completely different significance and importance with the introduction of the NLE. Therefore, it has gained more importance among students in order to set priorities in their learning process, and has gained more importance among teachers as it forms the basis of their teaching.

Theme I4: the NLE sets incentives for the faculties of medicine to develop competency-based curricula, including assessments

According to the participants, the affordances and methods of the Swiss NLE are considered whenever a reform takes place at the medical faculties. The assessment methods of the NLE influence the faculties’ assessment methods, e.g. in several medical faculties, OSCEs have been newly introduced or adapted. In written assessment, there has been a notable shift from factual recall MCQ to vignette-enriched MCQ focusing on applied clinical knowledge. New methods of teaching such as skills labs and communication skills training courses have been added to the curriculum. Since the faculties want to achieve a high pass rate of their cohort of candidates, the NLE is perceived as setting standards and benchmarks for the faculties. There was strong agreement that the NLE had, and continues to have, a great influence on the curriculum development, including assessment, at the Swiss medical faculties. However, some participants also expressed that it cannot always be clearly differentiated which reform caused which effect, because other reforms were also occurring at the same time in Switzerland (e.g. further development of the Swiss Catalogue of Learning Objectives from version one to version two, with version two including CanMEDs competencies).

Theme I5: the NLE leads to a closer collaboration between the faculties of medicine

The NLE has led to a closer collaboration between the faculties of medicine in Switzerland, characterised by a motivating atmosphere. A great deal of formal and informal exchange occurs between the faculties, especially within the working groups, which provides the opportunity to see and learn aspects of the other ‘players’ and allows for the building of a ‘community of practice’. The discussions and meetings triggered by the development of the NLE are seen as fruitful and valuable, and in turn have a positive impact on the further development and quality of the NLE. As such, the NLE supports the harmonisation of processes between the faculties. The NLE has brought about a coalescence of the faculties, especially through joint events of all the faculties (e.g. case/question-writing workshops, review board and SP trainer meetings, meetings of the working groups).

Perceived strengths

Theme S1: overall, participants regard the blending of the two assessment formats as an appropriate combination for the NLE, which should generally be retained

Participants regarded the combination of the two formats multiple-choice question exam (MCQ exam) and clinical skills exam (CS exam) as appropriate for the NLE, as this allows for the assessment of applied clinical knowledge as well as practical and communication skills. Overall, the specific design of the individual assessment formats and the combination of the two formats was perceived as a shift towards competency-based assessment. In detail, the multiple-choice questions with patient vignettes were regarded as suitable to test a broad spectrum of applied clinical knowledge, including clinical reasoning, while at the same time being cost-effective.

The CS exam was regarded as a standardised and fair practical exam of high quality based on a blueprint. It tests a good combination of candidate-patient interaction/communication and clinical skills.

Furthermore, participants perceived the exam as well accepted by the students, believing that students judge it as ‘tough but fair’, ‘relevant’ and ‘positive’.

Theme S2: overall, the collaborative development process among the faculties of medicine is seen as a major positive, constructive and enriching factor and perceived as creating something meaningful and sustainable

The work in the national groups with different languages and sub-cultures was seen as an enrichment.

This positive connotation was motivated by the fact that the faculties were able to learn new approaches and exchange ideas during these meetings. It was also important that the experiences and views of all participants were taken into account in the process of finding a sustainable solution. This is not to say that such processes could not at times be lengthy and complex.

The balance of power between centralisation and federalism and between the different institutes/bodies (Faculties/FOPH/Institute for Medical Education Bern) is perceived as good.

The discussions within the working groups are viewed as constructive and there is a constant perceived will to continuously optimise the processes and the exam.

In the establishment of the Swiss NLE, the collaboration between the different stakeholders (FOPH, faculties, working groups) is perceived as fruitful and constructive. On the one hand, national regulations have provided a strong framework and determined clear conditions, while on the other hand it has left sufficient flexibility to develop the exam on an expert level. Working on this unique project is considered meaningful, especially for the working groups, as they feel that they have a real impact on the further development, design and construction of the exam. However, participants reported that there were also some voices within the faculties, which were not only euphoric about the resulting obligation to implement the nationally developed examination contents at the individual faculties as local traditions had to be changed.

Perceived needs for further development

Theme N1: the assessment formats of the NLE should be further developed and complemented to sustain its fit for purpose. However, modifications should be well-reasoned and evidence-based whenever possible

While overall, the employed assessment formats are perceived as adequate, high-quality, standardised and as including assessment of applied knowledge and skills, other assessment formats should be considered in order to improve or complement the exam, e.g. additional assessment formats which are even more suitable for testing clinical reasoning.

The use of tablets/computers in the exams should be considered for future developments, e.g. new electronic-supported answer formats such as ‘long menu’ and inclusion of audio and video sequences. Another benefit of using tablets may lie in improved judgement processes e.g. reducing missing values by using electronic rating checklists.

The introduction of children as standardized patients could be considered in order to increase the authenticity of paediatric assessments. Since communication skills are elusive to measure, a multifaceted approach should be considered to validly assess these skills.

Further, the Swiss Catalogue of Learning Objectives (SCLO) is perceived as an inevitable component to create the exam (blueprints). Therefore, the revision of the SCLO should be kept up to date in order to reflect the competencies deemed to be necessary to function properly as physician.

Furthermore, participants emphasized that changes should be well-reasoned. Too many and too considerable changes in the exam process over a short period of time should be avoided as they reduce the feasibility and thus the acceptance of the exam by the persons involved.

Finally, modifications of the Swiss NLE should be based on scientific evidence, whenever possible and it would be desirable to intensify research regarding the Swiss NLE.

Theme N2: there is a need for better incentives for people involved in the NLE

Drafting and designing high-quality MCQ or CS stations requires deliberate effort from skilled clinician-authors, and the development of such skills requires time and engagement. However, an individual’s commitment to education and the NLE does not help his/her personal career. This can lead to problems in recruiting a sufficient number of motivated and competent people. Moreover, if clinicians engage only once or selectively in the development of the NLE, this is not optimal for the quality of the exam. The universities still recognise biomedical research more strongly than education. To remedy this, incentive structures like a tenure track in medical education or similar should be established.

Discussion

We explored experts’ and stakeholders’ perceptions of the impacts, strengths and needs for further development of the Swiss NLE. The analysis of four focus group interviews resulted in nine themes, including five perceived impacts, two strengths and two needs for further developments of the NLE.

One of the impacts perceived by the participants referred to the influences of the NLE in steering candidates’ learning behaviour towards a more practical preparation (Theme I1), which can be seen as a direct educational effect of the NLE. This resonates with findings on the effect of exams on student learning, which is becoming increasingly better understood (Cilliers et al. 2012; Huwendiek, Cilliers, et al. 2013). Moreover, it is also in line with a qualitative study by Hauer and colleagues, which demonstrated an impact of the USMLE Step 2 Clinical Skills Exam both on medical students’ preparation for the exam and on the curricula (Hauer et al. 2006).

Additionally, our findings also suggest effects of the NLE on the curricula of the involved medical faculties (Theme I4), on NLE assessors and teaching staff (Theme I2) and on the perceived importance of the Swiss Catalogue of Learning Objectives (SCLO) (Theme I4), which all can be interpreted as catalytic effects of the NLE. Gilliland and colleagues conducted a national survey which revealed that over a third of medical schools in the USA were implementing changes to the objectives, content, and/or emphasis of their curriculum, at least partially in response to the installation of Step 2 CS (Gilliland et al. 2008).

A further central finding was the impact of the collaborative development process on cross-institutional collaboration of the medical faculties in Switzerland (Theme I5). The collaborative development approach and the primary development of the exam was perceived as an exemplary process of how to create something meaningful and sustainable, and in that sense also seen as a strength (Strength S2). The strong move towards increased cooperation between the different faculties can be explained by the fact that the exam is centrally organised but locally administrated and all faculties are strongly involved within the working groups and group of site responsibles (Berendonk et al. 2015; Schurter et al. 2016). In this way, the organisational structure, with the involvement of all faculties and common goals, seems to have supported the development of a community of practice in its central sense, the idea of learning as a participatory social process (Buckley et al. 2019). Both aspects of the community of practice theory, the earlier idea of increasing membership of the community through community members’ actions, behaviours and language, and the more recent definition with a greater emphasis on ‘deliberate group cultivation’, might have contributed to the positive outcome (Buckley et al. 2019).

Taken together, the Swiss NLE was found to have relevant impacts on the medical faculties and on the collaboration between the faculties. This makes the NLE a major driver of change in Switzerland with regard to medical education and collaboration within and between the medical faculties on medical education issues. To our knowledge, there are no other studies addressing this latter issue with respect to a national licensing exam.

Participants perceived the combination of the assessment formats used (Theme S1) and the collaborative development approach (Theme S2) as strengths of the NLE. The formats used in the Swiss NLE are widespread, and are also employed in other national exams such as the United States Medical Licensing Examination (USMLE 2013) or the licensing exam in Canada (Reznick et al. 1992). Therefore, the positive response to these formats is fairly unsurprising. Swanson and Roberts (2016) expressed this aspect as follows: ‘A blend of assessments methods remains desirable in order to increase both the breadths of skills assessed and the efficiency with which content is sampled’.

A recent publication indicated that the Swiss NLE fulfils four of the seven criteria for good assessment (Guttormsen et al. 2013; Norcini et al. 2013): ‘validity or coherence’, ‘reproducibility or consistency’ and ‘equivalence’. With this focus group study, we were able to show that the criteria for good assessment regarding ‘educational effect’ (Theme I1) and ‘catalytic effect’ (Themes I2–I4) of the Swiss NLE also seem to be fulfilled from the stakeholders’ and experts’ perspective. Furthermore, the perceived strengths of this NLE (Themes S1–S2) indicate that the majority of stakeholders and experts have a high acceptance of the NLE, which is also a criterion for good assessment (acceptability).

Perceived main needs were seen in the further development of the NLE to sustain its fit for purpose (Theme N1), in well-reasoned and evidence-based modifications, and in improving the incentive structure for clinicians who actively participate in the NLE (Theme N2). While the first need seems intuitive, we are not aware of any previous publications stating the second need with regard to other NLE. With regard to Theme N2, an international survey suggested that medical educators believe that their work is not adequately recognised (Huwendiek et al. 2010). As such, Theme N2 seems to be a worldwide phenomenon rather than a typical Swiss phenomenon, although this does not make it any less important to address this issue. Indeed, the issue of incentivizing clinicians to engage in medical education is challenging, as outlined in a recent scoping review by Wisener and Eva (2018).

In the meantime, many of the needs for further development have already been addressed. For example, regarding the first need, several projects have been commissioned for the further development of the Swiss NLE, such as a project on how to improve the assessment of communication competencies within the exam. This four-year project had a positive impact on the assessment of communication skills. Further pilots within local faculty OSCEs on using children as standardised patients have been successfully performed (Krings et al. 2017). Furthermore, tablet-based checklists in the CS exam have been in use since 2015 (Wagner et al. 2016) and a full electronic written exam with tablets will be introduced in 2021. The establishment of an adequate incentive structure (Theme N2) is one of the most decisive steps for sustainable exam development. The framework in which the development of questions takes place plays an important role in this respect. Previously, clinicians received written information on how to write MCQ or, if the need was expressed, a half-day workshop was held. Building on experience from CS station development, clinician-authors for MCQ are now invited to a two-day question-writing workshop. This retreat − away from everyday clinical work − together with other clinicians and with methodological support from exam experts and IT staff has proven to be very successful: The quality and quantity of the questions increased noticeably, the authors’ satisfaction with the process improved, and cost savings were also achieved, since the follow-up of the questions was associated with less work. In the meantime, also the Swiss Catalogue of Learning Objectives has been revised now called ‘PROFILES’ (Michaud and Jucker-Kupper 2016). This ‘PROFILES’ document includes a chapter based on the CanMEDS 2015 report, a chapter supplying a list of nine entrustable professional activities (EPAs) and a chapter on ‘Situations as Starting Points’ listing a set of clinical symptoms, findings upon physical examination or laboratory tests/imaging (Michaud and Jucker-Kupper 2016). PROFILES acknowledges a longitudinal approach of assessment. Accordingly, the Swiss medical faculties are expected to orientate their examination system towards a longitudinal assessment program. As medicine itself and how we teach medicine develops constantly further, it is evident that the NLE will have to further develop as well.

In 2009, Melnick summarised the value of the national licensing exams in Canada and the USA as lying in the ‘equity of common standards, external audit providing transparency and accountability, providing a bounded environment for innovation, providing data for evidence-based education and regulation, encouraging evidence-based workforce mobility and fostering high quality assessment’ (Melnick 2009). With this article, we would like to add that NLE can also be a driver of change in medical education and of collaboration of medical faculties in the respective country. As stated above, the NLE supported − together with the national law that allows individual universities to shape their own profile − the further development of competency-based education in Switzerland. As Govaerts and colleagues (Govaerts et al. 2019) describe the management of tensions in assessment as moving beyond ‘either-or’ thinking to ‘both-and’ thinking, it is our experience that the combination of both assessment paradigms is possible and even supports innovation: give regulatory freedom and support for innovations like EPAs and longitudinal assessment approaches and have a NLE as outcome measure, benchmarking and quality control.

Strengths of this study

We are not aware of any other study investigating an NLE including its impacts, strengths and needs for further development. In this respect, the present study is therefore unique. Many relevant stakeholders and experts were involved, allowing for a broad coverage of the opinions of these groups.

Limitations

Our study involved primarily stakeholders who heavily invested in the success of the program and are therefore more likely to report the things that went well and not necessarily the ones that are wrong with it. Involving additional stakeholder might have brought up additional aspects. To gain an even clearer picture, further studies including candidates, young practising doctors and clinical supervisors could thus be helpful.

Conclusion

According to the participants of this study, the Swiss NLE has notable impacts on medical education and collaboration of the faculties of medicine in Switzerland. In this sense, this NLE can be seen as a driver of change in medical education. Overall, the NLE is perceived as appropriate for its tasks, and many of the needs for further developments have been addressed in the meantime. However, to keep the exam fit for purpose, constant further development is necessary. As our focus group study involved solely stakeholders and involved experts, other relevant involved groups, such as candidates, should be included in future studies.

Glossary

Communities of practice: Are socially interacting groups of persons that have learning as an aspect of membership. In its central sense, it means the idea of learning as a participatory social process. However, there are seen to be two aspects of the community of practice theory: the earlier idea of increasing membership of the community through community members’ actions, behaviours and language, and the more recent definition with a greater emphasis on ‘deliberate group cultivation’.

Buckley H, Steinert Y, Regehr G, Nimmon L (2019): When I say … community of practice. Med Educ. 2019;53(8):763–765.

Acknowledgements

We thank all experts (working group MC, working group CS, site responsibles and further experts) and stakeholders (Vice Deans, BAG, MEBEKO) for sharing their precious time to participate in this study. We are grateful to the exam commission (President (former): Andre Perruchoud) and the FOPH for their support and for partially funding this quality assurance project. We thank Glenn Regehr (Centre for Health Education Scholarship, University of British Columbia, Vancouver, Canada) for helpful advice on a previous version of this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Sören Huwendiek

Sören Huwendiek, MD, PhD, MME, is an associate Professor of Medical Education and Head of the Department for Assessment and Evaluation at the Institute for Medical Education in Bern, Switzerland. He is also a paediatrician. His research focuses on innovative teaching and assessment formats with the ultimate goal of improving patient care.

Daniel Jung

Daniel Jung, MS, economist, specialised in health economics, statistics and methodology. He worked at the Institute for Medical Education in Bern, Switzerland, from 2011 to 2015. Today he is an independent entrepreneur in Hamburg, Germany.

Christian Schirlo

Christian Schirlo, MD, MME, is Director of the Study Center at the Department for Health Sciences and Medicine at the University of Lucerne, Switzerland.

Philippe Huber

Philippe Huber, MD, is an associate Professor of Internal Medicine at the University of Geneva, Switzerland. He is also a geriatrician. His research focuses on integrated medical care models and medical education.

Gianmarco Balestra

Gianmarco Balestra, MD, is an internist and intensive care consultant at the University Hospital Basel and at the Dean’s office of the Medical Faculty Basel responsible for the NLE at the exam site Basel. His research focuses on basic understanding of microcirculatory and mitochondrial oxygenation of the heart as well as medical teaching (development of novel exam components to the FLE and feedback giving to enhance post-graduate education).

Sissel Guttormsen

Sissel Guttormsen, Prof. Dr. phil, is the director of the Institute for Medical Education. She was the head of the national group governing the development of the NLE until its introduction in 2011. Her main research fields are implementation and impact of technology supported learning and assessment; and research guided implementation of blended learning concepts in medical education.

Christoph Berendonk

Christoph Berendonk, MD, MME, is deputy head of the Department of Assessment and Evaluation at the Institute of Medical Education in Bern, Switzerland. His research focuses on performance assessment in the simulated and the workplace setting.