Trainers’ and trainees’ expectations of entrustable professional activities (EPAs) in a primary care training programme

ABSTRACT

Introducing Entrustable Professional Activities (EPAs) into primary care postgraduate medical education (PGME) programmes may be challenging, due to the general nature of primary care medicine, but trainers and trainees both stand to benefit from their use. We investigated the expectations of trainers and trainees in a primary care PGME programme regarding the use of EPAs. We held two focus group discussions with trainers and four with trainees from the Dutch General Practice training programme, to explore their views on the use of EPAs in their training programme. Focus group discussions were audio-recorded and transcribed verbatim. The transcripts were analysed using conventional content analysis. Trainers and trainees felt that the large number of EPAs in the training programme, and the general way they are formulated, made them unsuitable for use in formal assessments. However, they felt that EPAs can be a useful aid to trainee learning. EPAs may help trainers to give trainees specific feedback on their performance. While the use of the classic EPA method in primary care PGME programmes may be challenging, EPAs in such programmes might be more suitable as design and learning tools than as a tool for formal assessment.

KEYWORDS:

• Education
• medical [MeSH]
• entrustable professional activities (EPAs)
• primary health care [MeSH]
• qualitative research [MeSH]

Introduction

Primary care physicians, such as general practitioners (GPs), play a central role in healthcare systems all over the world. They provide comprehensive care for every patient, irrespective of their background, or the nature of their illness. In addition, they help to coordinate the care their patients receive [1–3]. Training doctors to become competent primary care physicians requires a robust postgraduate medical education (PGME) programme that properly addresses each important aspect of the profession.

Many PGME programmes base their curriculum on competency frameworks such as the CanMeds competency framework [4]. By defining the abilities required of health care professionals, frameworks can serve as a useful guide in trainee learning and assessment [5,6]. However, the use of competency frameworks in daily clinical practice risks an excessive emphasis on a limited amount of specific competencies, thereby undermining the holistic approach that characterises competency frameworks in which the integrated use of multiple competencies is required [7,8]. Furthermore, trainers and trainees have suggested that competency frameworks are too theoretical in nature to be applied in everyday practice [5,6].

Ten Cate et al. proposed that Entrustable Professional Activities (EPAs) be used to support trainee learning and assessment, as a way of overcoming the problems associated with competency frameworks [5,9,10]. EPAs define the tasks performed by healthcare professionals in everyday practice and integrate the competencies linked to such tasks [9]. They should contain a clear description of the activity in question, and of the knowledge, skills and attitudes needed to be able to independently perform that activity [10]. The entire body of EPAs must reflect the scope of a healthcare professional’s work (8, 9).

A defining feature of the assessment of an EPA is that it involves an entrustment decision. During the training programme, trainees learn to perform the specified tasks and acquire the associated competencies. As they acquire more competencies, trainees are gradually entrusted to perform EPA independently, without being supervised by their trainer [5,9,10]. When trainees reach the stage at which their trainer trusts them to perform all EPAs without supervision, they are considered to have acquired sufficient proficiency in all areas of competence, and are judged to be ready to assume the role of a healthcare professional [6,10]. In the interests of learning efficiency and to make each entrustment decision a truly important step in the trainee’s progress towards working independently, the Association for Medical Education in Europe (AMEE) guide for EPA-based curricula recommends a maximum of 10 EPAs to be formulated for each year of a training programme [10].

While many PGME programmes have already introduced EPAs into their curricula, very few primary care PGME programmes have done so [11,12]. This may be due to obstacles related to the generalist field of work in primary care as it seems impossible to describe all the activities carried out by primary care physicians in a limited number of EPAs. Any attempt to do so would either create too many EPAs to deal with in the period of the training programme or a smaller number of EPAs that are formulated in excessively general terms rendering them meaningless. Despite the above issues, we anticipate that trainers and trainees in primary care PGME programmes would benefit from the use of EPAs. As few primary care programmes are using EPAs, little is known about trainers’ and trainees’ expectations regarding the use of the EPA method in primary care PGME programmes. Accordingly, our investigation of these expectations forms the topic of this article.

Methods

Context

GPs play a central role as primary care physicians within the Dutch healthcare system. They act as gatekeepers for the health care system and provide healthcare to residents of the Netherlands when it is required. Dutch GP training programmes prepare GP trainees for a broad professional field of work. Trainees spend the first and the third year of their training programme working in a general practice setting, where they are supervised by a specially trained GP trainer. Trainers are responsible for coaching, teaching and assessing the trainees. The second year of the Dutch GP training programme is dedicated to traineeships in clinical or emergency medicine, care of the elderly, psychiatry or an elective course [13,14]. Trainees attend an educational programme for one day a week dealing with general aspects of general practice and activities not encountered in everyday clinical practice.

The curriculum of the Dutch PGME GP training programme is based on a modified version of the CanMeds competency framework [15]. To support the use of the competency framework in everyday clinical practice, the Netherland’s National GP Training Institute has drafted 81 EPAs, classified into 10 themes. Each theme reflects an important component of everyday general practice that must be addressed during the training programme. Each EPA within these themes contains descriptions of the professional activities that are central to the theme in question and the associated competences (Appendix) [16]. The themes and EPAs were developed by 10 expert groups. They are based on the collective vision of the Dutch College of GPs (NHG) on general practice in the Netherlands, the framework for GPs, and the blueprint for the Dutch GP training programme [17]. Within the National Training Scheme, these themes and EPAs support trainee learning as templates for monitoring the development of competencies and for the provision of feedback. They also serve as templates for the design of the local training institutes’ curricula [18–21]. The implementation of EPAs in the training programme commenced in March 2017.

Participants

We conducted this study among GP trainers and GP trainees at the Academic Medical Centre’s (AMC) GP Training Institute in Amsterdam. We selected separate focus groups for trainers and trainees to eliminate any hierarchy and promote free and open discussions [22–24]. For the purposes of this study, we approached trainers who guide trainees in the first and the third year of the AMC’s GP training programme, as the EPAs mainly reflect activities that are addressed in GP practices. All trainees at the AMC’s GP Training Institute were approached and invited to participate in this study. Potential participants were informed about the purpose of the study in person and by email, by the principal researcher (LB). The trainers’ focus groups were held during the faculty development programme, while those involving trainees took place during their weekly training day at the Institute.

Study design

Our decision to use focus group discussions was based on the exploratory nature of this study. Focus group discussions are particularly useful for exploratory purposes, as they can elicit a wide range of views and opinions that can be shared and explored in the group. This generates many different viewpoints, including opposing viewpoints [24,25].

Data collection

The data used in this study was obtained from a larger study into the trust relationship between trainers and trainees, and the role of trust in the use of EPAs. Our results concerning the trust-relationship between trainers and trainees will be published elsewhere. In the period from November 2016 to March 2017, we held two trainers focus groups and four trainees focus groups. All participants signed a consent form. Each focus group consisted of six to eleven participants and each session lasted for 45–70 minutes. Demographic data on the participants was collected by means of a questionnaire. Aside from a general introduction, all participants were naïve to the concept of EPAs, and none of them had ever worked with EPAs during training.

Each focus group discussion was led by a skilled moderator (MV, NvD), while an assistant-moderator (LB or a research assistant) took notes. The confidentiality of focus group discussions was ensured by the use of moderators and assistant-moderators who were not directly or personally involved in the education and supervision of trainee GPs and GP trainers.

Each focus group discussion was preceded by a general presentation about the EPA method. There were presentations on the EPAs used in the Dutch GP training programme, followed by an opportunity to ask questions. An opening question initiated the discussion, after which key questions were used to maintain the momentum (Table 1). Data collection and data analysis were performed iteratively. The results of the data analysis were used to guide subsequent data collection [22,23]. The topics for discussion were based on published material concerning the EPA method and EPAs in the Dutch GP training programme [10,21]. During the study, the topic list was modified in response to the findings of previous focus groups. After each meeting, the moderator and assistant-moderator conducted a post-meeting debriefing to discuss the course and the details of the discussion.

Table 1. topics used to facilitate focus group discussions.

Opening questions

How can the EPA method best be used in the context of Dutch GP training programme?

What are the advantages and disadvantages of the use of EPAs in the Dutch GP training programme?

Key questions

How can EPAs support the trainee’s learning process? How do EPAs aid in the identification of knowledge gaps? How do EPAs help in the creation of a personal learning agenda? What are advantages and disadvantages of using EPAs to support the trainee’s learning process?

How can EPAs help trainers evaluate their trainees? How can the EPAs be used to support an evaluation of the trainee’s progress in the various areas of competence? How do the EPAs help trainers to provide trainees with feedback? Can the EPAs provide support in the case of underperforming trainees? What are advantages and disadvantages of using EPAs to evaluate trainees?

Data analysis

Audio recordings of the focus group discussions were transcribed verbatim by LB and by an external transcription agency. Transcripts were anonymised. Exchange of the data with the external transcription agency took place by means of a secured digital environment (SURFfilesender) [26]. The external transcription agency signed a ‘processor-agreement’ and guaranteed that data was treated confidentially. Transcripts were analysed using conventional content analysis, to provide an overview of expectations concerning the use of EPAs [27]. As a first step, LB performed an inductive coding process on four transcripts. The resultant code book was used to identify each theme and concept that emerged from the discussions of people’s expectations about the use of EPAs. The details of this code book were discussed with NvD, to ensure coding reliability. If the transcripts failed to provide sufficient information, we listened to the audio-tapes to support the coding process. As a second step, LB and NvD compared the codes from the code book, to create categories and to organised these categories in the code book. As a third step, the remaining two focus groups were coded by LB, using the categories defined in the code book. No new topics and categories emerged from the last two focus groups, so we assumed that data saturation had been reached. The final code book was discussed with NvD and MV. In the last step, we analysed the code book categories to create themes that represented expectations concerning the use of EPAs in everyday practice. The process of data analysis was supported by the use of memos and diagrams [27–29]. MAXQDA software, version 12 [30], was used for the data analysis.

Results

Fifteen GP trainers and 34 GP trainees participated in the study. The median ages of trainers and trainees were 50 (range 39–62) and 29 (range 27–38), respectively. Sixty percent of the trainers were male, as were 18% of the trainees (Table 2). Trainers had a median working experience in general practice of 10.0 years (range: 3.0–22.0 years). Of the participating trainees, 52.9% were in the first year of the training programme, and 47.1% were in the second year. Trainees had a median of 2.0 years of working experience (range: 0.5–4.5 years) before the start of the training programme. None of the trainees had worked in general practice before starting the training programme. None of the participants was involved in the development of the EPA-based curriculum for the Dutch GP training programme. Trainers and trainees share their opinion regarding the application and use of EPAs in a primary care PGME programme. Only in the case of ‘Disadvantages of using EPAs’, trainers mention an additional disadvantage, this is indicated in the text. Quotes that support the results are shown in Table 3.

Table 2. Demographic characteristics of the participants.

		GP trainers	Trainee GPs
Number of participants		15	34
Age (median (min-max)		50.0 (39.0–62.0)	29.0 (27.0–38.0)
Gender (N (%))
	Male	9 (60.0%)	6 (17.6%)
	Female	5 (33.3%)	28 (82.4%)
	Unknown	1 (6.7%)	0 (0.0%)

Table 3. quotes of participants.

Advantages of using EPAs
Quote (GP trainer): ‘Reading through them [EPAs], I see that they are all about situations. And situations can easily be recognised, which makes it very practical’.
Quote (Trainee GP): ‘They [EPAs] can help me to discuss my performance. I can ask my trainer: Why am I not performing well in that area?’
Disadvantages of using EPAs
Quote (GP trainer): ‘I do feel that many important things you experience in everyday practice are missing’.
Quote (GP trainer): ‘I wonder if we really want to use it [EPAs] as a checklist. I would rather leave it with the trainees, who could then use it [EPAs] to identify learning goals as well as any gaps in their own knowledge, and work from there’.
Application of EPAs in the training Programme
	EPAs as trainee learning tools
		Feed up
		Quote (GP trainer): ‘This is the essence of a GP’s profession, the end-product you have to deliver as a GP trainer. All the activities are precisely defined’.
		Quote (Trainee GP): ‘I know this list covers all of the important areas in our profession. It gives me peace of mind to know that I am not inadvertently being incompetent by forgetting to cover various vital aspects of the profession’.
		Feedback
		Quote (GP trainer): ‘I think you can also use them [EPAs] in a positive way, when a trainee is performing very well, for instance, you could say “That went very well, let’s take a look at the list of EPAs and see what you already know”’.
		Quote (Trainee GP): ‘A trainer often provides general feedback, such as “I’m happy with the way you’ve been performing lately”. But I think this [EPAs] can make your feedback much more specific, enabling trainers to say: “In the area covered by this topic, I would assess your performance as…”’.
		Feed forward
		Quote (GP trainer): ‘I have encountered this issue with one particular trainee who was ready to become a GP. This individual was looking for further challenges, and I can imagine that this list [of EPAs] could help in situations like that’.
		Quote (Trainee GP): ‘I like the way it works. You can focus on one specific subject for a certain period. Afterwards you can evaluate the period and your progress with your trainer’.
	EPAs for Trainers and training Practices
	Quote (GP trainer): ‘You can also use them [EPAs] to identify any shortcomings in training situations, thus helping trainees to understand what learning activities a specific training practice can and cannot offer them’.
	EPAs for curriculum design
	Quote (GP trainer): ‘And of course, how to set up a training programme, as it can be very useful to involve others in these programmes’.

Advantages of using EPAs

Both trainers and trainees feel that the themes and EPAs provide a broad, structured overview of training requirements that addressed all aspects of general practice. This helped trainers and trainees keep track of the latter’s learning experiences and progress. The EPAs provided them with specific situations and tasks commonly associated with everyday clinical practice. This helped trainees to request targeted feedback and enabled trainers to provide it. The EPAs are provided with a list of corresponding competencies, making it easier for trainers and trainees to recognise the areas of competence involved.

Disadvantages of using EPAs

Trainers and trainees felt there were too many EPAs, which tended to obscure the overview of training requirements. They felt that some EPAs were formulated in overly general terms and therefore explained very little, which made them unsuitable for use in formal assessments. Trainers and trainees feared that the long list of EPAs may lead to a situation in which this is used as a checklist, rather than as a supportive educational tool. They were concerned that the use of EPAs in formal assessments may result in an increased workload, in terms of the assessment itself, and of the work involved in incorporating assessments into the trainee’s portfolio. Even though trainers and trainees felt there were too many EPAs, both groups identified important aspects of a GP’s work that were not covered in the current list. These aspects were related to medical skills and to general competencies, such as communication with patients, professional development and co-operation with co-workers. The trainers indicated that they were unable to cover all EPAs during the training periods in their general practice. The reasons they cited include the composition of the patient population in their practice, and the low incidence of certain disorders.

Application of EPAs in the training programme

Trainers and trainees saw various opportunities for the use of EPAs in the GP training programme. These are summarised in Table 4.

Table 4. Proposed options for EPA use in the GP training programme.

Trainee learning Feed up: providing insight into training requirements Feedback: providing the trainee with targeted feedback on all aspects of their performance in the GP’s practice. Identifying existing expertise and any gaps in performance. Feed forward: writing a personal development plan, planning future internships. Providing high-performing trainees with challenges.

Trainer and training practice Identifying the trainer’s existing expertise and any gaps in their knowledge Identifying learning opportunities in training practice

Curriculum design for the GP training programme Adapting the GP training institute’s educational activities to the activities in the GP’s practice

EPAs as trainee learning tools

The EPAs can support trainee learning in various phases of the learning process.

Feed up – Both trainers and trainees indicate that EPAs give them added insight into the training requirements of the GP training programme. The EPAs’ detailed descriptions of the activities in question show what is expected of the trainees. Trainers can use this information to help them assess the trainees’ competencies. The trainees, in turn, can use this information to formulate targeted learning objectives for their personal development plan.

Feedback – Although they did not consider them to be suitable for use in formal assessments, trainers and trainees did feel that EPAs could be used to support trainee assessments, by underpinning the assessment of areas of competence. Each EPA contains a detailed description of the associated activities, as well as information about the competences needed to independently execute that particular EPA. This makes it easier for trainers to substantiate their assessment of their trainee’s competences.

The EPAs’ detailed descriptions of the associated activities can be used in learning discussions, to clarify what trainees already know and to identify any gaps in their knowledge. These descriptions may help trainees to request targeted feedback and enable trainers to provide such feedback on many aspects of their trainee’s performance.

Feed forward – Trainees can structure their learning process in clinical practice based on an understanding of what they already know and of the gaps in their knowledge, as well as on feedback from the trainer. They can use EPAs to write a personal development plan, placing extra emphasis on any EPAs they have not yet mastered. The trainees can use their understanding of what they already know and of the gaps in their knowledge to plan a follow-up to their training programme, involving additional learning strategies or internships in themes that have not yet been adequately addressed. The trainers are keen to use EPAs to challenge high-performing trainees with new learning topics.

EPAs for trainers and training practices

The trainers indicated that they could use EPAs to identify their own strengths and weaknesses. Such information could help them plan Continuing Medical Education (CME) courses, to fine-tune their own knowledge and skills. EPAs can help trainers create profiles with learning opportunities for training practices, based on a practice’s population and on frequently encountered problems in that practice. This information can help trainees to choose a training practice offering learning opportunities that match their learning needs.

EPAs for curriculum design

EPAs can be used to establish the training programme at local training institutes, to provide trainees with educational activities that are tailored to the activities encountered in everyday practice.

Discussion

Summary

The aim of this study was to identify expectations about EPAs among the trainers and trainees involved in a primary care PGME programme. One advantage they mentioned was that EPAs provide a structured overview of the training requirements. They valued the EPAs’ descriptions of specific situations and tasks, and the way in which they linked competence areas to the tasks in question. One disadvantage cited was the large number of EPAs, and another was that some of them are formulated in general terms and therefore explain very little. In addition, some expressed the view that the list of EPAs is not complete. Both trainers and trainees feel that the large number of EPAs and their general formulation make these tools unsuitable for formal assessments in primary care PGME programmes. However, they do feel that EPAs can be used to support the assessment process, the trainee’s learning process and the design of training programmes.

Comparison with existing literature

Our study shows that trainers and trainees believed there were too many EPAs in the Dutch GP training programme. At the same time, they felt that the list was not complete and that these EPAs have been too generally formulated. It seems that many primary care PGME programmes face a common dilemma. They must cover the broad range of medical tasks involved in general practice while, at the same time, working through a comprehensive list of EPAs for educational purposes. Other primary care PGME programmes have also formulated more EPAs than is recommended [10–12]. They have indicated that their list of EPAs lacks certain essential skills and expertise required by primary care physicians [12]. As a result, the EPA frameworks used in primary care PGME programmes are too extensive, lack essential elements of general practice, or fall short in terms of specificity. These considerations may well undermine the effectiveness of EPA frameworks in assessment and training [10]. However, in situations where EPAs are not primarily used for assessment purposes, issues relating to their number and formulation may not be quite so important.

One question arising from our study is whether or not the EPAs’ perceived unsuitability as an assessment tool is actually a problem. One of the reasons for developing EPAs is the assumption that trainers and trainees have difficultly using competency frameworks in everyday clinical practice [5,6]. However, our results tend to contradict this assumption. Trainers appear to have no difficulty in recognising the general competencies of a GP, given their ability to specify the shortcomings of the EPAs in this regard. This suggests they may not consider the competency framework to be too theoretical and may, indeed, find it more applicable to everyday clinical practice than was expected when the EPAs were first introduced. This may be due to the passage of time since competency frameworks were first introduced into PGME programmes, and that people have become accustomed to using them. In situations where a competency framework appears to be more fully integrated than might be expected, the use of EPAs to support competency-based training may have too little added value to make it worthwhile.

Trainers and trainees both recognise the educational value of EPAs in generalist PGME programmes. Our study shows that the information provided by EPAs can be used in every component of formative assessment (feed-up, feed-back and feed-forward) [31]. Given the pivotal role of formative assessment in self-directed learning, this could promote the process of life-long learning [32].

Strengths and limitations

Focus group discussions proved to be invaluable, in terms of clarifying participants’ expectations of a new educational programme [33–35]. They showed how the participants expect to use this new educational programme in everyday practice. The participants were largely naïve to the concept of EPAs. This enabled us to explore the trainers’ and trainees’ expectations of these tools, and to understand their views on how EPAs should be used in everyday practice. This information could help us to understand how EPAs might best be used in generalist PGME programmes.

The results of this study were obtained in the context of a larger study into trust relationships between trainers and trainees. However, given the complex nature of the relationship between trust and assessment [36], the research project’s design may have biased the answers given by its participants. Non-complex, non-innovative assessment methods make it easier to maintain trust. Thus, discussing trust as well, may have influenced the participants’ views regarding the use of EPAs in assessments [36]. We only studied the planned use of EPAs in the Dutch GP training programme, rather than their actual implementation, as that took place after our focus group discussions had been concluded. This was entirely in keeping with the aim of our study, which was to inform the implementation process by gaining insights into the expectations of those involved. Nevertheless, the reader should be aware that expectations may not correspond to actual experience in practice [37]. Finally, all participants were naïve to the concept of EPAs, and were shown same general introduction about the EPA-method. As trainers and trainees received the same general introduction, this might have led to the similarities in opinions regarding the use and application of EPAs. Future research might benefit from exploring differences in trainers and trainees’ opinions regarding EPAs.

Implications for research and everyday practice

This study shows that while the introduction of EPAs into the training programmes offered by primary care PGME programmes may involve some difficulties, it presents a number of opportunities [5,6,9]. An exploratory study of other PGME programmes could identify any problems they may experience when introducing the EPA method. The results obtained in this way could enhance the implementation of the EPA method and promote the use of EPAs in PGME programmes. From a practical point of view, the use of focus group discussions to generate input for educational changes, prior to their implementation, might enhance actual uptake.

Results of this study indicate that trainers and trainees share their opinions regarding the use and application of EPAs in a primary care training programme. Currently, we are undertaking a study evaluating whether trainers and trainees share their opinions regarding the attributes that they award towards EPAs. This information may help to support the implementation-process of EPAs in a (primary care) PGME programme.

Conclusion

EPAs are intended to support trainee learning and assessment. In this context, they can be useful in primary care PGME programmes, such as GP training programmes. However, the wide range of work performed by primary care physicians may make it more difficult to use the EPA method, as it was originally defined. This is because more EPAs might be needed to describe all aspects of the work involved in everyday practice than the number recommended for assessment purposes. The existing EPAs might not cover all aspects of the work involved in everyday practice. Thus, EPAs might be more suitable as design and learning (or formative learning) tools than as tools for formal assessments in primary care PGME programmes.

Ethical approval

This study was approved by the Ethical Review Board of the ‘Netherlands Association for Medical Education’ (Nederlandse Vereniging voor Medisch Onderwijs (NVMO, file number 725)).

Acknowledgments

The authors thank the participating trainers and trainees for their willingness to contribute to the study. We would like to especially thank Suzanne van Rhijn for her support during the study.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This publication was written as a part of the project ‘The use of Entrustable Professional Activities in Assessment in General Practice Specialty Training’ (project number 839130004), that received funding from the ‘Netherlands Organisation for Health Research and Development’ (ZonMW).

Appendix EPA from the Dutch GP-training programme (translated from Dutch)

Table

Theme	Emergency Care
EPA	Determines the condition of the patient using the ABCDE-system and performs necessary actions.
Description	Involves the urgency and the condition of the patient in clinical reasoning, rather than diagnosis-focused thinking.
	Identifies the reasons of acute need for care, including psychiatric and psychosocial crisis situations, frequently in unknown patients, without access to additional information.
	Identifies potentially life-threatening diseases and disorders, or rules them out; involves the differences in prevalence, incidence, and the a-priori probability of emergency cases in the GP-practice, the OOH-GP-centre, ambulance services and the emergence care department in this process.
	Handles emotions, fears and concerns in a professional manner, takes into account the safety of everyone involved in the situation.
Competence areas	Medical Expert, Communicator, Collaborator, Leader, Scholar