The recommended description of an entrustable professional activity: AMEE Guide No. 140

Abstract

Entrustable professional activities (EPAs) have received much attention in the literature since they were first proposed in 2005. Useful guidelines, workshops, courses, and conferences have supported faculty in developing programs and designing assessment procedures using EPAs and entrustment decision-making. Yet, the need for clarification remains, particularly as more programs make the step from design to implementation.

Well-written EPAs provide a natural construct to establish the outcome of training. To be useful, EPAs require more than a suitable title. This AMEE Guide elaborates eight sections of a full EPA description, and provides explanations and justifications for each. These sections are: title; specification and limitations; risks in case of failure; most relevant competency domains; knowledge, skills, attitudes and experiences; information sources to assess progress and support summative entrustment; entrustment/supervision level expected at which stage of training; and time period to expiration if not practiced.

Constructing fully elaborated EPAs creates a shared mental model amongst learners and programs, informs competency-based curriculum design, directs ad-hoc and formal entrustment decision-making, and provides standards for certifying bodies and boundaries for scope of practice. The framework intends to support curricular leaders looking to adopt new EPAs, or revise and define established EPAs for competency-based education.

Keywords:

• General
• outcome-based
• general
• work-based

Introduction

Entrustable professional activities (EPAs) have become immensely popular in a relatively short period of time. EPAs are now adopted or being introduced in a variety of postgraduate medical specialty training programs and undergraduate medical education programs in several countries (ten Cate 2019). More recently, other health professions have expressed interest in the concept, such as nursing, physiotherapy, dentistry, pharmacy and veterinary medicine (Pittenger et al. 2016; Chesbro et al. 2018; Lau et al. 2020; Tonni et al. 2020; Duijn et al. 2019).

Practice points

• Entrustable professional activities have become a popular component of competency-based medical education.

• While many proposals for EPAs in programs have been published, practical applications emerge at lower speed.

• AMEE Guide 99 has been devoted to curriculum development with EPAs.

• This Guide is intended to further support curricular leaders in creating new EPAs, or revise and define previously established EPAs.

A quick search in PubMed (May 2020) yields over 450 journal articles since 2007 with the word “entrustable”–a neologism–in their title or abstract, with a rapid increase since 2015.

While many publications refer to original texts (ten Cate 2005, 2013; ten Cate et al. 2015), and use the concept as intended; in practice there is diversity in the application of the term EPA, and sometimes confusion. In 2013, when EPAs became popular in some communities, one educator once complained that “EPAs have become a label de jour for virtually everything.” In fact, concern has emerged that this dilution of the concept and the criteria used for identifying and describing EPAs is undermining their value in advancing competency-based medical education (CBME) (ten Cate et al. 2015; Taylor et al. 2017, 2020; O’Dowd et al. 2019). This has been a reason to explain and stipulate its definition, purpose and use in several publications (ten Cate 2013, 2018, 2019, 2020; ten Cate et al. 2015, 2020). Over the years a consolidation has emerged about how to define and describe EPAs, but an updated practical guideline focused on the description has not yet appeared.

Compared to earlier examples, slight improvements have been proposed for the full description of EPAs, which is reason to rehearse and elaborate the details of an EPA.

This contribution is meant to explain and justify the elaborate description of entrustable professional activities, in support of development of curricula and assessment procedures that focus on EPAs.

EPA: Rehearsing its definition, purpose, and features

An entrustable professional activity is a unit of professional practice that can be fully entrusted to a trainee, once he or she has demonstrated the necessary competence to execute this activity unsupervised. “Unit”signifies a discrete task (e.g. “Managing patients with cataract”) or bundle of tasks (e.g. “Performing the procedures of internal medicine” or “Managing an inpatient medical service”) and should be suitable for credentialing. EPAs are legitimate contributions to health care practice made by trained professionals; learners performing EPAs can only achieve this in a clinical context. “Presenting a paper in a classroom setting” would not be an EPA. Even within the clinical context, not all activities can be qualified as EPAs. “Organizing birthday gifts for secretaries” would not be one, even if many professionals would regularly do that. But legitimate contributions to health care can be small. If a student nurse is asked to take the temperature of a patient and enters it in a health record without a supervisory double check, there is entrustment of a contribution to health care, and the activity would principally qualify as a (tiny) EPA.

For practical purposes, EPAs are larger. EPAs can be conceived of as broad tasks or groups of tasks. The purpose of using EPAs is to operationalize competency-based medical education through a stepwise and safe engagement of trainees in clinical practice – linking progressive proficiency to progressive autonomy in patient care. Competency-based education aims at securing at least the minimum standards for unsupervised practice for all graduates. Initial attempts to secure assurance of this standard depended on directly assessing long lists of individual competencies. However, competencies can feel too detached from the clinical duties in which they are applied to be optimal for assessment. EPAs were introduced to make the connection between competencies (characteristics of professionals or learners) and the professional activities to be entrusted to them (ten Cate and Scheele 2007). The MD license or specialty registration should guarantee readiness for unsupervised clinical practice, but its breadth is too large to oversee for any educator. EPAs break them down into units of practice that can be overseen, assessed, monitored, documented and certified.

The benefit of identifying units of practice is also that formal certification for all contributions to health care does not have to occur at the end of training, but may be achieved earlier once learners have established their readiness for the responsibility of performing those EPAs. While a summative entrustment decision to certify for an EPA may not have legal status, its philosophy aligns with the often recommended graduated increase of professional responsibilities and contributions to professional practice (Kennedy et al. 2005; Dijksterhuis et al. 2009; Kashner et al. 2010; Yardley et al. 2018). “Legitimate participation” in a community of practice, following the model of “cognitive apprenticeship” have often been cited as theories to support this model (Lave and Wenger 1991; Collins 2005) – and EPAs can serve to realize this in practice.

Although commonly recognized by title alone, elaborated descriptions of EPAs serve important purposes for both regulation of health care provision and support of clinical development in learners. Full descriptions of EPAs create clarity among learners, educators, employers and colleagues, professional and interprofessional, about what exactly the individual is ready, permitted, or entitled to do in health care. It directs how much supervision is needed when the clinical learner is formally entrusted with this EPA, within an educational program or outside a program. For what can this individual be deployed? Additionally, the full EPA description serves as a guide to learners, which can empower them in their development to set goals and to help self-monitor progress. EPA titles alone cannot achieve this. Although the title is often thought of and used as though it was the complete EPA, successful implementation of EPAs in a competency-based curriculum requires careful construction of fully described and defined EPAs.

The features of EPAs, most of which were already included in their initial definition in 2005, are summarized in Box 1 (ten Cate 2005; ten Cate et al. 2015; Taylor et al. 2017).

Practice points

• Entrustable professional activities have become a popular component of competency-based medical education.

• While many proposals for EPAs in programs have been published, practical applications emerge at lower speed.

• AMEE Guide 99 has been devoted to curriculum development with EPAs.

• This Guide is intended to further support curricular leaders in creating new EPAs, or revise and define previously established EPAs.

Box 1 Features of entrustable professional activities.

• Has a clearly defined beginning and end

• Independently executable to achieve a defined clinical outcome

• Is specific and focused

• Observable in process and measurable in outcome

• Clearly distinguished from other EPAs in the framework

• Reflects work that is essential and important to the profession

• Leads to recognized output or outcome of labor

• Is restricted to qualified personnel

• Requires application of knowledge, skills, and/or attitudes acquired through training

• Involves application and integration of multiple domains of competence

• Describes a task, not qualities or competencies of a learner and avoids adjectives (or adverbs) that refer to proficiency

Box 2 Recommended sections in a full EPA description.

1. EPA Title

2. Specification and limitations

3. Potential risks in case of failure

4. Most relevant competency domains

5. Required knowledge, skills, attitudes and experiences to allow for summative entrustment

6. Information sources to assess progress and support summative entrustment

7. Entrustment / supervision level expected at which stage of training

8. Time period to expiration if not practiced

EPAs versus skills and competencies

EPAs, as units of practice, constitute the description of work that operationally define a profession. They represent a job description, not a person description. They encompass the task list each clinical department, clinical ward, or health care worker may have for the day, for the week, or any period of time. Job descriptions can list EPAs in general terms, and task lists apply those to specific things that must occur in a plannable period of time. In contrast, competencies describe persons. Trainees who become competent professionals must acquire competencies that include knowledge, skills, and attitudes. It is how they synthesize and apply them to patients that generates professional work.

More than just doing a clinical task with a patient, an EPA must reflect an actual contribution to health care, in whichever form. Something that, to say it briefly, “does not have to be checked again.” For instance, if an EPA “Prioritize a differential diagnosis after a clinical encounter”when performed by a medical student will always be reviewed by a supervisor before any action is taken (if this EPA stipulates that team members must “endorse” and “verify” the working diagnosis), the EPA may be just a sandbox activity that is, no doubt, an excellent exercise, but not a true contribution to health care. EPAs are acts that invite learners to become members of a professional community by contributing to necessary practice.

The distinction of EPAs from competencies keeps confusing educators and we apologize for any failure to have explained that well enough in the past. Learners and professionals can possess competencies, knowledge or skills, as these all characterize abilities individuals bring to the role; but individuals cannot possess EPAs, which just characterize the professional work to be done. Confusion arises when lists of competencies or skills in fact show both. For example, the six ACGME qualities originally introduced as the “core competencies” included concepts like “patient care,” “interpersonal skills,” and “systems-based practice.” Interpersonal skills are clearly qualities of persons, but what are “patient care” and “systems-based practice”? If in surgery “cholecystectomy” is called a competency, it is “the ability to perform cholecystectomies” what educators actually mean. But what specific ability is this? If “Performing cholecystectomies” (a viable EPA) is what the learner is expected to be able to do eventually, it is not helpful to signify that as a competency, because the competencies the learner must possess to do this task are not specified. They could include manual skills, decision making skills, but maybe also interpersonal skills when negotiating the need for surgery with the patient. The word competency has been used in different ways, which is a source of confusion.

Recommended elaboration of an entrustable professional activity

In 2013 and 2015, guidelines for EPA descriptions, in a 7-item framework, were published (ten Cate 2013; ten Cate et al. 2015). Taylor and colleagues have used these guidelines to develop a useful rubric (“EQual”) to evaluate the quality of EPAs with a score (Taylor et al. 2017). The current guideline is a update of these earlier guidelines (ten Cate 2019) (including one added item), but foremost an explanation of the rationale for each of the items in the framework. Box 2 summarizes the eight items. A Supplementary Appendix provides a worked example of a fully described EPA “Managing patients with cataract”. The guideline includes comments on examples of EPAs that could be reformulated. The authors wish to stress that in many cases they respect and support the pioneers who wrote these EPAs but also provide recommendations for adaptations, as the concept is still new and can still benefit from future updates.

Box 3 Example of an EPA specification.

EPA: Obtaining Research Finances

Specification: This activity may contain the following elements:

• Searching and selecting research funds and suitable grants.

• Consulting and collaborating with grant support services if necessary.

• Writing a grant proposal according to the funding requirements. This usually includes a description of at least (a) context, aim and relevance (b) why the individual, center and/or consortium are suitable to perform this study (c) how project management is arranged, (d) a budget that suits the grant and the study and (e) the impact and dissemination of the project’s results.

• Defending a grant proposal by responding to review comments and giving an interview or presentation for a grant jury.

EPA title

An adequate title is paramount and must make the EPA comprehensible to all relevant stakeholders (learners, supervising clinicians, regulators, examination boards, nursing staff, etc.). The title should read as a generalized activity (as an item in a job description), not as specific for a person or a context. The recommendation is to use the continuous verb form (-ing) and plural when the title includes a verb and an object. So, instead of “Assess a patient using echocardiography” (Tanner 2020) the preferred title would be “Assessing patients using echocardiography” as if it were an item in a job description. Structuring titles this way emphasizes that a formal entrustment decision on an EPAs is not just for a single patient at one time but is on-going entrustment for that task across different patients. Next, the title should be parsimonious. In this case, just “Echocardiography” may even be enough, as any informed reader will understand that echocardiography is to be used to assess patients, but in other cases adding a verb may be necessary. When verbs are used, the verb should be the action done in performing the EPA. For example, “Performing resuscitation in unstable patients,” written as “Resuscitating unstable patients.” The title must be unambiguous and should not include choices, such as “Give or receive a patient handover” (Englander et al. 2016). A qualification of entrustment should hold for all sub-activities, so, in this case “Giving and receiving patient handovers” is advisable.

The title should preferably not be a chain of interrelated components. A proposed global health EPA “Engage with stakeholders to support strategic planning that addresses locally identified priorities” is fairly complex (an activity with main and nested aspects). Confusion could easily arise in interpreting the title: is engaging with stakeholders the core activity or making a strategic plan? Stakeholder engagement may have an independent purpose outside of its use in strategic planning and therefore be a separate EPA. Alternatively, it may only be required as a specified sub-activity, nested within the larger EPA of strategic planning. Either approach may be appropriate to the professional context, however, in constructing the EPA(s), it is best is to keep the title shorter and focus on a single activity.

While EPA titles preferably reflect one concrete activity, this is not always possible. Implementation of EPAs in educational programs force program committees to reconsider their breadth. Broad EPAs can be bundles of underlying activities and the title alone may then sound vague (“general procedures of the physician” (Englander et al. 2016) or “Communicating and collaborating with colleagues” (ten Cate et al. 2018a)), but if they have a clear specification of which smaller (or “nested”) activities are included when entrustment decisions are made, a vague title can still work. Specification is the topic of the next rubric.

Specification and limitations

There should be no ambiguity as to what the EPA is. Summative entrustment decisions about EPAs have serious implications, such as for patient safety. Confusion or differences in interpretation when assessing or awarding the EPA can potentially cause harm for patients and for trainees. More than just a title, a clear, detailed description is essential. Depending on the nature of the EPA, it may be specified and described in different ways. EPAs can be single tasks with components that may be chronological and can be detailed in a bulleted list. EPAs can also be a bundle of various tasks (e.g. “General procedures of a physician”; Carraccio et al. 2017), or contain smaller, nested EPAs (e.g. “Neurological consultation” as part of an EPA “The clinical consultation”; ten Cate et al. 2018b), or specialized tasks sometimes called Observable Practice Activities (e.g. “Titrate insulin based on glucose readings”; Warm et al. 2014). Summative entrustment decisions cannot serve small sub-activities that are only executed in conjunction with other activities. Instead, these decisions should be restricted to whole tasks, suitable for autonomous execution. Consequently, specifying an EPA must begin with establishing the scope of what makes the activity whole as a unit of professional practice. Certification to permit autonomous execution of an EPA by a trainee or professional, which may be done through awarding a STAR (statement of awarded responsibility ten Cate and Scheele 2007) or digital badge equivalent, has practical consequences for the contributions to health care, which directly arise from this specification. This aligns with recent call for more micro-credentialing (Norcini 2020).

This formal recognition may include specified elements of the EPA that are not always performed in one execution of the activity. That is obvious for bundles of tasks, but in other cases it should be stipulated that the EPA certificate or badge (Noyes et al. 2020) serves for an activity that may (i.e. or may not always) include all elements. An example, derived from a framework of EPAs for translational scientists, shown in Box 3 (Weggemans et al. [date unknown]), makes clear what individuals will be trusted to do by themselves if certified for this EPA. Even though not all components may always apply, they must all have been mastered.

Like the recommended title, the specification should not include reference to ability (e.g. “The resident should be able to…”) as that blurs the actual description of the activity.

Box 4 Section potential risks in case of failure: example from an EPA “Managing Patients with Cataract” (Chan et al. 2010).

• Pre-operative risks (medication use, pseudoexfoliation, mature cataract, trauma, small pupil, high refractive errors, previous ocular surgery, ocular pathology requiring treatment prior to surgery)

• Intra-operative complications (posterior capsular rupture, iris prolapse, zonular dehiscence, suprachoroidal hemorrhage)

• Post-operative complications (posterior capsular opacification, raised intraocular pressure, corneal decompensation, cystoid macular oedema, retinal detachment, endophthalmitis, refractive surprise)

• Anxiety, loss of confidence and psychological stress to patients

• Undue financial costs and societal impact

Limitations include elements or situations in which the individual is not necessarily qualified to perform when certified for this EPA, which further serve to clarify its scope to the outside world. After the introduction of robotic surgery, a discussion arose whether surgeons trained in it should be certified for a wide range of applications or very specific surgeries only. After voluntary skills courses or hands-on proctoring from other surgeons, they are now free to use the robotic surgical technology at their discretion (Sheetz and Dimick 2019). Physical therapists employ ultrasound (US) imaging technology, and similar discussions have arisen (Whittaker et al. 2019). Clearly, new technologies can benefit from clear EPA descriptions with specifications and limitations.

A special limitation regards the performance outside the context of certification. We have specified in the past that EPAs are applicable in “a given context” (ten Cate 2005). While this recommendation still holds in general terms, the purpose of using EPAs should always aim at transferable credentialing. In other words, an EPA earned in one institution or hospital should be recognized in other, comparable settings. But competence includes the ability to work in a context of interest (Ten Cate et al. 2010; Ten Cate and Billett 2014) and may, after transfer, require a period of adjustment and orientation before assuming full responsibility. Likewise, an OB-Gyn resident with an EPA “complex child delivery,” awarded in a general hospital at level 4 (“unsupervised practice,” i.e. distant supervision only (ten Cateet al. 2020)), may have difficulty when transferring to a tertiary referral center with a mix of highly complex patients only. A period of direct or indirect supervision (level 2 or 3) may be needed. That limitation is not written in the EPA description but should apply for any complex EPA.

The specification and limitations section serves as the “inclusion and exclusion criteria”of literature reviews and patient recruitment in clinical trials; for EPAs it clarifies the privileges implied with summative entrustment. For that reason, one section was chosen to specify both.

Potential risks in case of failure

All entrustment decisions bear implicit risks (ten Cate et al. 2016; Damodaran et al. 2017; Alanazi et al. 2019) and this section is used to specify and understand (to some extent) the adverse events that can occur when the task is not performed properly. Too often, difficult assessment decisions solely take into consideration the implications for learners; consideration of the consequences on future patients and society remains largely absent (Cleland et al. 2008). With struggling learners, promotion becomes the default when there is insufficient evidence to withhold it (i.e. insufficient grounds to fail) (Dudek et al. 2005). Considering the implications for patient welfare helps supervisors make challenging assessment decisions for learners not ready to progress (Yepes-Rios et al. 2016). This is no less true for EPA-based assessment. Ensuring those tasked with either ad hoc or formal entrustment decisions have potential negative implications clearly described draws the patient into discussion and enables critical decision-making. To address this need, we added this section to the original framework (ten Cate 2013; ten Cate et al. 2015) to make learners, programs, and assessors aware of the implications of entrustment.

This section of the EPA description may also serve to support supervisors who conduct 'Entrustment-based discussions' (EBD) (ten Cate and Hoff 2017). An EBD is a structured conversation with a learner to support an entrustment decision for an EPA, guided by four questions: Does the learner know what to do? Has the learner adequate background knowledge? Is the learner aware of risks and possible complications of the activity? and What would the learner do in case of unusual patients, rare findings, emerging risks or complications? A supervisor conduction an EBD may refer to this section of the EPA description. Risks and adverse event are manifold and unpredictable and can only be described to some extent. But even the simple question “what will you do when you don't know what to do?” may provide insight in the adaptive behavior expected when adverse events happen.

An example of this section, derived from ophthalmology is shown in Box 4.

Most relevant competency domains

Many programs in health professions education (must) use frameworks of competencies that have been created locally or nationally. Examples include the Canadian CanMEDS Physician Competency Framework (Frank et al. 2015), the Accreditation Council for Graduate Medical Education (ACGME) Core Competencies in the USA (Batalden et al. 2002), and the General Medical Council’s Tomorrow's Doctors framework in the UK. These frameworks meticulously describe the skills and abilities required of physicians but have struggled to provide the quality assurance expected from assessment in competency-based design. EPAs do not aim to replace competencies; instead EPAs capture how learners integrate relevant competencies into specific tasks in the healthcare settings. As stipulated earlier, performing an EPA requires the integrated application of relevant competencies (ten Cate 2005).

While EPAs generally draw upon multiple competencies, not all are equally prominent. Sometimes content expertise is indispensable, in other cases communication skills or professional behavior are more important. This section serves to make the connection between the EPA and the relevant competency framework by identifying the most relevant domains or roles. Those domains can subsequently guide the development of curriculum, assessment tools, learning plans, etc. Mapping the EPAs of a program against the competency framework that is being used in its jurisdiction will yield a competencies-EPAs matrix to establish content validity evidence for learning and assessment within the program (see Figure 1 for an example; ten Cate et al. 2015).

Figure 1. EPA – competency matrix.

Required knowledge, skills, attitudes, and experiences

While Sections 2, 3, and 4 can all guide the development of assessment tools, prepare assessors, and inform formal entrustment decision-making, this section is also meant to guide learners toward the expectations that supervisors and programs may hold as criteria for readiness for decreased supervision. Much of knowledge, skills, and attitude will not be explicitly itemized in assessment tools but will still be expected.

Knowledge that is expected to safely perform an EPA may be vast for some EPAs. For example, “Assessing patients with acute medical presentations” could require knowledge from any area in internal medicine (or even beyond). For practical purposes, elaborated descriptions of EPAs should identify knowledge expectations in broad strokes (e.g. listing the most common and critical presentations as key content). For learners, knowledge can be summarized by pointing to a common handbook, or a particular knowledge examination that reflects the knowledge standards for this EPA. Skills, if applicable, are typically drawn from a more focused list. For the purpose of the EPA description, these may be specifically stipulated in the description, identify a relevant comprehensive resource or may reference a particular skills test passed.

While knowledge and skill are mostly EPA-specific, attitude reflected in behavior is more often general, in the realm of professionalism and applicable across many EPAs. In describing attitudes required for an EPA, focus on those that are most relevant to the patient’s experience. For example, altruism is venerated in health professions and often identified as a core value, however, in patient care activities, empathy and compassion are more clearly identified and influential.

For all KSAs the crucial question is: what features must the learners possess before a confident decision can be made to decrease supervision. Knowledge and skill may be examined on planned moments, but attitudes are less easily captured. Several authors have investigated what types of attitudes and behaviors should weigh in when entrustment decisions are made. A recent compilation suggests that five types of general features are critical to establishing trust in clinical learners, inspired by and building on seminal work by Kennedy et al. (2008), summarized in Table 1 (ten Cate and Chen 2020). These expectation can be discussed with trainees at the beginning or throughout rotations, and its acronym reflects A RICH [entrustment decision].

Table 1. General features that enable trust in learners.

Agency	Proactive attitude towards work, team, safety and personal development that includes awareness of and responding to the need for action even when outside of the strict definition of one’s responsibilities and practice of adaptive expertise. Agency can manifest within the context of one or more of the other four factors
Reliability	Consistent, predictable, and conscientious behavior driven by a sense of accountability and responsibility
Integrity	Truthfulness, benevolence, and patient-centeredness, where expertise is employed to benefit patients and decisions are motivated by concern for and made in the best interest of patients
Capability	The ability to perform a specific task in a variety of contexts and within an appropriate time frame, requiring a reasonable understanding and overall view of the clinical situation and ability to communicate and work effectively with others within a system
Humility	Discernment of one’s limitations; willingness and ability to ask for help and feedback; receptivity to insights of patients and co-workers; and ability to learn and develop from mistakes, feedback, and the expertise of others

Experiences can be clinical rotations completed, numbers of procedures practiced under supervision, or other experiences. There has been discussion whether such criteria, if weighed into entrustment decisions, are truly competency-based, but in many cases experiences and numbers of procedures are important (Sedlack 2011). While the reverse reasoning (“if X instances of the procedure have been done, we can just assume without observation the trainee is skilled”) is not valid, high proficiency requires much practice (Ericsson 2015) and quantity and quality of experiences can be reasonably demanded before entrustment is even considered (ten Cate 2015). Another example is having seen a series of pathologies in pediatrics, documented in a portfolio, before an EPA “Serving the outpatient clinic of common pediatric problems” can be entrusted without supervision (Smit et al. 2019).

Information sources to assess progress and support summative entrustment

Entrustment decisions, particularly summative entrustment decision, should be based on multiple sources of information. While ad hoc entrustment decisions by a clinical supervisor (“you go assess and admit this patient; if the patient is stable, we can review the case in the morning”) will usually be less thoughtful, summative entrustment decisions (“from now on you may always see these patients on your own”) have the nature of credentialing – a sufficiently rigorous judgement is therefore expected. Multiple sources of information, not just one observation or test, should support such decisions. Ad hoc as well as summative entrustment decisions do not just report what has been observed (a retrospective focus), but deliberately include a judgment on what level of autonomy the learner is ready for, which has a prospective focus (ten Cate et al. 2020). A suitable phrase for assessment forms would be “Based on my observations, I suggest this trainee is ready for supervision at level x” (ten Cate et al. 2018a).

Workplace-based assessment (WBA), assessment in the natural working environment, has seen many new tools and approaches in the past 20 year. Table 2 summarizes common tools that can be used, grouped by approaches educators and supervisors can do in the clinical workplace (“watching, talking, reviewing results”). A recommended reference for more details is Holmboe et al. (2018). Viewing this through the lens of Miller's Pyramid, lower level sources of information, such as written or online knowledge tests and skills and simulation tests, can also be important, but happen outside the workplace and cannot replace WBA.

Table 2. Assessment tools suitable as sources of information in the workplace to inform EPA entrustment decisions.

Approaches	Methods	Tools
Watching	Brief and focused observation	Mini Clinical Examination exercise (mini-CEX); Direct observation of procedural skills (DOPS) (with or without video recording); in some cases insitu simulation (Patterson et al. 2013) or audio-based evaluation (Sanatani et al. 2020)
	Longitudinal observation	Multisource feedback (MSF) or 360^0 Evaluation
Talking	Brief conversations	Case-based discussions, Chart-Stimulated Recall, One-minute Preceptor, SNAPPS, Entrustment-based discussions
Reviewing results	Product evaluation	Entries in Electronic Health Record; Discharge letters; QI reports; Resident-Sensitive Quality Measures (RSQM)(Schumacher et al. 2018)

A practical question often posed is “How many instances of assessment is enough for a summative entrustment decision?” This section is meant to include not only the types of information sources that can be helpful but also a quantity that should be done. A general answer, applicable for all EPAs cannot be given, so for each EPA a rule may be created. Next, while most sections of the EPA descriptions should apply across institutions, for the quantity of assessment instances, local examination regulations may weigh in. As an example, the EPA-based clerkships of UMC Utrecht's medical school program require that for most nested EPAs students must have two recent satisfactory entrustment-based discussions and two recent satisfactory brief observations. Satisfactory would be that the clinical teacher is happy to state that the student is ready for indirect supervision. If not satisfactory, more observations or discussions are needed (ten Cate et al. 2015). This rule may be different in other schools, and may be totally different from other EPAs in other programs. For example, fellows in gastroenterology need to master colonoscopy. Spier et al found that, while 140 supervised colonoscopies serves as a rule before independence is warranted, in fact 500 were needed before all fellows studied reached a level of 90% success rate (Spier et al. 2010). In some settings (e.g. surgery) observation happens naturally and in other settings it must be planned and requires extra time investment. Seeking a right balance between wishes and feasibility will determine this rule. Technology may help to increase the number of documented observations. Young et al. (date) recently reported how a mobile EPA-app allowed to complete useful assessment in little over one minute.

There are a few important considerations that can help ground decisions on the numbers of assessments targeted for summative entrustment decisions. First, consider the breadth of possible contexts (e.g. patient presentations) for the EPA. You do not want to sample all of these, but you do want to ensure there are sufficient assessments of the EPA to capture a sample of that is representative of the profession or specialty. Second, it is important to have enough assessments that educators can have confidence that the learner is performing the activity consistently at the standard expected. This is not to say variance will not occur, but holistic review of the assessments should capture consistency. Third, the sample of assessments should also capture the different perspectives required to formally entrust the EPA. Including assessments from allied health professions, patients, and/or peers may be appropriate for some EPAs. This should be accounted for in the assessment plan. Fourth, the stakes of the entrustment decision are important. Entrustment decisions in which the learner will remain supervised in some capacity may require fewer observations to ensure safe patient care compared to certification of an EPA for unsupervised practice.

The issue of numbers is most relevant for clinical competency committees (or whichever name these educational teams hold). A valid summative entrustment decision should be grounded in sufficient data, that may be displayed in a trainee's e-portfolio dashboard, reviewed by the committee, and used for an intelligent decision, i.e. a decision that is not just based on an algorithm dictated by the numbers prescribed in this EPA description Section 6. The authors have seen several cases where a “calculation of competence” approach was applied to the detriment of learning and quality of summative entrustment decisions. Numbers of assessments expected should serve as guidelines; entrustment remains a human decision made by experts when, based on their opinion, sufficient evidence supports it.

Entrustment/supervision level expected at which stage of training

Section 7 states for which level the entrustment decision is being made. Five main levels have been described: Level 1: the learner is allowed to be present and observe, not to enact an EPA; Level 2: the learner is allowed to execute the EPA with direct, pro-active supervision, present in the room; Level 3: the learner is allowed to carry out the EPA without a supervisor in the room, but quickly available if needed, i.e. with indirect, reactive, supervision; Level 4: the learner is allowed to work unsupervised; Level 5: the learner is allowed to provide supervision to more junior learners. This framework has been recommended widely (ten Cate 2013; ten Cate et al. 2015). While ad hoc entrustment decisions may distinguish more granular levels of entrustment and supervision, summative decisions usually focus on limited levels: either the move to indirect supervision (level 3), to distant supervision (level 4) or to act as a supervisor (level 5). As is the case with section 6, this section may need to match rules about privileging and credentialing in a specific jurisdiction or hospital so would not necessarily be universal for the EPA.

The essence of this section is to support an individualized, competency-based application of the programs' curriculum. For each EPA in a program, a map can be made identifying when indirect or distant supervision is expected to be effectuated. See Figure 2 (from ten Cate and Billett 2014) for an example. From that protocol, and in consideration of a given trainee’s individual record of learning experiences and rotations, deviations in expected development can be established; this can inform decisions on how to personalize training experiences and support progression towards certification. This individualized approach should also empower trainees who are being regularly assessed to ask for “promotion” when they feel they are ready. Encouraging trainees to request certification at the next level of entrustment-supervision for an EPA, which can be considered and awarded if the committee feels there is sufficient grounding for the trust to do so, promotes self-regulation. This is an adaptation of the initial scheme that can only arise during training.

Figure 2. Decreasing levels of supervision needed, varying by EPA and by individuals.

Time period to expiration if not practiced

This section is less locally determined and more general than 6 and 7 because its significance extends beyond the educational program. Decisions to formally entrust EPAs must follow careful deliberation as these acts bring inherent risk that potentially affect patient safety. Several EPAs require sustained practice to remain safe. The skill to perform can decay if an activity is not practiced or just by prolonged routine practice (Choudhry et al. 2005; Norcini et al. 2017). D'Angelo et al. reported that surgery program directors perceive a marked skill decay among residents after a research fellowship (usually 1–3 years) (D’Angelo et al. 2018). Custers and colleagues found that most of biomedical knowledge, if not rehearsed, is not retained after a few years (Custers 2010; Custers and ten Cate 2011; Weggemans et al. 2017). In other words, a decision to entrust a trainee or graduate with a critical and/or complex EPA, even if well-grounded in satisfactory assessments, may not hold true if the EPA is not practiced. For some EPAs practice needs to continue every week or month to maintain skill, for others a few years of non-practice may not be very critical. There is variation among EPAs, and expiry dates may need to be set for EPAs individually.

What is the significance of an expiry date? A summative entrustment decision is meant to support the future privilege of unsupervised practice, just as a medical license or a specialty board certification does. Delivering a graduate is, somewhat disrespectfully said, like selling a new car. The seller will guarantee the quality of the car up to a number of years or mileage. If well maintained the car can last long, if not maintained, adverse events may happen. So, what should be the consequence of an EPA that needs maintenance but is not practiced for several years? Either the practitioner can decide to leave the EPA outside his or her scope of practice, or, if so desired, he or she can seek supervision, long enough to reestablish trust. The educational institution can set the expiry date but cannot enforce sanctions if the practitioner desires to pick up an activity after a long hibernation. Other regulatory bodies than the educational program may choose to set rules to enforce safe patient care; this section of the EPA description can serve only as a signal.

Conclusion

As health professions move toward implementing competency-based approaches to education, adoption of EPAs into curricula has become increasingly popular. Well-described EPAs are authentic descriptions of the essential work of a profession, providing a natural construct for establishing the required outcome of training. However, EPA titles alone are insufficient to establish these requirements or to direct EPA implementation in education programs. Constructing fully elaborated EPAs creates a shared mental model amongst learners, programs and regulators, informs competency-based curriculum design, directs ad hoc and formal entrustment decision-making and provides standards for certifying bodies and boundaries for scope of practice. This 8-part framework is intended to support curricular leaders looking to adopt new EPAs, or revise and define previously established EPAs for competency-based education.

Acknowledgement

Thanks to Douglas Jones, MD, University of Colorado, for the quote in the introduction section, and to Saskia M Imhof, MD, University Medical Center Utrecht for the catact example.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Notes on contributors

Olle Ten Cate

Olle ten Cate, PhD, is a Professor of medical education and Senior Scientist at the Center for Research and Development of Education at University Medical Center Utrecht, The Netherlands.

David R. Taylor

David R. Taylor, MD, is an Associate Professor, Department of Medicine, Queen’s University School of Medicine, Kingston, Ontario, Canada.