Feasibility, and validity aspects of Entrustable Professional Activity (EPA)-based assessment in general practice training

Figures & data

Table 1. Quotes from participants

Feasibility
Trainer: ‘Completing an EPA-based assessment requires some mental prework.’
Trainer: ‘Usually you have to do it [EPA-based assessment] fast, otherwise, you forget the details.’
Trainer: ‘Since we did not do any shifts in the past 3–4 months due to the COVID-19 pandemic, I’m basing my assessment on the situation as it was before the pandemic.’
Trainer: ‘I do not think it is up to a PGY-1 trainee to fulfil a “directing doctor”-like function yet, so we did not practice that. I think it is better to learn from doing consultations.’
Trainer: ‘To discuss a patient with the trainee is not to sit next to the trainee for direct supervision, but it is also not the trainee functioning independently, so what independency score should I assign?’
Educational validity
Trainer: ‘The EPA-based assessment did not provide me with new insights.’
Trainer: ‘This [the EPAs] is more specific to a particular situation. Competency assessment is more general and with these EPAs, only the functioning [of the trainee] in the OOH GP Center is evaluated. Because of this, feedback is more specific.’
Trainee: ‘The EPA-based assessment led me to formulate more concrete learning goals that are dynamic within situations. My ComBeL assessment stays the same all the time.’
Trainee: ‘EPAs are more specific, this could be very useful for evaluating difficult cases in the OOH GP Center.’

Table 2. Correlation between independency score for working at OOH GP Center and the EPA scores

EPA	Independency score: sixth month (Correlation coefficient (N = 17)(95% CI)	Independency score: ninth month (Correlation coefficient (N = 17)(95% CI)
1. Determines the condition of patients suffering from acute illnesses, using the ABCDE-system, and performs necessary actions.	0.68 (0.33–1.00)*	0.51 (-0.13–0.89)
2. Handles consultations in order of urgency.	0.18 (-0.65–0.87)	0.66 (0.39–0.88)*
3. Records findings, diagnosis and treatment, and communicates them clearly.	0.54 (0.15–0.84)*	0.64 (0.24–0.95)*
4. Supervises delegated tasks.	0.29 (-0.54–0.82)	0.58 (0.35–0.75)*
5. Reports on occurring incidents and analyses them for improvement of acute care.	0.34 (0.34–0.65)*	0.41 (-0.03–0.72)
6. Monitors the fluent handling of the patient flow with a view to safe, efficient and bridging care.	0.54 (0.16–0.83)*	0.48 (-0.12–1.00)

* CI≠0.00.

Table 3. Correlation between independency scores per EPA and the ComBeL scores for the CanMEDS roles related to the EPA

CanMEDS role	Medical expert (Correlation coefficient (95% CI))	Communicator (Correlation coefficient (95%CI))	Collaborator(Correlation coefficient (95% CI))	Manager (Correlation coefficient (95% CI))	Health advocate (Correlation coefficient (95% CI))	Scholar (Correlation coefficient (95% CI))	Professional (Correlation coefficient (95% CI))
EPA
Sixth month
1. Determines the condition of patients suffering from acute illnesses, using the ABCDE-system, and performs necessary actions. (n = 6)	0.67 (0.20–1.00)*	0.49 (-0.71–1.00)	0.49 (-0.71–1.00)	0.82 (0.43–1.00)*	-	0.83 (0.45–1.00)*	-
2. Handles consultations in order of urgency. (n = 6)	−0.22 (-0.71–3.2)	−0.42 (-0.87–0.28)	-	0.16 (-0.98–0.95)	0.42 (-0.63–0.95)	0.12 (-0.71–0.78)	-
3. Records findings, diagnosis and treatment, and communicates them clearly. (n = 6)	0.32 (0.20–0.71)*	0.14 (-0.71–0.99)	-	0.14 (-1.00–1.00)	-	-	0.42 (-0.76.-0.791)
4. Supervises delegated tasks. (n = 6)	0.00 (-0.71–0.71)	−0.31 (-0.95–0.63)	−0.31 (-0.895–0.50)	-	-	-	0.11 (-1.00–1.00)
5. Reports on occurring incidents and analyses them for improvement of acute care. (n = 6)	0.00 (-0.74–0.78)	0.38 (-0.87–1.00)	0.81 (-0.06–1.00)	-	-	-	0.49 (-0.48–1.00)
6. Monitors the fluent handling of the patient flow with a view to safe, efficient and bridging care. (n = 6)	0.71 (0.45–1.00)*	0.73 (0.45–0.95)*	0.31 (-0.63–0.95)	-	0.11 (-0.89–1.00)	-	-
Ninth month
1. Determines the condition of patients suffering from acute illnesses, using the ABCDE-system, and performs necessary actions. (n = 11)	0.56 (0.08–0.91)*	0.41 (-0.20–0.81)	0.38 (-0.33–0.87)	0.52 (-0.28–1.00)	-	0.41 (-0.66–1.00)	-
2. Handles consultations in order of urgency. (n = 11)	0.28 (-0.76–0.96)	−0.05 (-0.64–0.69)	-	0.20 (-0.86–1.00)	0.51 (-0.61–0.98)	0.11 (-0.41–0.67)
3. Records findings, diagnosis and treatment, and communicates them clearly. (n = 11)	0.57 (-0.23–0.92)	0.23 (-0.61–0.83)	-	0.31 (-0.50–0.81)	-	-	0.16 (-0.64–0.70)
4. Supervises delegated tasks.	0.41 (-0.06–0.76)	−0.24 (-0.63–0.16)	−0.27 (-0.91–0.56)	-	-	-	0.00 (-0.53–0.49)
5. Reports on occurring incidents and analyses them for improvement of acute care. (n = 11)	0.00 (-0.19–0.18)	−0.33 (-0.69–0.17)	−0.39 (-0.77–0.09)	-	-	-	−0.39 (-0.74–0.13)
6. Monitors the fluent handling of the patient flow with a view to safe, efficient and bridging care. (n = 11)	0.34 (-0.26–0.80)	−0.09 (-0.67–0.43)	0.03 (-0.48–0.49)	-	0.07 (-0.46–0.56)	-	-

*CI≠0.00.

Data availability statement

Data are available from the authors upon reasonable request.