Do entrustment scales make a difference in the inter-rater reliability of the workplace-based assessment?

Figures & data

Table 1. Entrustment scale [11]

Level	Descriptor
1	Requires complete guidance; ‘I had to do.’ i.e., The learner requires complete hands-on guidance, ‘did not do’ or ‘was not allowed to do.’
2	Able to perform with repeated direction; ‘I had to talk them through.’ i.e., The learner can perform tasks but requires constant direction.
3	Some independence but intermittent prompting required; ‘I had to direct them from time to time.’ i.e., The learner demonstrates some independence but requires intermittent direction.
4	Independent for most things but requires assistance for nuances; ‘I had to be there just in case.’ i.e., The learner showed independence but was unaware of risks and still requires supervision for safe practice.
5	Complete independence: ‘I did not need to be there.’ i.e., complete independence understands risks and performs safely, practice-ready.

Table 2. Performance descriptors

Rating	Descriptor
0	Demonstrates little knowledge and lacks the ability to evaluate issues resulting in only a rudimentary contribution to the management plan
1	Demonstrates some knowledge and limited evaluation of issues resulting in a limited management plan
2	Demonstrates satisfactory knowledge and logical evaluation of issues resulting in an acceptable management plan consistent with higher training

Table 3. Learners’ grades using different workplace-based assessment (WBA) tools

Assessment (n. of assessment point)	Min%–Max%	Mean of percent scores ± SD
PBA (1)	50.10–90	69.92 ± 10.20
PBA (2)	55–95	73.14 ± 9.94
DOPS (1)	52–91	70.11 ± 10.44
DOPS (2)	53–95	73.27 ± 9.90
mini-CEX (1)	50–92	70.26 ± 10.04
mini-CEX (2)	52–96	73.68 ± 9.61
CBD (1)	45–84	57.97 ± 12.59
CBD (2)	46–95	83.51 ± 11.99
	Min% – Max%	Mean of percent scores ± SD	** Median of percent scores
CBD* (1)	15–87	68.66 ± 17.76	57
CBD* (2)	26–99	84.72 ± 17.10	69.5

*CBD grades of students without using any rating rubrics (before undergoing the staff training)

**CBD data were not normally distributed.

Gofton WT, Dudek NL, Wood TJ, et al. The Ottawa surgical competency operating room evaluation (O-SCORE): a tool to assess surgical competence. Acad Med. 2012;87(10):1401–1407.

 PubMed Web of Science ®Google Scholar