Programmatic assessment: Can we provide evidence for saturation of information?

Figures & data

Table

Practice points

No relation is present between the number of data points exceeding the minimum requirements and the consensus between portfolio assessors, suggesting that possibly saturation of information is already attained. Students collect more feedback than they should according to the minimum requirements. Consensus between independent portfolio assessors in programmatic assessment is substantial.

Figure 1. Histogram of the frequency of the number of collected mini-CEXs exceeding the minimum requirements.

Table 1. Overview of the average given grade including the standard deviation (rounded off on 2 decimals) and the number of high-stakes assessments per assessor.

Assessor	Mean grade (SD)	No. of assessments
A	7.52 (0.91)	29
B	7.68 (0.77)	34
C	7.47 (1.19)	15
D	7.55 (0.73)	51
E	7.68 (0.58)	19
F	7.82 (0.64)	33
G	7.57 (0.83)	70
H	7.47 (0.84)	68
I	7.31 (0.78)	67
J	7.55 (0.73)	64
K	7.38 (0.88)	58
L	6.78 (0.67)	9
M	7.27 (0.95)	93
N	7.62 (0.51)	13
O	7.57 (0.86)	30
P	7.35 (0.80)	51

Table 2. Frequency table of the given high-stakes grades by the assessors.

	Grade assessor 2
Grade assessor 1	4	6		7		8		9		10
4	1	0		0		0		0		0
5	0	5		0		0		0		0
6	0	25		3		2		0		0
7	0	3		125		17		2		0
8	0	0		10		127		2		0
9	0	0		2		3		23		1
10	0	0		0		0		1		0