Development of a modified Cohen method of standard setting

Figures & data

Figure 1. Example score CDFs.

Figure 2. Effect of applying modified Cohen on failure rates, Years 1 and 2, 2008/2009 and 2009/2010. PM, pass mark.

Table 1.  Variation in failure rates across modules

	Standard deviation of failure rate
Year of course and cohort	Fixed 50% PM	Modified Cohen PM
Year 1 2009/2010	4.95	5.66
Year 1 2008/2009	4.71	3.48
Year 2 2009/2010	3.42	2.35
Year 2 2008/2009	8.28	4.61
Combined	6.63	4.81

Table A1.  Percentiles between which the second-order gradient is a local minimum and the gradient of the CDF between the Cohen PM and the 90th percentile, by module and year

	Module 1	Module 2	Module 3	Module 4	Module 5	Module 6	Module 7	Module 8
Year 1 2008/2009
Percentiles between which second-order derivative of CDF is a local minimum	90–95	90–95	90–95	90–95	90–95	80–90	90–95	80–90
CDF gradient	3.11	3.65	3.24	3.19	3.38	3.45	3.32	3.27
Year 1 2009/2010
Percentiles between which second-order derivative of CDF is a local minimum	90–95	90–95	90–95	95–100	90–95	90–95	95–100	90–95
CDF gradient	2.70	3.42	3.08	2.70	2.93	3.12	3.33	3.46
	Module 9	Module 10	Module 11	Module 12	Module 13	Module 14	Module 15	Module 16
Year 2 2008/2009
Percentiles between which second-order derivative of CDF is a local minimum	90–95	95–100	90–95	90–95	95–100	95–100	95–100	90–100
CDF gradient	3.16	3.14	3.03	3.23	2.97	2.67	3.76	3.37
Year 2 2009/2010
Percentiles between which second order derivative of CDF is a local minimum	90–95	90–95	90–95	90–95	90–95	90–95	95–100	90–95
CDF gradient	3.29	3.51	3.22	3.38	3.15	2.90	3.70	3.53

Note: CDF gradients outside the local ‘threshold’ range are shown in bold.