Teachers’ perceptions and A‐level performance: is there any evidence of systematic bias?

Abstract

Applications for places in UK Higher Education are usually made before the results of A‐level examinations are known, so references from schools and colleges normally refer to expected (or predicted) grades. Inaccuracies in these predictions may be systematically related to key characteristics of the applicant and could lead to under‐representation from various groups of students. This paper examines data on predicted A‐level grades for 415 recently‐enrolled university students. In contrast to the findings of previous studies however, we find that prediction bias is not particularly related to the gender, class or schooling of the student, but is closely linked to the predicted grades themselves—students predicted low grades performed above expectations, and vice‐versa. The implications of this for current UK government initiatives intended to widen participation in Higher Education are considered briefly in the conclusion.

Notes

1. While just 3% of school‐leavers drawn from the lower social classes went into HE in 1950, by 1998 the proportion had risen to 17%—although this was still well below the number (45%) drawn from the three highest social class groupings (Connor et al., 2001, p. 5).

2. As well as school‐leavers from lower socio‐economic backgrounds, targeted groups include people: from neighbourhoods with lower than average HE participation; living in deprived areas; whose family have no prior experience of HE; from ethnic minority groupings under‐represented in HE; who are disabled; and groups that are presently underrepresented—such as women in engineering—in certain subject areas (HECFE, 2004, p. 12).

3. Spence, a state‐school pupil, rejected by Magdalen College Oxford (to read medicine) but accepted by Harvard (to read bio‐chemistry), in 2000 prompted the then UK Chancellor Gordon Brown to describe the Oxford decision as ‘absolutely scandalous’ and precipitated an intense media debate on class inequalities and access to the élite UK universities.

4. Curriculum 2000 introduced AS levels, intermediate examinations at the mid‐point of the A‐level programme, and encouraged students to study a fourth subject for the first year of their A‐level study programme. Preliminary research, however, suggests this curriculum‐widening has, in fact, had little aggregate impact on HE admission decisions (Waring et al., 2003, p. 4).

5. The sector‐led Delivery Partnership—established under the leadership of Universities UK and GuildHE (DfES, 2006a, p. 20)—is charged with overseeing the implementation of the various reforms to the HE applications system proposed following the 2006 Consultation Response (and subsequent HE policy documents).

6. A‐levels are graded from A to E (F is a fail) and a university may make the offer conditional upon the student achieving, say, three B grades (students normally take three A‐levels). Alternatively, an offer may be made conditional on achieving a certain minimum number of points, using a scoring system whereby an A grade is worth 120 points, a B grade is 100 points, C is 80, D is 60 and E is 40. Clearly those institutions and/or courses where demand is high relative to supply will demand higher tariffs from aspiring applicants.

7. Significantly, however, like McManus et al. (1995), the authors did uncover evidence of an ‘ethnic penalty’ among old universities (those incorporated prior to 1992), these universities being more likely to select white candidates from among a group of similarly qualified applicants.

8. One reason for this, as Dhillon herself recognises (2005, p. 78), relates to the admitted cohorts. As selective schools source largely from high‐achievers the scope for overestimating grades is accordingly reduced. Non‐selective schools (including FE colleges) recruit a more heterogeneous cohort of students with a correspondingly greater variance in actual grades.

9. The reference is typically completed by the student’s ‘Director of Studies’ or ‘Tutor’, who will most likely have obtained information about expected grades from relevant subject teachers.

10. From 2007, UCAS applications will normally be required to be submitted electronically and we understand that predicted grades will be stored in a separate field in the database.

11. Where predictions were given as B/C, for example, then the point score was calculated as the mean of a B and a C. Some awards are ‘double awards’ and the point score is doubled. Points achieved at A/S level are excluded in all analyses.

12. ‘Double’ equivalent examinations (such as some AVCE examinations) have been excluded from the analysis.

13. Up to 190 points; 200–230; 240–260; 270–300; 310 and over.

14. While care is required in interpreting these tests since the population distributions may not be normal and we are undertaking multiple‐comparison tests (effectively requiring some increase in the probabilities to take account of this), these tests can be used as an indicative measure.

15. These groups include named intermediate occupations, small employers and own account workers, lower supervisory and technical occupations, semi‐routine and routine occupations and unpaid household workers.