Recounting the Scores: An analysis of the QAA Subject Review Grades 1995–2001

Abstract

In this paper the grades awarded in the subject level review process that ran in England between 1995 and 2001 are examined. These grades have subsequently been used to sustain most published league tables and are often cited in debates about institutional quality. However, the grades were never subject to any moderation, either at the time of the individual visit or subsequently. Accordingly, there are substantial variations between subjects and marked grade inflation over time. This paper considers how to control for these effects and, once this is done, what the impact on the published league tables would be. The difference between the adjusted and unadjusted scores are significant and this further calls into doubt both the validity of the original process and the uses to which the outputs have subsequently been put. This serves as a warning to other quality assurance agencies who have used or are considering using audits based on subject review.

Keywords:

• quality assessment
• subject review
• general linear model