Determining validity in national curriculum assessments

Abstract

Background: Validity is a central concern in any assessment, though this has often not been made explicit in the UK assessment context. This article applies current validity theorising, largely derived from American formulations, to national curriculum assessments in England.

Purpose: The aim is to consider validity arguments in relation to the multiple purposes which national curriculum assessments serve. This is because validity is now understood in terms of how the test results are interpreted and used. Multiple purposes mean multiple uses of the results, each requiring justification.

Sources of evidence: A validity framework for national curriculum assessments is presented which is organised around purposes, fitness-for-purpose, reliability and the interpretation and impact of the results. Current tests are evaluated against this framework.

Main argument and conclusions: The analysis finds the national test regime successfully meets some of the common threats to validity. The main concerns are about the multiple high-stakes purposes for which the tests are used and their impact on teaching and learning. Some alternative approaches are then considered, which could reduce the burden of interpretation which the tests currently bear.

Keywords:

• validity
• national curriculum assessment
• reliability
• construct validity
• threats to validity

Notes

1. Cambridge Assessment, one of the major UK exam boards and testing companies is also developing a statement on validity. The House of Commons Select Committee on Testing and Assessment (2008a) began with considerations of validity and used current definitions.

2. The recent House of Commons Select Committee on Testing and Assessment (2008a) used this definition as the basis of its analysis (20).

3. Scotland has never used this system and Wales has recently abandoned national testing and ‘league tables’ of schools. Northern Ireland does not use national tests of the type found in England.

4. A leftover from when they were Standard Assessment Tasks – but the name ran into copyright problems with the American college entrance test – the SAT – and had to be dropped, though the term continues to be widely used by teachers, parents and pupils.

5. The General Certificate of Secondary Education, the examination taken at the end of compulsory education at 16. This is a single subject examination and students would typically take 8–10 subjects. It is graded on an A∗–G scale, with A∗ the highest grade. Government targets are based on the percentage of students in a school gaining five grades A∗–C, including maths and English.

6. I draw heavily on this because the inter-party committee made a sustained enquiry into the evidence on national testing and examinations and presented this in a very accessible manner. The government response in its fifth report (2008b) challenged virtually every major recommendation, though the Ofsted response in the same report undermined many of the government's claims – particularly on the impact on teaching and learning.

7. This is a claim made for the Single Level Tests in England (DCSF 2008), though the reality is that, while teacher judgement is the basis for entering pupils for a particular level, it is the test result not the teacher that determines the level awarded.