Improvement in National Test arithmetic scores at Key Stage 1: grade inflation or better achievement?

Abstract

The aim of the National Numeracy Strategy is to raise standards in numeracy. Strong evidence for its success has, however, been lacking: most of the available data come from performance on National Test assessments administered in schools or from Ofsted reports, and is vulnerable to suggestions of bias. An opportunistic analysis of data from a population cohort study extending over three school years compares school‐based scores at school entry and at age 7–8 with clinic‐based scores on similar tests. The results show a small but statistically significant rise between 1998 and 1999 and between 1998 and 2000 in scores on both KS1 arithmetic SATs taken in schools and the arithmetic component of the WISC test taken in an independent research clinic. This is evidence for a real rise in generalised arithmetic ability over this period which may be attributable to the children's experience of the National Numeracy Strategy.

Notes

1. Odds ratios are, simply, an indication of how much more likely an outcome (such as a low level in EA or KS1 SATs) is in one group compared with another. Thus, for example, comparing the 1996 cohort's entry assessment scores with the 1997 cohort's scores, an Odds Ratio (OR) of 1.2 suggests that the 1996 cohort are 20% more likely to be doing badly. If the OR had been 1.0, the two cohorts would have been exactly the same. Comparing the entry assessment scores for the 1995 and 1997 cohorts, the OR is 1.03, which is very close to 1; the odds of being a low scorer are almost the same in these two cohorts. The value of an OR, like that of a Mean, is affected by how well the sample used represented the whole population. We need to know how confident we can be that it is the correct value for the population, not just for the sample. Confidence Intervals indicate the range within which the true value is 95% certain to lie. Thus, the OR of 1.2 comparing the entry assessments of the 1996 and 1997 cohorts is within the Confidence Interval (CI) range [1.01, 1.4]; we can be 95% certain, therefore, that the true OR is greater than 1 and the cohorts are different. Comparing the EA scores for the 1995 and 1997 cohorts, the 95% CI is from 0.83 to 1.3, which includes the value of 1 where the two cohorts are exactly the same. This suggests that the 1995 and 1997 cohorts are not really different in their EA scores.