Using seen examination questions in Economics: an evaluation

Abstract

This paper presents a detailed quantitative analysis of the use of seen questions within examinations in Economics option modules at one UK university. 4622 marks for seen and unseen questions awarded over a period of three years were analysed; the impact of personal characteristics was analysed using a sub-sample. It was found that the number of attempted answers to the seen question was somewhat lower than might be expected if students had engaged in random selection of examination questions and the proportion of students who attempted to answer the seen question varied both across modules and years. The majority of students who answered the seen question did so in only one module, and answering the seen question in more than one module did not have a positive impact on obtaining a good mark for the seen question. Most notably students who answered seen questions were more likely to obtain a good mark for seen than for unseen questions. The personal factors found to correlate with the likelihood of answering a seen question were gender, performance in Economics core modules, and type of school attended. Reasons for the findings are discussed and set within the context of innovation in Economics assessment.

Keywords:

• economics
• assessment
• student-centred learning

Notes

1. With random selection we would expect the SCL question to be attempted by around 38% of candidates.

2. We have observations for a total of three years and three cohorts of students. Hence, in each case two dummy variables are included in the regression equation.

3. The Jarque-Bera test for normality of the distribution of the regression disturbances produces a chi-squared statistic (with 2 degrees of freedom) of 378.6, which is highly significant.

4. An ordered logit specification was estimated as an alternative. In each case the results were essentially the same.

5. The coefficient for the slope dummy indicating a candidate who answered the student-centred learning question in more than one examination has a value of 0.0003, with a z-statistic of 0.003.