Exploring features that affect the difficulty and functioning of science exam questions for those with reading difficulties

Abstract

This research explored the measurement characteristics of two science examinations and the potential to use access arrangements data to investigate how students requiring reading support are affected by features of exam questions. For two science examinations, traditional and Rasch analyses provided estimates of difficulty and information on item functioning. For one examination, the performance of students eligible for support from a reader in exams was compared to a ‘norm’ group. For selected items a sample of student responses were analysed. A number of factors potentially making questions easier, more difficult or potentially contributing to problems with item functioning were identified. A number of features that may particularly influence those requiring reading support were also identified.

Keywords:

• examination
• question difficulty
• GCSE
• science
• reading difficulties

Notes

1. Rasch analyses were conducted with a random sample rather than the entire cohort to avoid the occurrence of inflated fit residuals and chi-squared statistics.

2. Item Q7bii had a high positive difficulty estimate, but a plot of item and student locations did not suggest that this item was too difficult for the candidature.

3. In Table 2 it can be noted that Question 5b also has a high fit residual potentially indicating underfit but the Item Characteristic Curve showed an unclear pattern of fit so the item was not investigated further. Additionally, Question 8cii has a high difficulty estimate, but a plot of item and student locations suggested that this item was not too difficult for the candidates.

4. Items Q1ai and Q1aii also have small differences in facility values between groups but will not be considered because the ease of these items has created a ceiling effect.

5. There were a few other items for which the fit residuals suggested possible differences in fit between groups, but these were not confirmed by the Item Characteristic Curves.