Understanding achievement differences between schools in Ireland – can existing data-sets help?

Abstract

Recent years have seen an increased focus on school accountability in Ireland and calls for greater use to be made of student achievement data for monitoring student outcomes. In this paper, it is argued that existing data-sets in Ireland offer limited potential for the value-added modelling approaches used for accountability purposes in many other countries. Some reasons for this include: few longitudinal studies of student achievement, few population data-sets and limited data on student home background in the State examination data-sets. A cross-sectional contextualised attainment model (CAM) is presented using Junior Certificate data from 2009. While the model offers some potential for identifying schools performing significantly above or below average, its explanatory power is limited by the availability of appropriate student background data. It is proposed that the use of statistical models for the purposes of identifying high or low performing schools in Ireland would need to take account of findings from other countries regarding technical difficulties such as variable selection, missing data, and the need to avoid aggregated data. Other practical concerns discussed include the appropriateness of publishing league tables and the usefulness of such information for informing parental choice.

Keywords:

• junior certificate
• school effectiveness
• accountability

Acknowledgements

The author gratefully acknowledges the detailed comments on drafts of this article from Dr Peter Archer and Dr Jude Cosgrove of the Educational Research Centre, St Patrick's College, and those of anonymous reviewers. The author would also like to thank Hilary Walshe and Eileen Corbett from the Educational Research Centre for assistance in sourcing reference material.

Notes on contributor

Dr Lorraine Gilleece is a Lecturer in Education at Liverpool Hope University and a Research Associate at the Educational Research Centre, St Patrick's College, Dublin. Much of her research centres around the analysis of large-scale educational assessment data from national and international assessments at primary and secondary levels.

Notes

1. The Junior Certificate is currently undergoing substantial revision whereby it is envisaged that students will take a reduced number of examinations (see http://ncca.ie/framework/doc/NCCA-Junior-Cycle.pdf, last verified 19 January 2012).

2. Readers less familiar with the Irish system may be interested in the following detail on the types of second-level school in Ireland. Voluntary secondary schools are owned by religious groups or organisations or by their trustees and are often single-sex schools. Vocational schools are owned and run by Vocational Educational Committees and are usually co-educational. Community or comprehensive schools are established by the state and managed by boards of partners and trustees. All school types offer a similar education (McCarthy et al. 2013).

3. According to Cohen (1977), values of Cohen's d less than 0.2 are considered to represent a small effect; values from 0.20 to 0.50 represent ‘small’ effects but are worth noting; values from 0.51 to 0.80 represent medium effects; and, values above 0.80 represent strong effects.