Governance structure and standard setting in educational assessment

ABSTRACT

Seldom have comparative studies of educational assessment systems been undertaken, especially regarding their standard setting procedures. This study examines the effects of governance structures on the power relations in standard setting in the dominant school-leaving or university-entrance examinations. We present acritical analysis of the published research and policy documents, including sense-checking with senior assessment practitioners from 22 jurisdictions. The nature of standard setting systems in three cases of Ireland, the USA and India are described in detail to showcase the differences between the following three models of governance systems: nationalised, commercial market and quasi-market. The contribution of this article, then, is to describe the three models of governance systems, to classify the 22 jurisdictions using the three models, and to generate propositions inductively. Thus, the article provides aconceptual basis for extension of this work to other cases to advance the literature cumulatively by theory-building.

KEYWORDS:

• Governance
• standard setting
• university entrance examinations
• assessment
• comparative assessment

Disclosure statement

No potential conflict of interest was reported by the authors.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

1 In this paper, the term university is used interchangeably with higher education institution and tertiary education is used interchangeably with higher education.

2 Russia has arguably become more centralized since this these reviews were undertaken.

3 The SAT (formerly the Scholastic Aptitude Test, now an ‘empty’ acronym) and ACT (originally American College Testing) tests are alternative selection tests that are themselves not closely aligned. Both the SATs and ACTs use familiar psychometric and judgemental methods to put scores on a common scale in order to provide comparable year on year and test form to test form comparisons. Scores on the SAT range from 400 to 1600, combining test results from two 800-point sections: mathematics, and critical reading and writing. Both the points and the sections have seen changes over time, for example, there has been a conscious attempt to align the tests more closely with typical school curricula.

4 https://www.theguardian.com/education/2015/aug/09/exam-boards-students-a-level-results-nick-gibb-reform.

5 https://www.theguardian.com/education/2013/feb/07/gove-gcse-reforms-from-2015.

6 https://schoolsweek.co.uk/hinds-im-happy-with-system-of-multiple-exam-boards/.

7 https://feweek.co.uk/2018/07/02/if-a-levels-need-multiple-exam-boards-why-dont-t-levels/.

Additional information

Notes on contributors

Dennis Opposs

Dennis Opposs works at the Office of Qualifications and Examinations Regulation (Ofqual). Ofqual is the examinations regulator in England, and his current post is Standards Chair. He acts as an in-house expert on qualifications, standards and assessments. His present work focuses on standards and standard setting in GCSEs and A levels. In his early career, Dennis taught chemistry and other sciences in comprehensive schools in London and Hertfordshire. He worked in regulatory organizations in the 1990s including for the Qualifications and Curriculum Authority (QCA). His early responsibilities included GCSEs and A levels in science subjects and the initial development of National Curriculum science tests. In the last decade, Dennis has headed up Ofqual's Standards and Research Directorate and been in charge of the organization's reliability programme. He also led the development of standard setting of reformed GCSEs using a new grading scale and directed work on the comparability of standards between subjects. He is presently the representative for the Europe region on the Board of Trustees of the International Association for Educational Assessment. (IAEA).

Jo-Anne Baird

Jo-Anne Baird is Director of the Department of Education, University of Oxford. Before moving to Oxford, Jo-Anne held academic posts at the Institute of Education, University of London and the University of Bristol. Jo-Anne previously held the position of Head of Research at the Assessment and Qualifications Alliance, where she managed the research programme and was responsible for the standard-setting systems for public examinations. Her first degree and doctorate were in psychology and she has an MBA. Her research interests are in educational assessment, including system-wide structures and processes, examination standards, marking and assessment design. Jo-Anne conducts a lot of work with government and industry partners, including acting as the Standing Adviser to the House of Commons Education Select Committee, a member of Ofqual's Standing Advisory Group and membership of the Welsh Government's Curriculum and Assessment Group. She is a member of the Editorial Board of the Oxford Review of Education journal and the International Advisory Board of Assessment in Education: principles, policy & practice. She has been a Visiting Professor at the universities of Bergen, Queen#x27;s (Belfast) and Umea. From 2013 to 2015 she was President of the Association for Educational Assessment - Europe.

Maia Chankseliani

Maia Chankseliani is Associate Professor of Comparative and International Education and International Education and course leader for MSc in Comparative and International Education at the Department of Education, University of Oxford. Maia's research in comparative and international education examines participation in and outcomes of higher education and VET/apprenticeships. She draws primarily on the scholarship in social and political theory and also employs ideas from history, economics, geography, and demography to understand of how macro- and meso-level factors link with individual opportunities of accessing and benefitting from post-compulsory education at home and abroad. Methodologically, she moves between qualitative and quantitative methods. She often complements secondary data analysis with narratives from semi-structured interviews and contextualised case studies.

Gordon Stobart

Gordon Stobart is Emeritus Professor of Education at the Institute of Education, University College London and a Senior Research Fellow at the Oxford University Centre for Educational Assessment. He has worked in education as a teacher, psychologist, policy researcher and academic. His expertise is in assessment, with much of his recent work focusing on Assessment for Learning.

Amit Kaushik

Amit Kaushik is CEO at the Australian Council for Educational Research (India). He is a former civil servant and development professional with experience at policy and implementation levels. He has had multiple roles over the last thirty years, including in business, with an NGO, and international agencies such as UNESCO and UNICEF.

Hugh McManus

Hugh McManus is Assistant Head of Examinations and Assessment at the State Examinations Commission, Ireland, and has responsibility for the Commission's research unit. Along with other management colleagues, he oversees the standard setting and other work of the subject specialists responsible for the Commission's various examinations. He represents the Commission on the Transitions Research Group - an inter-agency working group overseeing research related to the Irish government's Transitions Reform agenda. These reforms are intended to improve the transition from second to third level education in Ireland. He also represents the Commission in its membership of Association for Educational Assessment - Europe and the International Association for Educational Assessment. Prior to his current appointment, Hugh managed all Leaving Certificate mathematics examinations. In this capacity, he managed the examination changes arising from the government's Project Maths curricular reform initiative. Before joining the Commission in 2003, he spent five years as a post-primary schools inspector, including a brief secondment to the World Bank as an Education Specialist, and eight years as a mathematics teacher in a large secondary school.

David Johnson

David Johnson is Reader in Comparative and International Education. He convenes the Centre for Comparative and International Education at the Department of Education, University of Oxford. He is the Programme Research Lead for the ESRC-DFID Raising Learning Outcomes in Education Systems Research Programme and Academic Lead for the Aga Khan Foundation funded Research Programme on Education and Uncertainty. David is a Chartered Educational Psychologist who studies learning and cognition and is particularly interested in tracking learning progression in national systems of education over time. Recent work includes time series studies of learning in Sierra Leone, Nigeria and Sudan.