Investigating changes in high-stakes mathematics examinations: a discursive approach

Figures & data

Figure 1. University of London, GCE Ordinary Level, Pure Mathematics (a) Arithmetic and Trigonometry, Summer 1957, question 8.

Figure 2. Oxford and Cambridge Schools Examination Board, GCE Ordinary Level Mathematics Syllabus B, paper 2, Summer 1980, question 6.

Figure 3. London East Anglian Group GCSE, Mathematics A, June 1991, Paper 4, question 10.

Figure 4. Edexcel, GCSE Mathematics (Linear) – 1380, Paper 3, Higher Tier, June 2011, question 10.

Table 1. Years of examinations included in the EDSM data base.

Year	Reasons for choosing this year
1980	As indicated above, we chose this year as a pre-Cockcroft Report base-line GCE ‘O’ level
1987	This was the final examination prior to GCSE. We chose a transitional syllabus in this year: a ‘joint’ GCE/CSE examination.
1991	This was the third year in which GCSE examinations were set. Students taking the examination this year were the first cohort to have been prepared for GCSE through the whole of their secondary school education.
1995	Students taking the examination this year were the second cohort to have followed the National Curriculum throughout their secondary education. A report by the School Curriculum and Assessment Authority following the examination this year called for increased emphasis on algebra and non-calculator question papers (SCAA, 1996).
1999	This year was chosen to reflect changes made to examinations as a result of the SCAA (1996) recommendations.
2004	A report by the Qualification and Curriculum Authority remarked that question papers this year were ‘more accessible linguistically’ than in 1999 and ‘more clearly laid out’ (QCA, 2006, p. 10).
2010	The final year of examinations before the start of the project.
2011	Added to the data base after the start of the project. Informal discussion with an ex-official of the QCA suggested that examination boards were introducing less structured questions in 2010 and 2011.

Table 2. Analytic framework for mathematising aspects of examination discourse.

I. Aspects of the discourse	II. Questions guiding the analysis	III. Textual indicators
Vocabulary and syntax (lexico-grammatical aspects)
A. specialisation	To what extent is specialised mathematical language used?	lexical items used in accordance with mathematical definitions, considered at the level of: vocabulary sentence text unit extra-mathematical context depth of engagement with context
B. objectification of the discourse	To what extent does the discourse speak of properties of objects and relations between them rather than of processes?	nominalisation: use of a ‘grammatical metaphor’, converting a process (verb, e.g. rotate) into an object (noun, e.g. rotation) the use of specialised mathematical nouns such as function, sequence which encapsulate processes into an object complexity of compound nominal groups
C. logical complexity	What kinds of logical relationships are present and how explicit are they?	the types and frequencies of conjunctions, disjunctions, implications, negations and quantifiers
Visual mediators
D. the presence of multiple visual mediators	To what extent does the discourse make use of specialised mathematical modes?	presence of tables, diagrams, algebraic notation, etc.
	How are multiple visual mediators incorporated into the discourse?	provided in the text or to be produced by the student linguistic, visual and/or spatial relationships between modes
E. transitions between visual mediators	What transformations need to be made between different modes?	presence of or demand for two or more modes of communicating ‘equivalent’ information, e.g. an equation formed from a word problem; a unit of text that involves table, graph and algebraic expressions corresponding to the same function
	How are transformations indicated in the discourse?	provided in the text or to be produced by the student explicit linguistic or visual links between modes
Routines
F. the types of action demanded of students	What areas of mathematics are involved?	topics
	What are the characteristics of the routine procedures?	algorithmic or heuristic? complexity explicitly hinted at?
Endorsed narratives
G. the origin of mathematical knowledge	What is the degree of alienation of the discourse?	mathematical objects as agents in processes agency obscured by: non-finite verb forms passive voice
	To what extent is mathematics construed as involving material action or as atemporal objects and their properties?	mathematical objects involved in: material processes relational or existential processes
	To what extent is mathematics presented as a human activity?	human agents in mathematical processes thinking scribbling
H. the status of mathematical knowledge as absolute or contingent	To what extent does the text indicate that decisions or choices are possible during mathematical activity?	modifiers indicating degree of certainty (e.g. may, can, will … ) conditional clauses (e.g. if … or when … ) explicit decisions have been or need to be made

Table 3. Analytic scheme for subjectifying aspects of examination discourse.

I. Aspects of the discourse	II. Questions guiding the analysis	III. Textual indicators
A. student–author relationship	What kind of relationship is constructed between the student and a mathematical community?	use of personal pronouns inclusive or exclusive we other personal pronouns
	Is the student given instructions or invited to consider mathematical questions?	interrogative (questions) imperative (instructions)
B. student autonomy	In responding to an examination question, how many independent decisions is the student allowed/required to make in: designing the path to follow?	the grain size of the task
	interpreting the task?	complexity of utterances lengths of a sentence grammatical complexity: the depth of ‘nesting’ of subordinate clauses and phrases logical complexity
	choosing the form of the ‘answer’?	the layout the physical size of the answer the space provided for the work to be done on the way toward solution format of the answer (units, precision, no. of solutions) modality of the answer (graph? algebraic expression?)
	choosing/constructing the mode of response?	visual mediators: verbal, symbolic, or graphic: supplied or to be produced?

Table 4. Average number of logical connectives per sentence as a function of time.

year	1980	1987	1991	1995	1999	2004	2010	2011
Average number of logical connectives per sentence	0.68	0.48	0.42	0.39	0.37	0.33	0.35	0.34

Notes: The appearance of given or given that used as a connective has been calculated separately and it is not included in the evaluation of logical complexity shown in this table. These terms were not identified as relevant to this analysis until a late stage in the project. Given, used as a connective, appeared extensively in the examination papers from 1980 (23 instances) and 1991 (11 instances), but was found only in three instances in 2011. Its almost complete eradication is consistent with the more general simplification of the complexity of the syntax of recent examination papers evidenced in this table.

Figure 5. Oxford and Cambridge Schools Examination Board, GCE Ordinary Level Mathematics Syllabus B, Summer 1980, paper 1, question 2.

Figure 6. Edexcel GCSE Mathematics Higher Tier Specimen Paper, 1987, paper 3, question 7.

Table 5. Average and maximum number of words in a sentence for two examination papers from 1987 and two from 2011.

Year/examination	1987a	1987b	2011a	2011b
Average number of words in a sentence	13.5	14.5	8.6	8.3
Number of words in the longest sentence	37	42	22	27

Table 6. Compound nominal groups.

	longest nominal groups	no. of words
Figure 1 (1957)	a uniform circular metal disc of diameter 15 in a number of ¼ in. diameter holes	9 7
Figure 2 (1980)	the volume of copper in a piece of the tube one metre long the ratio of the masses of equal volumes of copper and lead	13 12
Figure 3 (1991)	a sector of a circle, centre O, radius 12 cm the radius of the base circle of the lampshade which has the length BC for its diameter	10 17
Figure 4 (2011)	the cross-section of a solid prism the mass of the prism	7 5

Table 7. Solutions of factorisation and equation tasks from examination questions presented in Figures 5 and 6.

Factorisation questions		Equations
Figure 5 a ab^2c^3 + bc^2a^3	choose the highest n so that a^n can be used as a common factor	Figure 5 b	choose a transformation [operation on both sides: x(r+s)]
= a(b^2c^3 + bc^2a^2)	choose the highest n so that b^n can be used as a common factor	r(r+s) =s	choose transformation [expand]
= ab(bc^3 + c^2a^2)	choose the highest n so that c^n can be used as a common factor	r^2 + rs = s	choose transformation [operation on both sides: -rs]
= abc^2(bc + a^2)	3 decisions	r^2 = s – rs	choose transformation [factorise]
Figure 6 b y^2 + 5y	choose the highest n so that y^n can be used as a common factor	r^2 = s(1-r)	choose transformation [operation on both sides: divide by 1-r]
= y(y + 5) -	1 decision		5 decisions
		(Remark: this problem should also include deciding on the numerical constraints imposed by the operation of dividing and concluding that if r=1, there is no solution for s; we doubt, however, whether the examiners expected this addition).
Figure 6 c 2x^2 + 6xy	choose the highest n that can be used as a common factor of 2 & 6	Figure 6 d Option 1 Step 1: decide what needs to be substituted for a, b, and c in the formula ; implement Step 2: calculate the numeric expression obtained (No decision is needed: in algebraic problems such as this one, we are not interested in decisions involved in numerical calculations; thus, the only algebraic step here is the proper substitution of numerical values for the variables a, b, and c). 1 decision Option 2 Step 1: choose two new numbers Step 2: check whether the numbers you choose give -15 when multiplied and give 2 when added. If not, go to step 1. 1 decision
= 2(x^2 + 3xy)	choose the highest n so that x^n can be used as a common factor
= 2x(x + 3y)	2 decisions

Table 8. Distribution of tasks among the different grain sizes in examinations of 1987 and in 2011.

% of tasks with grain size:	1	2	3	4
1987	60.32	23.81	12.70	3.17
2011	60.83	27.66	6.38	2.13

Note: Given that since 2010 there has been pressure on examination boards to decrease structuring of questions, it seems likely that exams from a little earlier than 2011 (1990s and early 2000s) would show more marked lack of tasks with higher grain sizes.

SCAA. (1996). Standards in public examinations 1975 to 1995. London: Author.

 Google Scholar

QCA. (2006). Review of standards in mathematics: GCSE 1999–2004 and A level 1998–2004. London: Author.

 Google Scholar