Metaphors and the psychometric paradigm

ABSTRACT

This paper analyses the psychometric paradigm for understanding educational measurement and standard setting by considering the extent to which that understanding is based on metaphors. I argue that through the metaphors ‘more is up’ and ‘good is up’ we spatialise concepts that are not intrinsically spatial. Human abilities form a distinct conceptual category, with complex and varying rules for the correct usage of different ability terms. The fact that most abilities can be discussed in terms of both ‘possession’ and of ‘more or less’ invites reification, spatialisation, and hence the application of measurement metaphors. Insights into the measurement of abilities can be gained by careful consideration of how ability terms are used in normal discourse, especially of athletic abilities which are partly quantified in terms of attributes that are physically measureable. The fuzziness of natural language creates some of the problems in understanding standards in both psychometrics and physical measurement.

KEYWORDS:

• Psychometrics
• standard-setting
• measurement
• metaphors

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. In this paper ‘E&P attributes’ simply means whatever is the focus of educational and psychological tests and assessments, for example ‘mathematics achievement’ (educational) or ‘self-esteem’ (psychological). The term ‘construct’ is frequently seen instead of attribute but here ‘attribute’ is used throughout for consistency, while acknowledging a reviewer’s point that both terms have some different nuances in the philosophical literature.

2. In their afterword to the 2003 re-print of the book they acknowledge that this 3-way distinction is arbitrary inasmuch as all metaphors can be viewed as structural and ontological.

3. But perhaps a majority of practitioners and the general public?

4. Cheating (e.g. by taking performance-enhancing drugs in athletics) could be seen as ‘choosing to exceed your ability’ but arguably the reason cheating is deemed unacceptable is that it removes the grounds for ascribing ability on the basis of performance. This kind of ‘defeasibility’ is noted by Maraun (1998) as a grammatical characteristic of everyday psychological concepts that may be incompatible with measurement.

5. Actually minimum, in the case of running times, which would seem to be a counter-example to Lakoff & Johnson’s ‘up is better’ orientational metaphor. However, time is inversely proportional to speed and races are about identifying the fastest (Faster, Higher, Stronger being the Olympic motto). So the ‘up is better’ metaphor does apply to the speed of the runners.

6. This standard could be defined in several ways – the average of a population of jumpers, or the average of a population of different bar heights being (analogously) the most common ways of defining the zero point of the scale in psychometric models.

7. An extreme example of the ‘Rasch paradox’: “Those who think error will improve observation are unlikely to know when the wool is pulled over their eyes.” (Michell, 2008c, p. 122).

8. 16 degrees Celsius in the UK according to http://www.hse.gov.uk/temperature/law.htm .

9. His example uses ‘short’ instead of ‘tall’.

10. https://en.wikipedia.org/wiki/Obesity.

Additional information

Notes on contributors

Tom Bramley

Tom Bramley is the director of Cambridge Assessment’s Research Division. His main areas of interest are Rasch measurement, and the psychometric issues associated with assessment validity, reliability, comparability, standard setting and standard maintaining.