Twelve tips for developing effective marking schemes for constructed-response examination questions

Abstract

Constructed-response questions (CRQs) require effective marking schemes to ensure that the intended learning objectives and/or professional competencies are appropriately addressed, and valid inferences regarding examinee competence are drawn from such assessments. While the educational literature on writing rubrics has proliferated in recent years, this is largely targeted at classroom use and formative purposes. There is comparatively little guidance on how to develop appropriate marking schemes for summative assessment contexts. The different purposes mean that different principles and practices apply to mark schemes for examinations. In this article, we draw on the educational literature as well as our own practical experience of working with medical and health professional educators on their questions and marking schemes to offer 12 key principles or tips for designing and implementing effective marking schemes.

Keywords:

• Constructed-response questions
• levels-based
• marking schemes
• rubrics

Disclosure statement

The authors have no declarations of interest to report.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Notes

1 While these terms are often used interchangeably, as we outline below, we will preferentially use the former in this article.

2 For example, British usage tends to prefer mark/marking scheme, while in the US the term rubric seems to be preferred.

3 As we are providing this exemplar for illustrative purposes only, we have not specified the eye condition.

4 Note that we provide alternative instructional verbs for the question prompt. This is intended to draw attention to the fact that question instructional verbs and marking schemes are closely linked. The decision rests with the question writer as to which of these version (both instructional verb and marking scheme) is most appropriate in the given context, especially in relation to the relevant learning objectives.

Note too that in the example of the levels-based marking scheme provided, the consequence of inappropriate or potentially detrimental suggestions can also be specified. In this case, the marker is prompted to give a lower mark than might otherwise have been given.

5 In the corresponding points-based version, this is only indirectly assessed through the partitioning of marks across the two categories of medical and surgical management options. However, a candidate’s score will not necessarily convey whether this distinction has been appreciated by the candidate.

6 The desirable number of sample responses will depend on many factors, including the number of levels or points allocated for marking, the complexity of the question, the expected extent of variation in the responses, and the level of experience of markers. As a general rule of thumb, however, and where resources permit, we would recommend approximately 10% of the number of responses (randomised).

7 That is, the tendency for assessor judgment of a student’s response to be influenced by the student’s previous responses. See, for example, Dennis (2007).

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Notes on contributors

Neville G. Chiavaroli

Neville Chiavaroli, MEd, Principal Research Fellow, Australian Council for Educational Research (ACER), Australia.

Jacob Pearce

Jacob Pearce, PhD, Principal Research Fellow, Australian Council for Educational Research (ACER), Australia.