Making the certainty based marking pilot study a reality

Dear editor

We thank the authors of “Use of certainty-based marking in a second-year medical student cohort: a pilot study”¹ for their pilot study, we agree this is an interesting area that warrants further investigation. Schoendorfer and Emmett¹ aimed to assess student opinion of certainty-based marking (CBM) as a method of improving current curricula and explore the effectiveness of its implementation. If effective, the researchers believe CBM may increase the amount of certainty by those in professional health care roles.

CBM is an examination format that has been around for a number of years.^1,2 It aims to encourage students to feel confident in the answers they write in assessments; students are asked to rank their confidence in an answer, and a more confident answer will yield higher marks if correct and a lesser mark if incorrect. This has been seen as more effective than simple negative marking techniques that don’t offer reward for certainty in response.

Schoendorfer and Emmett’s study method was to change the weekly formative questions that students are provided to CBM format, and then provide an optional questionnaire to the students.¹ This included qualitative response collection as students were asked to provide opinions. This method increased the risk of bias in the study however, as it may be more likely that the academic students participate in both the formative exam and in the survey response – this is likely to give a poor representation of the year group as a whole. Whilst, qualitative responses do provide better insight to student opinion, most were short answers and no in-depth opinions were obtained.

The study was carried out over a 4 week period and students were only tested on one area of knowledge (the nervous system).¹ A longer trial period would have increased the reliability of the results, we acknowledge that this is a pilot study which may have restricted the research period.

Results show an uptake of ranging from 8% to 15% over the 4 week period from a cohort of students of 400.¹ Presenting the figures in percentages could be interpreted as slight manipulation of data, a potential route for bias. This uptake was much lower than the previous 4 week period ranging from 40% to 46%, perhaps indicating that a large quantity of students felt that CBM would not be of any benefit and this could have been represented in the number of students who felt it was a “waste of time”.

We appreciate this was a pilot study, should a larger study be repeated, a much larger sample size ranging across a number of different universities would be more beneficial.Different universities have differing learning opportunities and the institutions that do not provide regular examinations may find CBM style questions a more appropriate choice. If possible, a randomized selection of students from year groups should be selected as this would reduce the potential for bias use of the material. Furthermore, the questionnaire should include a wider range of responses for student feedback, including the options to strongly agree or disagree, and should also have a neutral opinion. This would provide a more in-depth analysis on the strength of student opinion on CBM. Moreover, the inclusion of anonymous attainment information would be useful as it would give an indication as to whether CBM is more effective at revealing lack of knowledge or is simply more difficult than standard assessment.

Disclosure

The authors report no conflicts of interest in this communication.