Acceptability of the extended matching question format as formative assessment in online continuing professional development

Dear Sir

Online learning resources allow users to undergo formative self-assessment of their applied knowledge and problem-solving skills (Kibble et al. 2011). Most online learning resources use true-false or single best answer questions as their formative assessments. Extended matching questions are rare if ever used in such resources and to our knowledge there are no reports in the literature as to their acceptability amongst learners.

BMJ Learning is the online learning website of the BMJ Group. Since 2006 the National Institute of Clinical Excellence (NICE) and BMJ Learning have published a series of online learning modules designed to help health professionals implement NICE guidelines. In 2011, we published an online learning resource on feverish illness in childhood that had extended matching questions as its end-of-resource formative assessment. We evaluated the acceptability of this question type by looking at how popular the module was, by examining the feedback that we received on the module and by looking at what percentage of users who started the module completed it. We analysed the feedback from a qualitative perspective.

A total of 1248 users completed the module in the seven months following its publication. This made it the 25th most popular module on BMJ Learning during this period (there were 1118 modules on the site). GPs and GP trainees were the biggest single user group who completed the module (36% of users were in this category). Hospital doctors (both junior and senior) also completed the module (26% of users were in this category). The remaining users were made up of nurses, allied healthcare professionals and students.

Four hundred and twenty four users left feedback on the resource. Feedback to the resource was positive. 186 users described the resource as useful or practical or helpful. 146 users described the resource as good and 59 described it as excellent. 30 users described it as informative. Three users left negative comments. One user said that the new module type did not work on a Mac; one user commented that there was a problem with the drop down boxes on the questions; and one user felt that an answer given was marked incorrectly. There may have been an element of responder bias in the feedback given. 34% of users left feedback and we do not know how those who did not leave feedback felt about the resource. It is possible that those who left feedback were more likely to be those who felt positive about the resource.

Seventy three per cent of those who started this module completed it. The average completion rate for modules of a similar type but which contain single best answer questions is 74%.

Overall we feel that the high usage (the module was amongst the 3% of most used modules on the site) and positive feedback demonstrates that the extended matching question format is acceptable to a wide range of users of online CPD.