Can computer-based assessment enhance the acceptance of formative multiple choice exams? A utility analysis

Abstract

Background Students’ motivation to participate is one of the main challenges in formative assessment. The utility framework identifies potential points of intervention for improving the acceptance of formative assessment [Van Der Vleuten C. 1996. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ 1(1):41–67]. At the Medical Faculty of the University of Cologne, the paper-based version of the Berlin Progress Test has been transformed into computer-based version providing immediate feedback.

Aim To investigate whether the introduction of computer-based assessment (CBA) enhances the acceptance of formative assessment relative to paper-based assessment (PBA).

Methods In a retrospective cohort study (PBA: N = 2597, CBA: N = 2712), students’ overall acceptance of the two forms of assessment was surveyed, their comments were analyzed, and we analyzed their test behavior and categorized students into “serious” or “non-serious” test takers.

Results In the preclinical phase of medical education, no differences were found in overall acceptance of the two forms of assessment (p > 0.05). In the clinical phase, differences in favor of CBA were found in overall acceptance (p < 0.05), the proportion of positive comments (p < 0.001), and the proportion of serious participants (p < 0.001).

Conclusions Introduction of immediate feedback via CBA can enhance the acceptance and therefore the utility of formative assessment.