Are extended-matching multiple-choice items appropriate for a final test in medical education?

Abstract

At the Faculty of Medicine of the KU Leuven a 'final' test is being developed, part of which consists of extended matching multiple-choice questions (EMQs). These are problem-oriented questions referring to a realistic case, with the correct answer to be chosen from a list of 7-26 possible answers. The same list can be used for several questions (cases). Staff members of 10 medical specialities have developed about 900 questions. In order to test the quality of these questions, they were submitted in series of 100 questions to 251 final-year students who were not specifically prepared for content, or for question format. The organization of the test ran without problems. It was possible to answer 100 EMQs within half a day (i.e. 4 hours). Fatigue had no measurable influence on the score over this period but increasing familiarity with the question format was observed. In order to obtain a reliable series of questions those questions that correlated negatively with the total score were deleted until a Cronbach alpha of 0.80 was reached. Split-half reliability coefficients were of the same order of magnitude. The score on a questionnaire concerning the perception of the test indicated poor face validity. The procedure of test construction gave arguments for content validity. The correlation between test score and total examination score of each of the 7 years of medical education was statistically significant, which indicates criterion validity. Therefore, starting from 100 EMQs per student it was possible to develop a reliable and valid test. EMQs seem to be appropriate for a final test of clinical knowledge in medical education.