176
Views
12
CrossRef citations to date
0
Altmetric
Original Research

Medical students’ assessment preferences at King Faisal University, Saudi Arabia

, &
Pages 95-103 | Published online: 10 Mar 2011

Abstract

Objective

To assess the preferred methods for assessment among medical students at both preclinical and clinical stages of medical education and the possible correlates that promote these preferences.

Subjects and methods

All medical students from the third year onwards were surveyed. A self-administered anonymous questionnaire was designed to gather information on the preferred assessment method for course achievement. The preferred methods were also evaluated in relation to cognitive functions. Preference for specific exam format, in the form of multiple choices, short essay questions, or both, and the stated reasons for that preference, was also included in the questionnaire.

Results

Out of 310 questionnaires distributed, 238 were returned. Written tests, projects, portfolios, and take home exams were the preferred modes for assessing students’ achievements in a course; oral tests including a viva voce were the least preferred type of assessment. Questions that tested the domains of ‘understanding’ and ‘application’ were the most preferred type while those entailing ‘analysis’ were the least preferred. Multiple choice question format was the most preferred type of question (68.7%) at both pre- and clinical stages.

Conclusion

Students’ assessments at the College of Medicine, King Faisal University, Saudi Arabia, do not use the full range of cognitive domains. The emphasis on higher domains for medical students’ assessment incorporating critical thinking should increase as the students’ progress through their medical courses.

Introduction

Assessment is considered to be the most important factor that drives student learning.Citation1 Students tend to learn mainly the material on which they are to be assessed. Research shows that the type of assessment method adopted can influence student learning in any higher education context, including medical education.Citation2,Citation3 The objective of this study was to assess the preferred methods of assessment in medical courses at the College of Medicine, King Faisal University, Saudi Arabia from students’ perspectives and to define the possible correlates that promote these preferences.

Setting and design

This was a cross-sectional survey carried out at the College of Medicine, King Faisal University, which is located in the Eastern Province of Saudi Arabia. The college was established in 2003, with an undergraduate medical program. The college started to enroll female students from 2005, and adopts traditional forms of teaching in the major part of the courses delivered, English language being the primary medium for instruction. The program is divided into a preclinical stage; including the first 3 years and the first semester of the fourth year, followed by a clinical stage including rotations in different clinical departments which lasts for a period of 2 years. The students are also exposed to an integrated module based on problem-based learning (PBL) just before they begin their clinical postings. This lasts 6 weeks and covers all major organ systems, integrating basic and clinical science applications. The main modes of assessment used in our college include written tests, structured essays, short answer questions, and multiple choice questions (MCQs); other modes include objective structured clinical examination, clinical mini-exams with viva, student presentations/symposiums, projects, assignments, and log-books. The total number of students enrolled at the time of the study was 752 (247 females, only up to fourth year).

Methods

Participants

All medical students from the third year onwards were surveyed. Second-year students were excluded due to less exposure to the diverse assessment methods used during their courses.

Measures

A self-administered anonymous pre-tested questionnaire was designed to gather information on:

  1. Assessment preferences: Adopted from the Assessment Preferences Inventory (API),Citation4 originally composed of 67 items, essentially a Likert-type questionnaire designed to measure 7 dimensions of assessment. To fulfill our objectives, while considering the medical students’ environment, 2 dimensions of the questionnaire were employed, namely, preferences of assessment type and preferences of cognitive processes using a 5-point Likert scale (1 = not at all to 5 = to a great extent). The “assessment types” domain was originally composed of 12 questions about students’ preferences for different modes of oral, written, and alternative tests. Cognitive processes comprised 15 questions about the preferences for assessing the cognitive processes (remembering, understanding, applying, analyzing, evaluating, and creating).

  2. The exam format: Preferred type of questions employed in various assessment methods. The most commonly used types used in our college include MCQs and short essay questions. The reasons for the relative preference of the two formats from the students’ perspectives were assessed through closed ended questions: for the MCQs preference, 15 inquiries were included, derived from the available literature,Citation5Citation8 on the reasons for preference for such types of questions. For the short essay questions another 5 options were also included to assess the reasons for their preference among the survey students.

  3. Personal data were also collected including age, type of secondary education, grade point average (GPA) for the previous year, and gender.

Questionnaire administration

All medical students were given an orientation session on the objective of the survey as well as the contents and the filling-out of the data collection form. Data were collected at the end of the mid-term examinations conducted in February. Data confidentiality and the right not to participate were maintained throughout.

Reliability analysis

The preference types measure of 12 items revealed an alpha reliability coefficient of 0.73. For the preferred domains of assessment (including the 15 items) alpha was 0.74, and for the reasons for preference of MCQs it was 0.69 (15 items). Alpha was 0.68 for the 5 items related to preference of the short essay questions.

Data analysis

Data were analyzed using SPSS software version 16.0; SPSS Inc., Chicago, IL). For categorical data, proportions and percent were used for expression, and Chi-square, Z statistical tests were used for comparison. Continuous data including different scores on the Likert scale, and median, mean, and standard deviation were used for reporting, and the Mann–Whitney test was used to compare different groups. Spearman correlation coefficient was employed to assess the relationship between dependent variables (preferred types, cognitive processes related to assessment, and preferred question formats) against the independent variables year of enrollment, gender, and GPA of the students.

Results

Distribution of medical students in relation to gender and years of enrollment

The study aimed to cover students in both preclinical and clinical stages. Third- and fourth-year (preclinical) students included both males and females, while fifth- and sixth-year students included only males (female students were enrolled into the College of Medicine It was only 4 years ago that, so all the senior female students at the time of this study were in their fourth year of study). Out of a total of 310 questionnaires distributed, 238 were returned. There was no significant difference in age among preclinical students who returned questionnaires (P = 0.54). Stated GPA was significantly higher among females (3.30 ± 0.57 vs 3.16 ± 0.39 among males; P = 0.034). Response rate was 67.6%, and was higher among males than females (65.9% vs 56.0%). Response rate was higher at the clinical stage than at the preclinical stage (81.1% vs 61.4%) ().

Table 1 Distribution of medical students in relation to gender and years of enrollment, College of Medicine, King Faisal University

Preferred methods of assessment for medical course achievement as stated by medical students

Taking into account all the responses, irrespective of stage of learning (preclinical or clinical), written tests without supporting material, with or without time-limit, ranked the highest among the preferred methods of assessment (written tests with time limits and without supporting materials – median score 3.3, rank 1; written tests without time limits and without supporting materials – median score 3.0, rank 2). Alternative assessment methods such as student projects/papers and student portfolios came next in the order of preference. Among the conventional methods, oral tests were the least preferred while computerized tests were least preferred among alternative assessment methods ().

Table 2 Preferred methods of assessment for medical course achievement as stated by medical students in relation to stage of study

Preferred methods of assessment for medical course achievement as stated by medical students in relation to stage of study

Comparing the assessment preferences based on stage of study, preclinical vs clinical, showed the following significant findings: written tests were among the top methods preferred by both groups (). The preclinical group preferred written tests without supporting material and without a time limit, while the clinical group preferred written tests with supporting material. Student projects/papers and portfolio were quite popular in both groups while computerized assessment was equally unpopular in both groups. While oral tests were generally not preferred by both groups, oral examinations in the form of observed group discussion had a very high ranking in the clinical group compared with the preclinical group ().

Table 3 Preferred questions formats in written exams as stated by medical students in relation to years of enrollment (N = 238)

Preferred aspects of cognitive processes assessed in using evaluations methods

Assessments based on the domain of recall related to assignments given to the students were highest in the order of preference. Assessments based on the domains of “Understanding” (especially comprehension questions related to the material taught by the instructor) and “Application” (specifically questions requiring the application of material learnt during the course to new situations) were next in the order of preference. While assessment methods falling under the domains of “Analysis”, “Evaluation”, and “Creation” were in general not preferred much, specific questions requiring scientific investigation were a popular preference. Questions relating to comparison of concepts and ideas (under the domain of “Understanding”) were the lowest in the order of preference ().

Table 4 Preferred domains of assessment using evaluations methods for medical courses as stated by medical students in relation to their stage of education

Preferred aspects of cognitive processes assessed in evaluations methods (in relation to their stage of education)

Those in the clinical stage showed a higher preference for questions that required critical thinking (in the higher domains of analyze/evaluate/create) compared with the preclinical students (mean score ± SD of 3.32 ± 1.21 in the clinical group vs 2.38 ± 1.23 in the preclinical group; P = 0.002). Other questions related to higher domains such as questions requiring personal explanations/opinions and questions requiring scientific investigation were more preferred among the clinical group compared with the preclinical group. Questions pertaining to the lower domain of recall were less popular among the clinical group. Assessment involving questions based on comparison/overall relationships between topics and concepts were unpopular among both groups ().

Preferred questions formats as stated by medical students in relation to years of enrollment and reasons for preference

Multiple choice format was the most preferred type (in 214/238, 68.7%) at both pre- and clinical stage. The main reason for stated preference was that both the question and answer being available tends to stimulate memory, thereby increasing chances for a correct answer in 151/214 (70.6%). The other factor mentioned was that there was at least a 25% chance of getting a right answer (in 80/214), even if the question was not familiar (a 5-choice format is used in our college). Essays were preferred less than MCQs and the important stated reasons included: “The essay does not judge knowledge as it is more about how well you can write”, in 155/214 (72.4%) and “I dislike structuring and writing essays and would prefer to have the answer to a question there in front of me somewhere” in 166/214 (77.6%) ().

Table 5 Stated reasons for MCQs preference among medical students in relation to stage of education, King Faisal University

Of those who stated their preference for essay questions (n = 85), 83.5% mentioned that the main positive aspect of the essay question format is that it gives time to really think critically and effectively about the course ().

Table 6 Stated reasons for preference of essay questions by the medical students included at King Faisal University (N = 85)

Females at the preclinical stage demonstrated a higher preference for essay questions than their male counterpart: 31/61 females (50.8%) preferred essays or combined MCQs and essay questions compared with only 14/87 males (16.1%, P = 0.006). Correlation coefficients showed that question format in the form of MCQs vs essay or both was significantly correlated with the year of enrollment of medical students (r = −0.332, P = 0.001), whereas those in the early years of medical education preferred the MCQs format. GPA and female gender of the students were positively correlated with the preference for essay questions format (r = 0.238, P = 0.002 and r = 0.216, P = 0.012, respectively).

Discussion

The investigation of students’ instruction and assessment preferences is valuable for understanding factors that drive the learning process and its outcomes. Previous research has shown that differences in students’ learning and testing preferences lead to performance differences.Citation9 Students’ preferences for assessment tend to reflect their perception of the learning environment, their learning concepts, and their approaches to learning, which consequently affect their achievement.Citation10,Citation11

Medical education is also driven by assessment and hence it is important to consider the students’ assessment preferences and how they affect student achievement. Preferences of students in higher education have been quite well studied, but there is a paucity of studies specifically related to assessment preferences of medical students.Citation10,Citation11 Our results therefore are discussed mainly in comparison with other studies in the context of higher education. Our study attempted to cover all forms of assessment – both formative and summative – conducted in our institution. The importance of formative assessment in medical education cannot be overemphasized.Citation3

Our study mainly attempted to answer the following questions

  1. What assessment methods do medical students in our college prefer and why?

  2. Is there a difference in the assessment method preference based on the stage of learning – clinical versus nonclinical, and based on other factors such as gender?

  3. Which cognitive processes do the students prefer for their evaluation?

Taking the group as a whole, the majority of the students preferred three types of assessment methods: written tests (with or without supporting material) and mainly formative assessment methods such as portfolios and projects. Interestingly, in our study, the students in the clinical stages preferred written tests with supporting materials such as reference texts compared with the preclinical students, of whom a majority preferred written tests without supporting material. Previous studies have also shown a similar preference for written tests, projects, and portfolios.Citation12 The preference for assessment formats with the use of supporting material has been show in previous studies in which students preferred easy-to-take and stress-reducing assessment formats.Citation13 We believe that as medical students enter the clinical stages they tend to have a higher learning load and hence more stress, which might explain the preference for such relatively stress-reducing assessment methods. This fits in with the concept of “constructivism” often quoted in the context of medical education. The study by van de Waterling et al also showed a lower preference for direct assessment methods such as oral tests. In their study they mentioned that one of the reasons for this could be that the students in their school were not very familiar with this method of assessment.Citation12 One of the reasons mentioned by the students in our study was that oral examinations may be affected by a lack of objectivity and examiner biases. Alternative tests such as computer-based tests were not popular, probably because the students were not familiar with such a concept in the context of assessment. Considering the importance of formative assessment in medical education and the apparent preference of the students for assessment methods such as supported written tests, projects, and portfolios, we should probably start giving more emphasis to such assessment methods, especially student portfolios. Assessment of medical students should give optimum emphasis to each domain from lower domains of factual knowledge/recall to higher domains of critical thinking such as evaluation, synthesis, and analysis. While designing assessment for a medical course the concept of an “end-process” assessment should be kept in mind, for example, doctors spend most of their time solving clinical problems, devising treatments plans, and appraising their efficacy, not recalling factual knowledge.Citation14,Citation15 On the other hand it is simply not possible to design a medical assessment module without giving due importance to factual knowledge. In our study the general preference was for assessment methods based on “understanding”/factual recall; however in the later clinical stages the preference for assessment methods based on higher domains such as critical thinking increased. Only when students have a good grasp of the basics concepts of medicine (including factual knowledge) can they successfully proceed to higher domains such as synthesis and evaluation. We therefore feel that while assessment should ideally encompass both factual recall and critical-thinking abilities, the stress on higher domains should gradually increase as the students move higher up in their course. Assessment items can be classified as selected-response (where the student selects the answer from a list) or as constructed-response (where the student constructs the answer, or performs a task). Well-written selected-response items can be used to measure higher order cognitive skills, such as problem-solving and clinical reasoning, and can be used to assess a wide area of knowledge in a short period of time.Citation3,Citation14,Citation16 As mentioned earlier, we do have a short 6-week period of an “integrated problem-based learning module” just before the students enter their clinical phase, but most of the other courses are not well integrated horizontally or vertically. This would partially explain the reason why questions based on comparisons/overall relationships between topics were not popular. We really have no good explanation for why the clinical group preferred supporting material. One possible explanation could be the increasing study load in the clinical years. Supporting materials would help decrease the stress and anxiety associated with this increase study burden. The preclinical students generally tend to be more diffident in any form of oral communication such as group discussions compared with their counterparts in the clinical years. It could be that their gradually increased exposure to clinical discussions and seminars makes them more confident and accommodating to modes that require good communication skills such as group discussions. These explanations align with the concept of “constructivism” often quoted in the context of medical education. For the specific assessment method for summative assessment, a majority of our students, irrespective of stage of learning and gender, preferred the MCQ format over open-ended questions/essay. The issue of why some students prefer MCQs while some others prefer essays has been discussed in previous studies.Citation17 It has been suggested that students with a deep study approach tended to prefer essay-type questions, while students with a superfical or surface study approach tended to prefer multiple choice formats.Citation17 Test anxiety seems to be another variable that can lead to specific attitudes towards assessment formats: students with high test anxiety have more favorable attitudes towards MCQs while those with low test anxiety tend to prefer open-ended formats. Clearly, students with a high level of test anxiety strive towards more certainty within the assessment situation.Citation17

The importance of this is that students are likely to be more motivated and hence perform better if they are given an assessment method with which they are comfortable, and this should apply to medical education also,Citation18 but how far and to what extent we are justified in offering such an option in a medical course (where students are expected to be able to have deep learning in many topics) is debatable.

Limitations

Ideally, student performance in terms of marks and grades achieved during their medical course should have been correlated with the preferred methods of assessment, to see if they actually performed better when given an assessment method of their choice. This was not possible, as only some of the assessment methods discussed are followed in a standardized manner in our college. We also did not try to evaluate student learning styles (surface vs deep study), which would have been relevant to our study.

Conclusion

Assessment drives learning in all forms of education including medical education. Students will be more motivated and hence perform better if they are assessed through methods of their choice. While it is not possible to tailor assessments completely according to student preferences, they can certainly be modified to accommodate student preferences, thus helping to improve student performance. One of the salient results of our study is that many students would be happy with an assessment mode that allows them to guess. It would be impractical to accommodate such a need completely in medical education assessment. However assessments can certainly be modified and made more accommodating. For example, MCQs can be made the major mode for summative assessment in our college to assess factual knowledge as it is the mode preferred by the majority. However the questions themselves can be made more complex, so as to ensure that higher domains are tested. Medical assessment should emphasize all domains of assessment from lower domains such as factual recall to higher domains such as evaluation or synthesis; however the stress on higher domains incorporating critical thinking should increase as the students progress through their course. Similarly the stress on formative assessment methods should increase.

Disclosure

The authors declare no conflicts of interest.

References

  • Ben-DavidFMAssociation of Medical Education in Europe guide no. 14: outcome-based education: part 3-assessment in outcome-based educationMed Teach1999212325
  • FelettiGISmithEKMModified essay questions: are they worth the effort?Med Educ1986201261323959927
  • EpsteinRMAssessment in medical educationN Engl J Med200735638739617251535
  • BirenbaumMTowards adaptive assessment-the students’ angleStud Educ Eval199420239255
  • Boulton-LewisGMThe SOLO taxonomy as a means of shaping and assessing learning in higher educationHigh Educ Res Dev199514143145
  • ScoullerKMInfluence of assessment method on students’ learning approaches, perceptions and preferences: The assignment essay versus the short answer examinationHigh Educ Res Dev199619776781
  • ScoullerKMProsserMStudents’ experiences in studying for multiple choice question examinationsStud High Educ199419267279
  • ParmenterDAEssay versus multiple-choice: student preferences and the underlying rationale with implications for test constructionAcad Educ Lead J200913318311847
  • BirenbaumMAssessment and instruction preferences and their relationship with test anxiety and learning strategiesHigh Educ200753749768
  • BirenbaumMNew insights into learning and teaching and their implications for assessmentSegersMDochyFCascallarEOptimizing New Methods of Assessment: In Search of Qualities and StandardsDordrecht, the NetherlandsKluwer20031336
  • StruyvenKDochyFJanssenSStudents’ perceptions about evaluation and assessment in higher education: A reviewAssess Eval High Educ200530331347
  • Van de WaterlingGGijbelsDDochyFvan der RijtJStudents’ assessment preferences, perceptions of assessment and their relationships to study resultsHigh Educ200856645658
  • Ben-ChaimDZollerUExamination-type preferences of secondary school students and their teachers in the science disciplinesInstr Sci199725347367
  • StarmerDLChapmanEMillwardMJApplying global frameworks to assessment in medical education: an example of a nationally produced curriculum for cancer educationJ Canc Educ201025285289
  • MillerDASadlerJZMohlPCThe cognitive context of examinations in psychiatry using bloom’s taxonomyMed Educ1991254804841758329
  • DowningSMBaranowskiRAGrossoLJItem type and cognitive ability measured: the validity evidence for multiple true-false items in medical specialty certificationAppl Meas Educ19958187197
  • BirenbaumMAssessment and instruction preferences and their relationship with test anxiety and learning strategiesHigh Educ200753749768
  • NewbleDIJaegerKThe effects of assessments and examinations on the learning of medical studentsMed Educ1983171651716865814