Exploring the Impact of Assessment on Medical Students’ Learning

Figures & data

Table 1: Assessment Formats and their Descriptors

Assessment Format	Assessment Types	Descriptor
Written Examinations	Clinical Investigations Examination (CLIX) Extended Matching Questions (EMQs) Key Feature Questions (KFQs)/Problems Multiple Choice Questions (MCQs) Short-Answer Questions (SAQs)	Written examinations assess basic knowledge and understanding as well as students’ ability to interpret key clinical investigations and diagnostic reasoning
Practical Examinations	Clinical Skills Mini-Clinical Evaluation Exercise (Mini-CEX) Multi-Station Assessment Task (MSAT) Objective Structured Clinical Examination (OSCE)	Practical examinations involve formal assessment of performance-related skills through set practical tasks/questions
Extended Written Assignments	Essay Reflective Writing Individual Learning Plan (ILP)	Extended written assignments typically require students to demonstrate conceptual understanding and higher order thinking skills (e.g. analysis, evaluation)
Oral Assessments	Poster presentation Informal presentations	Oral assessments are usually undertaken individually or as a group and can test communication and presentation skills, especially under pressure
Other	Case report Portfolio	Assessment tasks that require students to engage in processes wherein they identify or respond to a problem, collect relevant information and generate possible solutions, appraise the best solution, plan for implementation and, where possible, implement and evaluate

Table 2. Survey on Perceptions of Assessment

The survey questions were divided into two sections.

Section A: The first section included questions about participants’ demographic characteristics (year of study, enrolment status and gender).

Section B: The second section contained six different questions which related to participants’ perceptions of assessment.

Ques 1: How accurately does each assessment tool reflect the effort that you put into learning? 1= least accurate; 2= moderately accurate; 3 = most accurate.
Ques 2: How accurately does each assessment tool reflect your knowledge of the content material? 1= least accurate; 2= moderately accurate; 3 = most accurate.
	Not Applicable	Least	Moderate	Most
Multiple Choice Question (MCQ)	–	□		◀
Extended Matching Question (EMQ)	–	□		◀
Short Answer Question (SAQ)	–	□		◀
Key Feature Problem (KFP)	–	□		◀
Objective Structured Clinical Examination (OSCE)	–	□		◀
Multiple Stations Assessment Tasks (MSAT)	–	□		◀
Mini CEX	–	□		◀
Practical Examinations (anatomy)	–	□		◀
VIVA	–	□		◀
Essay	–	□		◀
Case report	–	□		◀
Reflective writing	–	□		◀
	Not at all useful		Very useful
Ques 3: How useful is ON-COURSE assessment in driving your learning?	□       ◀     ▶      ▴
	Very light		Very heavy
Ques 4: Overall, how would you rate the assessment load in your current year of study?	□       ◀     ▶      ▴
Ques 5: Which description best reflects how you perceive assessment? (Tick only one) (a) assessment quantifies my level of knowledge and/or competence (b) assessment helps me identify current gaps in my learning (c) assessment enables me to address gaps in my learning
Ques 6: If you could change one thing about assessment, what would that be? (free text)

Table 3: Demographics of participants (quantitative survey)

Variable	Frequency	Percent
Year Level
1	130	23.6
2	106	19.3
3	75	13.6
4	80	14.6
5	84	15.3
6	75	13.6
Enrolment status
Domestic	472	85.8
International	78	14.2
Sex
Male	208	37.8
Female	342	62.2

Table 4. Demographics of participants (qualitative focus group discussions)

Participant Group	Number
Male	6
Female	18
Year 2	9
Year 3	5
Year 4	3
Year 5*	2
Year 6*	5
Held academic representation role	4
Total focus group participants	24

*One focus group was held with year 5 and 6 students

Table 5. Accuracy of different assessment tools in reflecting effort put into learning and knowledge of content material*

Assessment instrument	Efforts put into learning n (%)	Median (IQR)*	Knowledge of content material n (%)	Median (IQR)*
Multiple Choice Question (MCQ)	156 (28.5)	2 (1)	163 (30.4)	2 (1)
Extended Matching Question (EMQ)	136 (24.9)	2 (1)	148 (27.6)	2 (1)
Short Answer Question (SAQ)	252 (46.2)	2 (1)	246 (46.2)	2 (1)
Key Feature Problem (KFP)	277 (50.8)	3 (1)	254 (48.3)	2 (1)
Multiple Station Assessment Tasks (MSAT)	169 (31.6)	2 (2)	120 (23.5)	2 (1)
Objective Structured Clinical Examination (OSCE)	151 (29.2)	2 (3)	109 (21.9)	0 (2)
Mini-CEX	42 (8.4)	0 (2)	38 (7.8)	0 (2)
Practical Examinations (laboratory, anatomy)	62 (12.2)	0 (2)	55 (11.1)	0 (2)
Essay	44 (8.2)	1 (1)	26 (5)	1 (1)
Case Report	56 (10.8)	0 (2)	38 (7.5)	1 (1)
Reflective writing	35 (6.5)	1 (1)	18 (3.5)	1 (0)

* (0 = Not Applicable, 1 = least, 2 = Moderate, 3 = Most)

Figure 1. Students' performance in the different assessments

Medical students' mean performance scores in various on-course and end of semester assessment items from 2013 to 2017. Bars bearing different letter(s) are significantly different. Students’ scores in on-course assessment were significantly different to their scores on all other assessments P < 0.05 (a). MCQs scores were significantly different to SAQ (short answer questions) and MSAT/OSCE (multi-station assessment task/objective structured clinical exam) scores P < 0.01 (b), but not different to KFP scores. No significant difference was noted between SAQ, MSAT/OSCE and KFP.

Figure 2. Performance Scores by Year of study

First to fifth year medical students' performance scores in various on-course and end of semester assessment items from 2013 to 2017. Bars within same assessment type that bear different letter(s) are significantly different. Junior students had significantly higher performance scores in their MCQ (multiple-choice questions) and on-course assessments than the seniors p < 0.05, while the senior students had higher scores in their MSAT/OSCE (multi-station assessment task/objective structured clinical exam) p < 0.05.

Table 6. Perceptions of on-course assessment and assessment load (Mean$\pm$SD)

Variable	Usefulness of ON-COURSE assessment in driving learning	Ratings of assessment load in current year of study
Year of Study
Year 1	3.41 $\pm$ 1.17	3.47 $\pm$ 1.05
Year 2	2.53 $\pm$ 1.25	3.92 $\pm$ 0.78
Year 3	2.59 $\pm$ 1.22	3.82 $\pm$ 0.77
Year 4	3.14 $\pm$ 1.08	3.94 $\pm$ 0.68
Year 5	3.13 $\pm \mathrm{ }$0.99	3.82$\mathrm{ }\pm$ 0.77
Year 6	3.00 $\pm$ 1.18	3.08 $\pm$ 0.92
Sex
Male	2.92 $\pm$ 1.20	3.64 $\pm$ 0.90
Female	3.03 $\pm$ 1.18	3.58 $\pm$ 0.91
Status
Domestic	2.98 $\pm$ 1.20	3.67 $\pm$ 0.90
International	3.05 $\pm$ 1.21	3.66 $\pm$ 0.95
Overall	2.99 $\pm$ 1.20	3.67$\mathrm{ }\pm$ 0.94