Assessing students’ research reports: Development of a rating scale

Abstract

Background: There is a lack of explicit criteria for the assessment of students’ research reports. Teachers tend to use their own, idiosyncratic sets of implicit criteria. A well-defined set of criteria could foster reliable and valid assessment of students’ research reports.

Aim: The aim of this study was twofold: to detect the strengths and weaknesses of students’ research reports and to develop an assessment tool for students’ research papers.

Methods: On the basis of the literature and advice from experts, we developed a list of 19 criteria, comprising 15 specific items and 4 global items. Three raters, using the list, independently scored a sample of 18 research reports.

Results: The strengths of the reports were the structure of the results and the description of materials and methods. The weaknesses were the research questions, the grounding of the study in the research literature, the analysis, the statistics, and the clarity of visuals. With three raters, the reliability of the rating scale was reasonable.

Conclusion: The rating scale seems to be a useful tool for judging and giving feedback on students’ research reports. These findings will have to be confirmed in studies with larger samples.

Additional information

Notes on contributors

Debbie A. D. C. Jaarsma

DEBBIE A. D. C. JAARSMA, DVM, is a veterinarian and a PhD student at the Faculty of Veterinary Medicine, Utrecht University, The Netherlands.

Wouter J. Kranenbarg

WOUTER J. KRANENBARG, BSc, is a student at the Faculty of Veterinary Medicine, Utrecht University, The Netherlands.

Diana H. J. M. Dolmans

DIANA H. J. M. DOLMANS, PhD, is an educational psychologist and associate professor in the Department of Educational Development and Research at Maastricht University, The Netherlands.

Arno M. M. Muijtjens

ARNO M. M. MUIJTJENS, PhD, is a statistician in the department of Educational Development and Research at Maastricht University, The Netherlands.

Albert J. J. A. Scherpbier

ALBERT J. J. A. SCHERPBIER, MD, PhD, is a professor of quality improvement in medical education and scientific director of the Institute for Medical Education at Maastricht University, The Netherlands.

Peter Van Beukelen

PETER VAN BEUKELEN, DVM, PhD, is a professor of quality improvement in veterinary education at the Faculty of Veterinary Medicine at the University of Utrecht, The Netherlands.