Can we share questions? Performance of questions from different question banks in a single medical school

Abstract

Background: To use progress testing, a large bank of questions is required, particularly when planning to deliver tests over a long period of time. The questions need not only to be of good quality but also balanced in subject coverage across the curriculum to allow appropriate sampling. Hence as well as creating its own questions, an institution could share questions. Both methods allow ownership and structuring of the test appropriate to the educational requirements of the institution.

Method: Peninsula Medical School (PMS) has developed a mechanism to validate questions written in house. That mechanism can be adapted to utilise questions from an International question bank International Digital Electronic Access Library (IDEAL) and another UK-based question bank Universities Medical Assessment Partnership (UMAP). These questions have been used in our progress tests and analysed for relative performance.

Results: Data are presented to show that questions from differing sources can have comparable performance in a progress testing format.

Conclusion: There are difficulties in transferring questions from one institution to another. These include problems of curricula and cultural differences. Whilst many of these difficulties exist, our experience suggests that it only requires a relatively small amount of work to adapt questions from external question banks for effective use. The longitudinal aspect of progress testing (albeit summatively) may allow more flexibility in question usage than single high stakes exams.

Additional information

Notes on contributors

Adrian Freeman

Adrian Freeman, MMedSci FRCGP, is a lecturer in assessment at Peninsula College of Medicine and Dentistry.

Anthony Nicholls

Anthony Nicholls, MBBS FRCP, is an honorary professor of medicine at Peninsula College of Medicine and Dentistry.

Chris Ricketts

Chris Ricketts, PhD, is director of assessment at Peninsula College of Medicine and Dentistry, UK and a National Teaching Fellow.

Lee Coombes

Lee Coombes, PhD, is a psychometrician and assessment analyst for Peninsula College of Medicine and Dentistry, UK.