Assessing the Reliability and Validity of a Residency Selection Process Examination: A Preliminary Study Between the National Board of Medical Examiners and the University of Panama Faculty of Medicine

Abstract

Background: The Ministry of Health of the Republic of Panama is currently developing a national examination system that will be used to license graduates to practice medicine in that country, as well as to undertake postgraduate medical training. As part of these efforts, a preliminary project was undertaken between the National Board of Medical Examiners (NBME) and the Faculty of Medicine of the University of Panama to develop a Residency Selection Process Examination (RSPE). Purpose: The purpose of this study was to assess the reliability and validity of RSPE scores for a sample of candidates who wished to obtain a residency slot in Panama. Methods: The RSPE, composed of 200 basic and clinical sciences multiple-choice items, was administered to 261 residency applicants at the University of Panama. Results: The reliability estimate computed was comparable with that reported with other high-stakes examinations (Cronbach's α = 0.89). Also, a Rasch examinee proficiency item difficulty plot showed that the RSPE was well targeted to the proficiency levels of candidates. Finally, a moderate correlation was noted between local grade point averages and RSPE scores for University of Panama students (r = 0.38). Conclusions: Findings suggest that it is possible to translate and adapt test materials for use in other contexts.