Generalizability of performance on different‐station‐length standardized patient cases

Abstract

The relation between individual station length in performance‐based examinations and the generalizability of examinee scores is not well‐understood. This relation was examined in two studies measuring examinee performance over different time intervals. In the first study, performance measures at 5‐ and 10‐min intervals and complete 20‐min stations were obtained from checklists and generalizability estimates calculated at each interval. Results showed the greatest generalizability coefficient was obtained at the 10‐min interval. A second study, with actual 10‐min stations, also showed a higher generalizability coefficient at a 10‐min interval than at a 5‐min interval. Satisfactory psychometric properties may be obtained with 10‐min stations, realizing considerable savings in administration time and costs over longer station examinations. Changes in examinee scores within checklist organizational categories were also examined and interpreted in terms of possible changes in student interview strategies as a result of time demands.