Abstract
Purpose
In the assessment of basic medical knowledge, the composition of the reference panel between specialists and primary care (PC) physicians is a contentious issue. We assessed the effect of panel composition on the scores of undergraduate medical students in a script concordance test (SCT).
Methods
The scale of an SCT on basic nephrology knowledge was set by a panel of nephrologists or a mixed panel of nephrologists and PC physicians. The results of the SCTs were compared with ANOVA for repeated measurements. Concordance was assessed with Bland and Altman plots.
Results
Forty-five students completed the SCT. Their scores differed according to panel composition: 65.6 ± 9.73/100 points for nephrologists, and 70.27 ± 8.82 for the mixed panel, p < 0.001. Concordance between the scores was low with a bias of −4.27 ± 2.19 and a 95% limit of agreement of −8.96 to −0.38. Panel composition led to a change in the ranking of 71% of students (mean 3.6 ± 2.6 places).
Conclusion
The composition of the reference panel, either specialist or mixed, for SCT assessment of basic knowledge has an impact on test results and student rankings.
Author contributions
JA: Conceptualization, Data curation, Formal analysis, Writing – Original Draft JT: Investigation GT: Investigation SB: Investigation BS: Investigation BP: Formal analysis CG: Investigation CP: Investigation PC: Supervision, Writing Reviewing and Editing AEH: Supervision, Writing Reviewing and Editing AL: Conceptualization, Supervision, Writing Reviewing and Editing.
Acknowledgements
The authors would like to thank Jeffrey Watts for editing the manuscript.
Disclosure statement
All authors have no conflicts of interest to this study.
Additional information
Funding
Notes on contributors
Julien Aniort
Julien Aniort, MD, PhD, Department of Nephrology, University Hospital, School of Medicine, Clermont-Ferrand, France.
Jeromine Trefond
Jeromine Trefond, MD, Department of General Medicine, School of Medicine, Clermont-Ferrand, France
Gilles Tanguy
Gilles Tanguy, MD Department of General Medicine, School of Medicine, Clermont-Ferrand, France.
Stanislas Bataille
Stanislas Bataille, MD, PhD Department of Nephrology, University Hospital, School of Medicine, Marseille, France.
Stephane Burtey
Stephane Burtey, MD, PhD Department of Nephrology, Bouchard Clinic, Marseille, France.
Bruno Pereira
Bruno Pereira, PhD, Department of Biostatistics, University Hospital, Clermont-Ferrand, France.
Cyril Garrouste
Cyril Garrouste, MD, Department of Nephrology, School of Medicine, Clermont-Ferrand, France.
Carole Philipponnet
Carole Philipponnet, MD, Department of Nephrology, University Hospital, Clermont-Ferrand, France.
Pierre Clavelou
Pierre Clavelou, MD, PhD, Department of Neurology, University Hospital, dean of the school of Medicine Clermont-Ferrand France.
Anne-Elisabeth Heng
Anne-Elisabeth Heng, MD, PhD, Department of Nephrology, School of Medicine, Clermont-Ferrand, France.
Alexandre Lautrette
Alexandre Lautrette, MD, PhD, ICU, Jean-Perrin Hospital, School of Medicine, Clermont-Ferrand, France.