A competency based selection procedure for Dutch postgraduate GP training: A pilot study on validity and reliability

Abstract

Background: Historically, semi-structured interviews (SSI) have been the core of the Dutch selection for postgraduate general practice (GP) training. This paper describes a pilot study on a newly designed competency-based selection procedure that assesses whether candidates have the competencies that are required to complete GP training.

Objectives: The objective was to explore reliability and validity aspects of the instruments developed.

Methods: The new selection procedure comprising the National GP Knowledge Test (LHK), a situational judgement tests (SJT), a patterned behaviour descriptive interview (PBDI) and a simulated encounter (SIM) was piloted alongside the current procedure. Forty-seven candidates volunteered in both procedures. Admission decision was based on the results of the current procedure.

Results: Study participants did hardly differ from the other candidates. The mean scores of the candidates on the LHK and SJT were 21.9 % (SD 8.7) and 83.8% (SD 3.1), respectively. The mean self-reported competency scores (PBDI) were higher than the observed competencies (SIM): 3.7(SD 0.5) and 2.9(SD 0.6), respectively. Content-related competencies showed low correlations with one another when measured with different instruments, whereas more diverse competencies measured by a single instrument showed strong to moderate correlations. Moreover, a moderate correlation between LHK and SJT was found. The internal consistencies (intraclass correlation, ICC) of LHK and SJT were poor while the ICC of PBDI and SIM showed acceptable levels of reliability.

Conclusion: Findings on content validity and reliability of these new instruments are promising to realize a competency based procedure. Further development of the instruments and research on predictive validity should be pursued.

Keywords::

• Medical education
• general practice
• post graduate training
• selection

ACKNOWLEDGMENTS

The authors would like to thank Mariëlle Damoiseaux and Edger Gubbels for their assistance in developing and executing the procedure; Barend Koch for his elaborate work on the SJT and Huisartsopleiding Nederland for making the LHK available.

FUNDING

SBOH (Stichting Beroeps Opleiding Huisartsen, Dutch abbreviation).

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.