Validation of the General Medicine in-Training Examination Using the Professional and Linguistic Assessments Board Examination Among Postgraduate Residents in Japan

Figures & data

Table 1 Characteristics of the Participants

Characteristics	Total N = 91
	n	%
PGY
PGY1	32	35.2%
PGY2	59	64.8%
Sex
Male	22	24.2%
Female	69	75.8%
Hospital type
Community	91	100.0%
University	0	0.0%
University branch	0	0.0%
Self-study time per day
None	5	5.5%
0–30 min	36	39.6%
30–60 min	35	38.5%
60–90 min	10	11.0%
≥90 min	2	2.2%
ED duty per month
0	2	2.2%
1–2	7	7.7%
3–5	68	74.7%
≥6	11	12.1%
Unknown	3	3.3%
Number of inpatients in charge
0–4	10	11.0%
5–9	68	74.7%
10–14	8	8.8%
≥15	0	0.0%
Unknown	5	5.5%
Experience of taking international examinations
USMLE	4	4.4%
PLAB	1	1.1%
Other examinations	0	0.0%
None	83	91.2%
Examination score	Mean	SD
GM-ITE score	30.2	5.5
PLAB score	119.5	11.7

Abbreviations: PGY, postgraduate year; ED, emergency department; PLAB, Professional and Linguistic Assessments Board; USMLE, United States Medical Licensing Examination; GM-ITE, General Medicine In-training Examination.

Table 2 Pearson Correlation Coefficients of the General Medicine in-Training Examination and Professional and Linguistic Assessments Board 1 Examination Scores

GM-ITE	PLAB 1 Examination
	r	p-value
Total	0.582	<0.001
Subcategories
Medical interview/professionalism	0.252	0.015
Symptomatology/clinical reasoning	0.540	<0.001
Physical examination/procedure	0.376	<0.001
Disease knowledge	0.364	<0.001

Notes: r = Pearson correlation coefficient. r ≥ 0.40 was considered satisfactory. Correlations were considered significant at the 0.01 level (two-tailed).

Abbreviations: PLAB, Professional and Linguistic Assessments Board; GM-ITE, General Medicine In-training Examination.

Figure 1 Scatter plot of the General Medicine In-training Examination (GM-ITE) and Professional and Linguistic Assessments Board (PLAB) 1 examination scores. The vertical axis is the PLAB 1 examination score, and the horizontal axis is the GM-ITE score.

Notes: The dotted line indicates the trend line. The two solid lines indicate the 95% prediction limits. r = Pearson correlation coefficient. r ≥ 0.40 was considered satisfactory. Correlations were considered significant at the 0.01 level (two-tailed).

Abbreviations: GM-ITE, General Medicine In-training Examination; PLAB, Professional and Linguistic Assessments Board.

Figure 2 Discrimination indexes of the examination scores of the General Medicine In-training Examination (GM-ITE; A) and Professional and Linguistic Assessments Board 1 (PLAB; B). In both figures, the horizontal axis is the discrimination indices (DI), and the vertical axis is the proportion of questions within a certain DI range in the total questions. The figure with black bars shows the DI of GM-ITE, and the figure with shaded bars shows that of PLAB examination.

Notes: A discrimination index ≥ 0.20 indicates that the question has a high discriminatory power, and a discrimination index of >0.40 indicates that the question is a very good measure of the subject’s qualifications.

Abbreviations: GM-ITE, General Medicine In-training Examination; PLAB, Professional and Linguistic Assessments Board.

Table 3 Development Process of the General Medicine in-Training Examination

April	Question Development Committee:  -Examination analysis of the previous year’s examination
May	Question Development Committee (pre-meeting):  -Determination of the subject areas and diseases
Early June	Question Development Committee (phase 1):  -Asks members to create new questions
Mid-June to late August	Question development period
Early September	Peer-review evaluation period
Mid-September	Peer-review committee (phase 1):  -Selection of 60 questions from among 100 questions  -Preparing revision plans
Late September	Peer-review committee (phase 2):  -Asks committee members to revise questions for question development
Early October	Question development committee (phase 2):  -Correction of the questions
Mid to late October	Pilot study by several senior residents
Early November	Question development committee (phase 3):  -Analysis of the result of the pilot study  -Minor correction of the questions
Mid-November	The final coordination
Late November	The completion of the question development process
Late January	Examination implementation