Admission selection criteria as predictors of outcomes in an undergraduate medical course: A prospective study

Figures & data

Table

Practice points

A structured interview emphasising communication skills can add value to the selection of school-leavers into a medical course. The interview score was most closely associated with clinical outcomes. Previous academic achievement and female sex were consistent predictors of course outcomes. The effects of interview scores and aptitude test scores should continue to be evaluated post-graduation.

Table 1.  Selection into the 6-year undergraduate medical course at the UWA

Standard (school-leaver) applicants
1. TER An academic score formed from the aggregate of results in the state Tertiary Entrance Examinations held at the end of secondary school	Minimum rank of 96 on a scale to 99.95
2. UMAT total score of three sections: UMAT_1 Logical reasoning and problem solving UMAT_2 Understanding people UMAT_3 Non-verbal reasoning	Threshold set each year. Each section is standardised to a mean of 50 and standard deviation of 10
3. Score on structured interview (Table 2) Six criteria plus a global score for communication skills The six criteria were varied each year for security purposes	Threshold set each year 1999–2005, maximum score of 28 2006–2009, maximum score of 42 (revised scale)

Note: Ranking of applicants is by a combined score using the three components. Initially the three components were weighted equally. From 2007 entry, they were weighted in the ratio 2 : 2 : 1 for TER, Interview score, UMAT.

Table 2.  The structured interview

• Interviews were conducted by a panel of two consisting of a male and a female, a university member and a community member; and all interviewers were required to re-train each year

• Six criteria were assessed each year using three set questions for each criterion. The seventh criterion communication skills was assessed across the responses to the set questions

• The interview had a highly structured format in which all applicants were asked exactly the same questions and only standard prompts were used

• The basic format of the interview remained consistent over the years, with changes to the rating scales in 2006. Originally, the seven criteria were each scored 0–4, more recently each criterion was scored 0–6

• The final score was a consensus score determined after each interviewer had assessed the responses independently against clearly defined rating scales

• A bank of criteria had been developed. The criteria were based on qualities suited to the study and practice of medicine, such as ability to work in a team, ability to see from the perspective of others, social responsibility, recognising and responding to social diversity, ethics, coping with uncertainty, etc. After the criteria were selected each year, the questions and rating scales were revised or developed by a committee of five. One new criterion was developed each year

• The criteria commitment and motivation to study medicine and communication skills were assessed each year

• The assessment of communication skills was across four domains: comprehension, articulation, relevancy and interaction

• The time allocation for an interview was 60–70 min, with the actual interview averaging 35 min and the remainder of the time being used for individual and consensus ratings

Table 3.  Pearson correlation coefficients for each predictor variable versus the year level WAM

		WAM Year 1 (N = 1174)	WAM Year 2 (N = 1017)	WAM Year 3 (N = 901)	WAM Year 4 (N = 799)	WAM Year 5 (N = 688)	WAM Year 6 (N = 547)
Age at entry	Pearson correlation	−0.018	−0.036	−0.031	−0.065	−0.044	−0.038
	Significance (2-tailed)	0.536	0.245	0.350	0.066	0.253	0.375
Sex	Pearson correlation	−0.112	−0.122	−0.143	−0.226	−0.308	−0.210
	Significance (2-tailed)	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001
TER	Pearson correlation	0.468	0.401	0.321	0.230	0.208	0.206
	Significance (2-tailed)	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001
Z-score: interview score	Pearson correlation	0.012	0.040	0.054	0.138	0.102	0.111
	Significance (2-tailed)	0.675	0.202	0.104	<0.001	0.007	0.009
Z-score: UMAT total score	Pearson correlation	0.030	−0.001	−0.006	0.035	0.038	−0.044
	Significance (2-tailed)	0.308	0.970	0.860	0.316	0.318	0.299
Z-score: UMAT1 – logical reasoning and problem solving	Pearson correlation	0.043	0.011	0.005	0.057	0.085	0.042
	Significance (2-tailed)	0.136	0.726	0.892	0.107	0.026	0.322
Z-score: UMAT2 – interaction skills	Pearson correlation	−0.014	−0.044	−0.009	0.039	0.056	−0.023
	Significance (2-tailed)	0.634	0.160	0.787	0.276	0.143	0.595
Z-score: UMAT3 – non-verbal reasoning	Pearson correlation	0.034	−0.003	0.009	−0.045	−0.086	−0.070
	Significance (2-tailed)	0.243	0.929	0.795	0.203	0.025	0.104

Table 4.  Regression models of the relationship between selection criteria for standard entry students 1999–2009 and academic performance as assessed by yearly WAM

	WAM Year 1 (N = 1174)	WAM Year 2 (N = 1017)	WAM Year 3 (N = 901)	WAM Year 4 (N = 799)	WAM Year 5 (N = 688)	WAM Year 6 (N = 547)
Age (year)
Beta	0.059	0.027	0.021	−0.026	−0.011	−0.001
p-value	0.022	0.340	0.513	0.430	0.625	0.975
Sex (F = 0/M = 1)
Beta	−0.151	−0.16	−0.176	−0.245	−0.328	−0.235
p-value	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001
TER score
Beta	0.505	0.442	0.359	0.272	0.257	0.246
p-value	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001
Interview Z-score
Beta	0.027	0.046	0.061	0.139	0.089	0.114
p-value	0.306	0.207	0.054	<0.001	0.012	0.006
UMAT 1 Z-score
Beta	−0.005	−0.031	−0.023	0.048	0.068	0.030
p-value	0.837	0.165	0.473	0.56	0.058	0.467
UMAT 2 Z-score
Beta	−0.015	−0.047	−0.013	0.016	0.018	−0.038
p-value	0.571	0.092	0.682	0.629	0.614	0.369
UMAT 3 Z-score
Beta	−0.047	−0.072	−0.027	−0.025	−0.048	−0.037
p-value	0.076	0.071	0.405	0.462	0.187	0.389
R^2 (%)	25.1	20.0	14.3	14.3	17.5	11.7

Table 5.  Regression models of the relationship between selection criteria and academic performance in specific ‘knowledge’-based units

	Normal systems (levels 1 and 2) (N = 1074)	Pathology, pharmacology, microbiology (level 3) (N = 901)	Science and practice of medicine (levels 4–6) (N = 799)
Age (year)
Beta	−0.012	0.023	−0.022
p-value	0.676	0.467	0.511
Sex (F = 0/M = 1)
Beta	−0.087	−0.131	−0.227
p-value	0.002	<0.001	<0.001
TER
Beta	0.462	0.360	0.227
p-value	<0.001	<0.001	<0.001
Interview Z-score
Beta	0.031	0.042	0.084
p-value	0.259	0.183	0.014
UMAT 1 Z-score
Beta	−0.040	−0.029	0.096
p-value	0.151	0.359	0.005
UMAT 2 Z-score
Beta	−0.043	−0.011	−0.008
p-value	0.130	0.729	0.814
UMAT 3 Z-score
Beta	−0.043	−0.036	−0.056
p-value	0.130	0.265	0.109
R^2 (%)	20.3	13.1	11.4

Table 6.  Regression models of the relationship between selection criteria and academic performance in specific ‘clinically’ based units

	Foundations of clinical practice (levels 1–3) (N = 1165)	Clinical skills (levels 4–6) (N = 799)	General practice (levels 5 and 6) (N = 688)
Age (year)
Beta	0.023	−0.024	−0.050
p-value	0.384	0.478	0.176
Sex (F = 0/M = 1)
Beta	−0.336	−0.171	−0.255
p-value	<0.001	<0.001	<0.001
TER
Beta	0.352	0.182	0.132
p-value	<0.001	<0.001	0.001
Interview Z-score
Beta	0.075	0.182	0.131
p-value	0.005	<0.001	<0.001
UMAT 1 Z-score
Beta	−0.049	0.077	0.050
p-value	0.064	0.026	0.180
UMAT 2 Z-score
Beta	−0.007	−0.028	0.011
p-value	0.784	0.418	0.762
UMAT 3 Z-score
Beta	−0.078	−0.061	0.008
p-value	0.004	0.084	0.828
R^2 (%)	22.6	10.1	10.0