The Predictive Validity of the GRE Across Graduate Outcomes: A Meta-Analysis of Trends Over Time

Figures & data

Table 1. Exclusion and Inclusion Criteria Traced from Initial to Final Sample.

Inclusion and exclusion criteria	Number included	Number excluded
Peer-reviewed Journal articles published in English or theses and dissertations in English identified with searching terms of “GRE,” “graduate record exam*” in Title OR “graduate record exam*” in Abstract OR “graduate record exam*” in KW Identifiers OR “GRE” in Title OR “GRE” in Abstract OR “GRE” in KW Identifiers from PsycINFO, PubMed, ERIC, ProQuest, JSTOR, WorldCat databases;	3036
Exclusion of duplicates	2439	597
Number of relevant articles after irrelevance coding based on title and abstract: non-empirical, literature review, meta-analysis, and intervention studies, and if the articles belong to those of unrelated outcome of interest, lack of reported relationship between GRE & outcome, or lack of relevant information about the relationship between GRE & outcome;	1298	1141
Relevance coding based on full article review: GRE as independent variable, reporting relationship between GRE & outcome, codable information between GRE & outcome;	221	1077
Snowball sampling articles	46
Articles for final coding	260
Excluding articles in exclusively medical or veterinary samples		59
Articles for final analysis	201

Table 2. Coding Sheet for Each Effect per Article.

Variable of Interest	Input or dropdown choices
General Article Information
Year of publication	[NUMERICAL INPUT]
Author last name  Type of publication  Relevance	[TEXT INPUT] Peer-Reviewed Article Dissertation Meta-Analysis Literature Review Other article without code-able results Not Relevant Relevant
Study Design
Sample discipline	[TEXT INPUT]
Percentage men	[NUMERICAL INPUT]
Percentage white  Data Source	[NUMERICAL INPUT] [TEXT INPUT]
Independent variable	[TEXT INPUT] (must be a GRE variable)
Dependent variable	[TEXT INPUT] (must be a student outcome)
Data Characteristics
Statistical test sample size	[NUMERICAL INPUT]
Estimate being reported	Correlation Regression t-test Other: [TEXT INPUT]
Statistic reported	Pearson’s product-moment correlation Spearman’s correlation Correlation (unspecified type) Standardized B or β Odds ratio t-value Unstandardized B (Standard Error) Exact p-value from t-test or regression analysis
Standardized Results
Statistic value	[NUMERICAL INPUT]
Significance (p < .05)	Yes No
Directionality^a	Positive Negative Zero

Multiple effects could be coded per article. ^arefers to whether the effect is positive or negative on the student (e.g., a positive relationship between GRE and retention is a positive effect; a positive relationship between GRE and attrition is a negative effect; a negative relationship between GRE and attrition is a positive effect, etc.).

Figure 1. Funnel plot with Egger’s regression line.

Figure 2. Predictive Value of the GRE Over Time, Effects Across Studies.

Note. SED % non-white indicates the percentage of respondents for the Survey of Earned Doctorates in each decadal year, illustrating increasing diversity of the population of those successfully completing the Ph.D. Callout boxes report aggregate adjusted effect sizes (nesting effects within study) at the first and last time points.

Figure 3. Predictive Value of the GRE Across Gender, Effects Across Studies.

Note. Callout boxes report aggregate adjusted effect sizes (nesting effects within study) at the 0% and 100% points.

Figure 4. Predictive Value of the GRE Across racially minoritized, Effects Across Studies.

Note. Callout boxes report aggregate adjusted effect sizes (nesting effects within study) at the 0% and 100% points.

Table 3. Disciplinary Impact on the Predictive Validity of the GRE.

Sample Discipline	N Effects	Average $r_{ij}$
All Disciplines	1659	.18
Psychology, Counseling	653	.19
Non-Psychology Social Sciences	142	.17
Education	174	.18
Science & Engineering	189	.11
Applied Science	90	.19
Humanities & Liberal Arts	48	.23
Multiple Disciplines	228	.15
Other	63	.19
Discipline not Reported	72	.22

Other = Business, Home Economics, Architecture. Effect Size differences were not found.

Table 4. Differences in the Predictive Validity across GRE Subsections.

GRE Predictor	N Effects	Average $r_{ij}$
GRE Total	226	.19
GRE Verbal	562	.17
GRE Quantitative	561	.16
GRA Analytical	310	.20

Table 5. Differences in the Predictive Validity across Different Student Outcomes.

Student Outcome	N Effects	Average $r_{ij}$	Change in $r_{ij}$ across Year
			B	t	p
GPA	632	.20	.0014	−1.67	.10
Degree Progress	288	.08	−.0032	−2.06	.04
Attrition	10	.04	.0018	0.28	.79
Comprehensive Exam	69	.22	−.0009	−0.44	.66
Licensing Exam	50	.24	−.0004	−0.07	.94
Course Grades	214	.15	−.0028	−1.02	.33
Faculty Ratings	98	.15	−.0017	−1.14	.27
Skills	229	.15	−.0002	−0.13	.90
Reflection	69	.29	.0028	1.06	.31

Bold indicates significant effect; italicized indicates negative effect. Skills = Critical Thinking, Research, Professional Skills; Reflection = Self-evaluation, Goals, Successes, Disciplinary Action. NA indicates that the model assessed could not be computed; this was due to the combination of small sample sizes across student outcomes and lack of studies reporting demographic characteristics.

Figure 5. Predictive Value of the GRE on GPA Over Time, Effects Across Studies.

Note. SED % non-white indicates the percentage of respondents for the Survey of Earned Doctorates in each decadal year, illustrating increasing diversity of the population of those successfully completing the Ph.D. Callout boxes report aggregate adjusted effect sizes (nesting effects within study) at the first and last time points.

Figure 6. Predictive Value of the GRE on GPA Across Gender, Effects Across Studies.

Note. Callout boxes report aggregate adjusted effect sizes (nesting effects within study) at the 0% and 100% points.

Figure 7. Predictive Value of the GRE on GPA Across Race, Effects Across Studies.

Note. Callout boxes report aggregate adjusted effect sizes (nesting effects within study) at the 0% and 100% points.

Table 6. Differences in the Predictive Validity of the GRE on GPA across Disciplines.

Sample Discipline	N Effects	Average $r_{ij}$
All Disciplines	632	.20
Psychology, Counseling	153	.21
Non-Psychology Social Sciences	54	.26
Education	86	.24
Science & Engineering	90	.17
Applied Science	54	.18
Humanities & Liberal Arts	32	.30
Multiple Disciplines	104	.14
Other	27	.18
Discipline not Reported	32	.23

Other = Business, Home Economics, Architecture. NA indicates that the model assessed could not be computed; this was due to the combination of small sample sizes across disciplines and lack of studies reporting demographic characteristics.

Table 7. Differences in the Predictive Validity of the GRE on GPA across GRE Subsections.

GRE Predictor	N Effects	Average $r_{ij}$
GRE Total	90	.24
GRE Verbal	214	.20
GRE Quantitative	217	.17
GRE Analytical	111	.21