The influence of gender stereotype threat on mathematics test scores of Dutch high school students: a registered report

Figures & data

Figure 1. An overview of the planned analyses.

Table 1. Competing hypotheses Bayesian analysis.

Name	Hypothesis	Description
No stereotype threat hypothesis	H0: µthreat/girl = µcontrol/girl, µcontrol/boy = µthreat/boy	Equality constraints on the means for conditions. No constraints on the gender mean differences
Stereotype threat hypothesis	H1: µthreat/girl < µcontrol/girl, µthreat/boy = µcontrol/boy	For girls: mean in ST condition constrained to be lower than in the control condition. For boys: equality constraints on the means for conditions. No constraints on the gender mean differences
Stereotype threat and stereotype lift hypothesis	H2: µthreat/girl < µcontrol/girl, µthreat/boy > µcontrol/boy	For girls: mean in ST condition constrained to be lower than in the control condition. For boys: mean in ST condition constrained to be higher than in the control condition. No constraints on the gender mean differences
Complement hypothesis	HC: not H0, H1, or H2	The complement of the hypotheses described above

Table 2. Interpretation Bayes factors.

BFia	Evidence against Ha
1–3	Negligible
3–20	Positive
20–150	Strong
>150	Very strong

BF_ia = Bayes factor of inequality constrained hypotheses H_i against the null or complement hypothesis H_a. H_a = null or complement hypothesis.

Table 3. Averages and standard deviations for math performance (scored in several ways), missing values, math anxiety scale, domain identification, and gender identification.

	Guess corrected		Number correct		Accuracy		Missing		M.A.		D.I.		G.I.
	Mean (SD)	N	Mean (SD)	N	Mean (SD)	N	Mea (SD)	N	Mean (SD)	N	Mean (SD)	N	Mean (SD)	N
Girls-ST	9.10 (3.91)	510	11.41 (3.10)	510	0.60 (0.15)	510	0.87 (1.59)	510	19.45 (8.41)	505	33.90 (8.88)	493	12.73 (2.72)	494
Girls-C	9.31 (3.93)	526	11.61 (3.12)	526	0.60 (0.15)	526	0.70 (1.45)	526	18.98 (8.27)	522	34.12 (8.50)	509	12.86 (2.80)	511
Boys-ST	10.63 (3.99)	519	12.67 (3.13)	519	0.65 (0.16)	519	0.51 (1.21)	519	17.09 (7.82)	513	35.96 (8.86)	503	13.67 (2.93)	503
Boys-C	10.71 (3.97)	509	12.72 (3.13)	509	0.65 (0.15)	509	0.58 (1.28)	509	16.74 (7.73)	503	35.64 (9.24)	496	13.23 (2.91)	485

ST = stereotype threat condition, C = control condition, M.A. = math anxiety, D.I. = domain identification, G.I. = gender identification.

Table 4. Cohen’s d, Cronbach’s α, greatest lower bound, skewness, and kurtosis.

	Guess corrected	Number correct	Accuracy	M.A.	D.I.	G.I.
Cohen’s d STgirls-Cgirls (95% C.I.)	−0.05 (−0.18, 0.07)	−0.07 (−0.19, 0.06)	−0.03 (−0.15, 0.09)	0.06 (−0.07, 0.18)	−0.03 (−0.15, 0.10)	−0.05 (−0.17, 0.07)
Cohen’s d STboys-Cboys (95% C.I.)	−0.02 (−0.14, 0.10)	−0.02 (−0.14, 0.11)	−0.03 (−0.15, 0.09)	0.05 (−0.08, 0.17)	0.03 (−0.09, 0.16)	0.15 (0.02, 0.27)
Cohen’s d Girls-Boys (95% C.I.)	−0.37 (−0.46, −0.28)	−0.38 (−0.48, −0.29)	−0.35 (−0.44, −0.35)	0.29 (0.20, 0.37)	−0.20 (−0.29, −0.11)	−0.23 (−0.14, −0.32)
Greatest lower bound	–	.66	–	.93	.91	.67
Cronbach’s α	–	.59	–	.92	.86	.55
Skewness	−0.19	−0.19	−0.24	1.33	0.07	−0.06
Kurtosis	2.58	2.55	2.61	4.50	2.60	3.32

M.A. = math anxiety, D.I. = domain identification, G.I. = gender identification. The greatest lower bound (GLB) is calculated as the maximum value of three different estimation methods of the greatest lower bound with package “psych” in R.

Table 5. Main analyses: fit measures, deviance, unstandardized regression coefficients, and variance components for models without moderators.

		Fixed effect	Random part		Deviance (Dp−Dc) (df)	AIC		BIC
		Coefficient (S.E.)	t		Variance component
Model 0	Intercept	9.97 (0.20)	48.74	Level-two variance	3.04		11,312.1	11,318.1	11,335.0
				Level-one variance	13.00
Model 1	Intercept	10.73 (0.22)	48.70	Level-two variance	3.07		11,226.8 (85.3)* (1)	11,234.8	11,257.4
	Gender	−1.52 (0.16)	−9.33	Level-one variance	12.45
Model 2	Intercept	10.76 (0.23)	45.96	Level-two variance	3.07		11,226.7 (0.1) (1)	11,236.7	11,264.8
	Gender	−1.52 (0.16)	−9.34	Level-one variance	12.45
	ST	−0.06 (0.16)	−0.39
Model 3	Intercept	10.80 (0.25)	43.63	Level-two variance	3.07		11,226.4 (0.3) (1)	11,238.4	11,272.2
	Gender	−1.60 (0.23)	−7.07	Level-one variance	12.44
	ST	−0.14 (0.22)	−0.64
	ST × Gender	0.16 (0.31)	0.51
Model 4 (class-level predictors)	Intercept	10.10 (0.71)	14.18	Level-two variance	3.01		11,224.8 (1.6) (3)	11,242.8	11,293.5
	Gender	−1.61 (0.23)	−7.11	Level-one variance	12.44
	ST	−0.14 (0.22)	−0.64
	ST × Gender	0.16 (0.31)	0.51
	Prop. gender	0.92 (1.26)	0.73
	Gender teacher.d1	0.38 (0.42)	0.90
	Gender teacher.d2	0.95 (1.41)	0.68

AIC = Akaike information criterion; BIC = Bayesian information criterion. Gender is dummy coded with males being the reference group. ST is dummy coded with the control group being the reference group. Gender of the teacher is dummy coded, with male teachers being the reference group, dummy 1 for female teachers and dummy 2 for both female and male teachers. The difference in deviance between the previous model (D_p) and the current model (D_c) is given in brackets, and is χ² distributed. Models are fit with maximum likelihood estimation.

Table 6. Bayes factors for competing hypotheses.

	Hu	H0	H1	H2	HC
H0 (No threat hypothesis)	BF (H0, Hu) = 563.080	–	BF (H0, H1) = 28.177	BF (H0, H2) = 1,144.472	BF (H0, HC) = 481.677
H1 (Stereotype threat hypothesis)	BF (H1, Hu) = 19.984	BF (H1, H0) = 0.035	–	BF (H1, H2) = 40.618	BF (H1, HC) = 17.095
H2 (Stereotype threat and stereotype lift hypothesis)	BF (H2, Hu) = 0.492	BF (H2, H0) = 0.001	BF (H2, H1) = 0.025	–	BF (H2, HC) = 0.421
HC (Complement hypothesis)	BF (HC, Hu) = 1.169	BF (HC, H0) = 0.002	BF (HC, H1) = 0.058	BF (HC, H2) = 2.376	–

BF = Bayes factor.

Table 7. Main analyses: fit statistics and model comparison for moderating variables and stereotype threat.

	Math anxiety				Domain identification				Gender identification
	χ^2 (df)	p	AIC	BIC	χ^2 (df)	p	AIC	BIC	χ^2 (df)	p	AIC	BIC
Model 0 (random intercept)	–	–	11,199	11,216	–	–	10,953	10,970	–	–	10,933	10,950
Model 1 (moderator)	89.07 (1)	<.001	11,112	11,134	185.10 (1)	<.001	10,770	10,792	1.79 (1)	.18	10,933	10,956
Model 2 (Gender)	70.00 (1)	<.001	11,044	11,072	66.28 (1)	<.001	10,705	10,733	82.00 (1)	<.001	10,853	10,881
Model 3 (ST condition)	0.03 (1)	.86	11,046	11,079	0.56 (1)	.45	10,707	10,740	0.01 (1)	.92	10,855	10,889
Model 4 (ST × Gender)	0.49 (1)	.49	11,047	11,086	0.36 (1)	.55	10,708	10,748	0.12 (1)	.73	10,857	10,896
Model 5 (ST × Gender × Moderator)	3.60 (3)	.31	11,050	11,106	5.66 (3)	.13	10,709	10,765	3.15 (3)	.37	10,860	10,916
Model 6 (class-level predictors)	2.18 (3)	.54	11,053	11,126	1.29 (3)	.73	10,714	10,786	1.66 (3)	.64	10,864	10,937

AIC = Akaike information criterion; BIC = Bayesian information criterion.

Table 8. Unstandardized regression coefficients for models with moderators estimated with ML.

	Math anxiety					Domain identification					Gender identification
	Fixed effect				Random effect	Fixed effect				Random effect	Fixed effect				Random effect
	Coef.	t	S.E.		Variance component	Coef.	t	S.E.		Variance component	Coef.	t	S.E.		Variance component
Intercept	10.63	48.55	0.22	Lvl2	3.04	10.63	50.11	0.21	Lvl 2	2.83	10.74	48.54	0.22	Lvl 2	3.05
Moder.	−0.09	−8.44	0.01	Lvl1	11.96	0.12	13.32	0.01	Lvl 1	11.30	0.01	−9.15	0.03	Lvl 1	12.45
Gender	−1.36	−8.44	0.16			−1.30	−8.21	0.16			−1.52	0.29	0.17

Moder. = moderator; Coef. = unstandardized regression coefficient; Lvl = level.

Table 9. Final model: unstandardized regression coefficients and variance components for final model.

		Fixed effect		Random part
		Coefficient (S.E.)	t		Variance component
Model 3 (final model)	Intercept	10.520 (0.213)	49.33	Level-two variance	2.816
	Gender	−1.253 (0.158)	−7.94	Level-one variance	11.094
	Domain identification	0.078 (0.013)	6.08
	Math anxiety	−0.058 (0.015)	−3.91
	Gender × Domain identification	0.046 (0.019)	2.74
	Gender × Math anxiety	0.001 (0.020)	0.07
	Math anxiety × Domain identification	−0.004 (0.001)	−3.66

Gender is dummy coded with males being the reference group. ST is dummy coded with the control group being the reference group. Domain identification and math anxiety are grand mean centered. Models are fit with maximum likelihood estimation.