External validation of prognostic models for preeclampsia in a Dutch multicenter prospective cohort

Figures & data

Table 1. Baseline characteristics stratified per variable that were present for imputation ⁽¹¹⁾.

Characteristic	Cases with missing value n(%)	Complete questionnaires (n = 2614)	Cases with ≥ 1 missing questionnaire (n = 1122)	p value	Overall RESPECT cohort (n = 3736)
Age, yrs	168 (4.5%)	30.9 ± 4.3	30.7 ± 3.9	0.32	30.8 ± 4.2
BMI pre-pregnancy, kg/m^2	46 (1.2%)	23.3 [21.2, 26.3]	23.0 [20.9, 25.9]	0.02*	23.2 [21.1, 26.2]
BMI, kg/m^2	184 (4.9%)	23.8 [21.6, 26.9]	23.4 [21.3, 26.3]	0.01*	23.7 [21.5, 26.7]
Systolic BP, mmHg	65 (1.7%)	114 ± 12	115 ± 12	0.55	114 ± 12
Diastolic BP, mmHg	64 (1.7%)	67 ± 8	67 ± 8	0.10	67 ± 8
Etnicity, White African Asian Mixed Other	736 (19.7%)	1671 (89.0%) 18 (1.0%) 30 (1.6%) 44 (2.3%) 115 (4.4%)	1068 (95.2%) 2 (0.2%) 11 (1.0%) 15 (1.3%) 26 (2.3%)	<0.001*	3398 (91.0%) 31 (0.8%) 54 (1.4%) 77 (2.1%) 176 (4.7%)
Education, Low Middle High	225 (6.0%)	199 (7.6%) 826 (31.6%) 1364 (57.1%)	52 (4.6%) 364 (32.4%) 706 (62.9%)	0.001*	272 (7.3%) 1277 (34.2%) 2187 (58.6%)
Smoking during pregnancy	0 (0%)	260 (9.9%)	73 (6.5%)	0.001*	336 (9.0%)
History of chronic hypertension	1 (0.0%)	45 (1.7%)	15 (1.3%)	0.47	60 (1.6%)
Mother with PE	1 (0.0%)	75 (2.9%)	41 (3.7%)	0.24	116 (3.1%)
Method of conception Spontaneous Ovulation drugs IVF	30 (0.8%)	2407 (93.2%) 61 (2.4%) 82 (3.2%)	1035 (92.2%) 38 (3.4%) 28 (2.5%)	0.20	3469 (92.8%) 100 (2.7%) 111 (3.0%)
Nulliparous	4 (0.0%)	1149 (44.0%)	510 (45.5%)	0.44	1662 (44.5%)
History of PE	0 (0.0%)	86 (3.3%)	35 (3.1%)	0.87	121 (3.2%)
History of IUGR (<10^th perc)	0 (0.0%)	93 (3.6%)	39 (3.5%)	0.98	132 (3.5%)
Recurrent miscarriages (≥2)	4 (0.0%)	174 (6.7%)	59 (5.3%)	0.12	233 (6.2%)
History of fetal death	0 (0.0%)	58 (2.2%)	16 (1.4%)	0.14	74 (2.0%)
PE Late onset	265 (7.1%)	56 (2.4%) 45 (1.9%)	22 (2.0%) 20 (1.8%)	0.51 0.89	87 (2.3%) 71 (1.9%)
Gestational age at birth, days	344 (9.2%)	280 [273, 285]	280 [274, 286]	0.33	280 [273, 285]
Sex, male	360 (9.6%)	1159 (50.7%)	570 (52.3%)	0.41	1909 (51.1%)
Birth weight, g percentile <10^th percentile	374 (10.0%)	3503 [3200, 3855] 54 (30, 77) 138 (6.4%)	3540 [3219, 3880] 57 (32, 80) 63 (5.9%)	0.11 0.07 0.62	3520 [3190, 3875] 55 (30, 79) 270 (7.2%)

yrs, years; BMI, body mass index; BP, blood pressure; PE, preeclampsia; IVF, in vitro fertilization; IUGR, intra uterine growth restriction; Data are n, n(%), mean±SD, or median [IQR]. The column “overall RESPECT cohort” includes imputed data for those with missing values.

*Significant at the P < 0.05 level.

Table 2. Summary of predictors per model.

	Predictors	Maternal age	Weight	Length	BMI, prepregnancy	BMI	Blood pressure	History of PE	History of IUGR	Parity	Family history of PE	History of chronic hypertension	History of DM t 1/2	History of thrombo- embolic proces	Ethnicity	Method of conception	Smoking	Education level	a priori	PAPP-A MoM	PlGF MoM	Total
All preeclampsia	Baschat 2014						x	x		x		x								x		5
	Giguere 2014				x		x															2
	Goetzinger 2010					x						x	x		x					x		5
	Goetzinger 2013					x		x				x	x							x		5
	Myatt 2012					x	x								x			x				4
	Odibo 2011					x						x			x					x		4
	Plasencia 2007					x		x		x	x				x							5
	Poon 2008					x	x	x		x	x				x				x			7
	Syngelaki 2011	x				x		x		x	x	x	x		x	x	x					10
	Total all preeclampsia	1	0	0	1	7	4	5	0	4	3	5	3	0	6	1	1	1	1	4	0	5
Late onset preeclampsia	Akolekar 2011		x	x				x		x		x			x							6
	Crovetto 2014a					x		x				x			x		x					5
	Crovetto 2014b					x		x	x	x												4
	Kuc 2013	x	x							x												3
	Kuc 2014	x				x				x
	Plasencia 2007					x		x		x	x				x							5
	Poon 2009					x		x		x	x				x					x		6
	Poon 2010	x				x		x		x	x				x							6
	Scazzochio 2013					x				x		x	x	x								5
	Total late onset preeclampsia	3	2	1	0	7	0	6	1	8	3	3	1	1	5	0	1	0	0	1	0	5

All predictors are measured in the first trimester at the initial prenatal visit, unless otherwise specified. Modifications to the original predictors are described in Appendix A. Abbreviations: BMI, body mass index; PE, preeclampsia; IUGR, intra-uterine growth restriction; DM, diabetes mellitus; PAPP-A, pregnancy-associated plasma protein-A; PlGF, placental growth factor.

Figure 1. Calibration plots of original prognostic models.

In case of perfect calibration all groups of predicted probabilities fit close to the diagonal line, corresponding with an intercept of 0 and a slope of 1 for the calibration plot. Vertical lines in grouped observed represent 95% confidence intervals.

Figure 2. Calibration plots of recalibrated prognostic models.

In case of perfect calibration all groups of predicted probabilities fit close to the diagonal line, corresponding with an intercept of 0 and a slope of 1 for the calibration plot. Vertical lines in grouped observed represent 95% confidence intervals.

Table 3. Discriminative ability of all prognostic models in the external validation.

Prognostic model	AUC Development	AUC Recalibrated	AUC Refitted	AUC Nulliparous Recalibrated	AUC Nulliparous Refitted
ALL PE
Baschat ‘14	0.82 [0.78 to 0.86]	0.68 [0.61 to 0.74]	0.76 [0.71 to 0.81]	0.63 [0.55 to 0.71]	0.64 [0.56 to 0.72]
Giguere ‘14	0.75 [0.69 to 0.81]	0.63 [0.57 to 0.69]	0.64 [0.58 to 0.71]	0.61 [0.54 to 0.69]	0.63 [0.55 to 0.71]
Goetzinger ‘10	0.70 [0.65 to 0.72]	0.55 [0.48 to 0.61]	0.55 [0.49 to 0.61]	0.50 [0.43 to 0.57]	0.50 [0.43 to 0.57]
Goetzinger ‘13	0.76 [0.69 to 0.83]	0.56 [0.50 to 0.61]	0.56 [0.50 to 0.61]	0.52 [0.46 to 0.57]	0.52 [0.46 to 0.57]
Myatt ‘12	0.65 [0.61 to 0.69]	0.64 [0.58 to 0.70]	0.64 [0.58 to 0.70]	0.61 [0.53 to 0.68]	0.62 [0.54 to 0.69]
Odibo ‘11	0.77 [0.63 to 0.81]	0.56 [0.49 to 0.62]	0.57 [0.50 to 0.63]	0.52 [0.45 to 0.59]	0.53 [0.46 to 0.61]
Plasencia ‘07	0.81 [0.80 to 0.82]	0.72 [0.67 to 0.77]	0.73 [0.68 to 0.78]	0.53 [0.45 to 0.59]	0.54 [0.47 to 0.62]
Poon ‘08	0.85 [NR]	0.71 [0.66 to 0.76]	0.76 [0.71 to 0.81]	0.51 [0.43 to 0.59]	0.63 [0.55 to 0.71]
Syngelaki ‘11	NR	0.72 [0.67 to 0.78]	0.75 [0.70 to 0.80]	0.55 [0.47 to 0.63]	0.59 [0.51 to 0.67]
LATE ONSET PE
Akolekar ‘11	NR	0.71 [0.65 to 0.77]	0.72 [0.66 to .78]	0.53 [0.45 to 0.62]	0.54 [0.45 to 0.62]
Crovetto ’14a	0.72 [0.69 to 0.76]	0.66 [0.59 to 0.72]	0.73 [0.67 to 0.79]	0.48 [0.40 to 0.57]	0.57 [0.48 to 0.66]
Crovetto ’14b	0.75 [0.67 to 0.82]	0.58 [0.51 to 0.65]	0.58 [0.50 to 0.65]	0.53 [0.45 to 0.62]	0.53 [0.45 to 0.62]
Kuc ‘13	NR	0.66 [0.60 to 0.73]	0.68 [0.62 to 0.74]	0.53 [0.45 to 0.61]	0.53 [0.45 to 0.61]
Kuc ‘14	0.79 NR	0.67 [0.61 to 0.74]	0.68 [0.62 to 0.74]	0.59 [0.51 to 0.68]	0.60 [0.52 to 0.68]
Plasencia ‘07	0.80 [0.79 to 0.81]	0.58 [0.51 to 0.65]	0.60 [0.53 to 0.67]	0.53 [0.45 to 0.61]	0.55 [0.47 to 0.63]
Poon ’09	0.79 [0.78 to 0.80]	0.70 [0.64 to 0.76]	0.73 [0.67 to 0.79]	0.52 [0.43 to 0.60]	0.57 [0.49 to 0.66]
Poon ‘10	0.80 [0.76 to 0.83]	0.70 [0.64 to 0.77]	0.73 [0.67 to 0.79]	0.53 [0.45 to 0.61]	0.55 [0.47 to 0.63]
Scazzochio ‘13	0.71 [0.66 to 0.76]	0.67 [0.61 to 0.73]	0.69 [0.62 to 0.75]	0.54 [0.45 to 0.62]	0.57 [0.48 to 0.66]

The AUC “development” shows the AUC as reported in the original publication if available. The AUC “recalibrated” shows the AUC per model, recalibrated to the RESPECT cohort. The AUC “refitted” shows the AUC per model after complete refitting of the prognostic model to the RESPECT cohort. The AUC “Nulliparous” shows the AUC per model when applied to a subgroup of only nulliparous.

Abbreviations: AUC, area under the receiver operating characteristic curve; PE, preeclampsia; NR, not reported.

Data are presented in mean [95% confidence interval].

Table 4. Pregnancy outcome per predicted risk category.

All PE	Baschat 2014						Poon 2008						Syngelaki 2011
Predicted risk %	no PE	%	FPR	PE	%	DR	no PE	%	FPR	PE	%	DR	no PE	%	FPR	PE	%	DR
>5.0	369	10	10	35	40	40	309	8	8	21	24	24	398	11	11	33	38	38
4.5 to < 5.0	135	4	14	3	4	44	115	3	12	6	6	31	119	3	14	6	7	45
4.0 to < 4.5	195	5	19	10	11	55	214	6	17	7	8	39	173	5	19	5	6	50
3.5 to < 4.0	244	7	26	5	6	61	397	11	28	17	19	58	211	6	25	6	7	57
3.0 to < 3.5	264	7	33	7	8	69	509	14	42	20	23	82	256	7	32	10	12	69
2.5 to < 3.0	230	6	39	8	10	79	128	4	46	3	4	85	251	7	39	8	9	79
2.0 to < 2.5	188	5	45	6	6	85	5	0	46	0	0	85	201	5	44	5	5	84
1.5 to < 2.0	96	3	47	0	0	85	6	0	46	1	1	86	104	3	47	2	3	86
1.0 to < 1.5	170	5	52	2	2	88	75	2	48	0	0	86	186	5	52	1	2	88
0.5 to < 1.0	1105	30	82	9	10	98	1669	46	94	12	13	100	1068	29	81	9	10	98
<0.5	653	18	100	2	2	100	222	6	100	0	0	100	682	19	100	2	2	100
Late onset PE	Akolekar 2011						Kuc 2014
Predicted risk %	no PE	%	FPR	PE	%	DR	no PE	%	FPR	PE	%	DR
>5.0	349	10	10	25	36	36	108	3	3	5	7	7
4.5 to < 5.0	92	3	12	4	6	41	32	1	4	5	7	14
4.0 to < 4.5	146	4	16	3	5	46	58	2	5	1	2	15
3.5 to < 4.0	217	6	22	6	8	54	66	2	7	4	5	20
3.0 to < 3.5	442	12	34	9	13	67	132	4	11	5	6	27
2.5 to < 3.0	407	11	45	11	15	82	233	6	17	7	10	36
2.0 to < 2.5	37	1	46	1	1	83	400	11	28	14	20	57
1.5 to < 2.0	35	1	47	0	0	83	793	22	50	13	18	74
1.0 to < 1.5	91	2	49	1	2	85	1614	44	94	18	26	100
0.5 to < 1.0	1481	40	90	10	14	99	231	6	100	0	0	100
<0.5	371	10	100	1	1	100	0	0	100	0	0	100

PE, preeclampsia; FPR, false positive rate; DR, detection rate