Prediction of serum HIV-1 neutralization titers of VRC01 in HIV-uninfected Antibody Mediated Prevention (AMP) trial participants

Figures & data

Table 1. Summary table of the four approaches for predicting serum neutralization titer of VRC01. For concreteness, serum ID50 titers are considered but all descriptions apply to the prediction of either ID50 or ID80 titers

Approach for Predicting SerumID50 Titers	Summary*	Compared to Approaches 1–3 in ref #5	Pros	Cons
Approach 1	$\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{I}\mathrm{D}50_{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{d}}=\frac{\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{c}}_{\mathrm{M}\mathrm{e}\mathrm{a}\mathrm{s}}}{\mathrm{C}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{t}\mathrm{I}\mathrm{C}50}$	Same as Approach 1 in;^Citation5 Serum ConcMeas < LLoQ replaced by LLoQ/2 in both articles.	Less resource-intensive: does not require ID50 measured on any serum sample; requires serum concentration measured in the given sample only Simplest among all four approaches	Requires the availability of clinical lot IC50 In general, prediction performance not as high as Approaches 3 and 4
Approach 2	$\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{I}\mathrm{D}50_{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{d}}=\frac{\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{c}}_{\mathrm{p}\mathrm{o}\mathrm{p}\mathrm{P}\mathrm{K}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}}{\mathrm{C}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{t}\mathrm{I}\mathrm{C}50}$	Previous work in Ref^Citation5: Serum ConcMeas < LLoQ replaced by LLoQ/2 in the popPK model. This current work: Likelihood method used to account for serum ConcMeas < LLoQ in the popPK model.	Less resource-intensive: does not require ID50 measured on any serum sample Generally, as good as or better performance than Approach 1	Requires the availability of clinical lot IC50 Compared to Approach 1, Approaches 2–4 require serum concentrations measured at multiple time-points to permit popPK modeling In general, prediction performance not as high as Approaches 3 and 4
Approach 3	popPK modeling of measured longitudinal serum concentrations from all individuals; PD modeling of measured longitudinal serum concentrations and ID50 titers from individuals who have both types of data available; Serum ID50Pred obtained by plugging in popPK model-predicted serum concentration from step 1 into the PD model of step 2.	Previous work in Ref^Citation5: Serum ConcMeas < LLoQ replaced by LLoQ/2 in the popPK model; ID50Meas < LLoQ replaced by LLoQ/2 in the PD model. This current work: Likelihood method used to account for serum ConcMeas <LLoQ, and ID50Meas <LLoQ in the popPK and PD models.	Compared to Approaches 1 & 2, does not require availability of the clinical lot IC50 Generally, better prediction performance than Approaches 1 and 2	More resource-intensive: in addition to serum concentrations measured at multiple time-points to permit popPK modeling, requires ID50 measured on a subset of those PK serum samples to permit PD modeling
Approach 4	popPK modeling of measured longitudinal serum concentrations from all individuals; calculate serum IIP for all individuals; PD modeling of calculated longitudinal serum IIP and ID50 titers from individuals who have both types of data available; Serum ID50Pred obtained by plugging in calculated IIP from step 1 into the PD model of step 2.	NA (newly described here)	Accounts for not only serum concentrations but also the slope of the dose–response neutralization curve of the clinical lot product Generally, better prediction performance than Approaches 1 and 2	More resource-intensive: in addition to serum concentrations measured at multiple time-points to permit popPK modeling, requires ID50 measured on a subset of those PK serum samples to permit PD modeling Compared to Approach 3, additionally requires the availability of both IC50 and IC80 to permit the calculation of IIP as the predictor of ID50 titers

*Intended as an overview summary of each Approach; please see Methods for the full statistical description.

Conc = concentration; IIP = instantaneous inhibitory potential; Meas = measured; Pred = predicted; PK/PD = pharmacokinetics/pharmacodynamics.

Table 2. Performance of approaches 1–4 for predicting ID50 (Panel A) and ID80 (Panel B) neutralization titers broken down by the status (detectable vs. undetectable, + vs. -) of measured serum concentration and serum neutralization titer

	ID50
		Detectable
A Virus	IC50 (µg/ml)*	Serum conc.^a	Serum neut. Titer^b	# of samples	Appr. 1	Appr. 2	Appr. 3	Appr. 4	Reported Values
H704_2544_140eN01	1.02	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)^c
		+	−	95	0.72 (0.62, 0.80)	0.73 (0.63, 0.81)	0.74 (0.64, 0.82)	0.74 (0.64, 0.82)	Classification accuracy (95% CI)
		+	+	127	0.68 (0.48, 0.82)	0.65 (0.42, 0.80)	0.77 (0.61, 0.87)	0.76 (0.6, 0.86)	CCCrm^d (95% CI)
H703_1750_140Es	0.92	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	94	0.66 (0.56, 0.75)	0.66 (0.56, 0.75)	0.69 (0.59, 0.77)	0.68 (0.58, 0.77)	Classification accuracy (95% CI)
		+	+	128	0.70 (0.52, 0.82)	0.69 (0.48, 0.83)	0.85 (0.76, 0.91)	0.85 (0.76, 0.91)	CCCrm (95% CI)
H703_1471_190s	0.19	-	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	9	NA^e	NA^e	NA^e	NA^e	Classification accuracy
		+	+	213	0.80 (0.68, 0.88)	0.77 (0.63, 0.86)	0.87 (0.8, 0.92)	0.85 (0.77, 0.91)	CCCrm (95% CI)
H704_1535_030sN	0.12	−	−	22	1.00 (0.85, 1.00)	0.86 (0.66, 0.95)	1.00 (0.85, 1.00)	0.91 (0.72, 0.98)	Classification accuracy (95% CI)
		−	+	2	NA^e	NA^e	NA^e	NA^e	NA^e
		+	−	6	NA^e	NA^e	NA^e	NA^e	NA^e
		+	+	216	0.75 (0.61, 0.85)	0.74 (0.59, 0.84)	0.86 (0.78, 0.92)	0.82 (0.71, 0.89)	CCCrm (95% CI)
Breakthrough virus geometric mean^f	0.38	−	−	22	1.00 (0.85, 1.00)	1.00 (0.85, 1.00)	1.00 (0.85, 1.00)	1.00 (0.85, 1.00)	Classification accuracy (95% CI)
		−	+	2	NA^e	NA^e	NA^e	NA^e	NA^e
		+	-	4	NA^e	NA^e	NA^e	NA^e	NA^e
		+	+	218	0.78 (0.65, 0.86)	0.72 (0.57, 0.82)	0.85 (0.75, 0.91)	0.86 (0.77, 0.92)	CCCrm (95% CI)
PVO.4	0.57	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	64	0.50 (0.38, 0.62)	0.52 (0.40, 0.64)	0.70 (0.58, 0.80)	0.59 (0.47, 0.70)	Classification accuracy (95% CI)
		+	+	158	0.59 (0.36, 0.76)	0.57 (0.33, 0.74)	0.89 0.81, 0.93)	0.86 (0.78, 0.92)	CCCrm (95% CI)
CH0505TF.gly4	0.002	−	+	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	0.96 (0.80, 0.99)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	+	222	0.62 (0.43, 0.76)	0.65 (0.47, 0.78)	0.82 (0.74, 0.88)	0.74 (0.59, 0.84)	CCCrm (95% CI)
	ID80
		Detectable
B Virus	IC80 (µg/ml)*	Serum conc.^a	Serum neut. Titer^b	# of samples	Appr. 1	Appr. 2	Appr. 3	Appr. 4
H704_2544_140eN01	2.98	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	156	0.88 (0.82, 0.92)	0.88 (0.82, 0.92)	0.87 (0.81, 0.91)	0.91 (0.85, 0.95)	Classification accuracy (95% CI)
		+	+	66	0.80 (0.62, 0.9)	0.83 (0.69, 0.91)	0.73 (0.54, 0.85)	0.81 (0.67, 0.9)	CCCrm (95% CI)
H703_1750_140Es	2.64	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	157	0.83 (0.76, 0.88)	0.84 (0.77, 0.89)	0.85 (0.79, 0.90)	0.90 (0.84, 0.94)	Classification accuracy (95% CI)
		+	+	65	0.83 (0.67, 0.91)	0.86 (0.71, 0.94)	0.78 (0.61, 0.88)	0.86 (0.74, 0.92)	CCCrm (95% CI)
H703_1471_190s	0.57	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	53	0.68 (0.55, 0.79)	0.70 (0.57, 0.81)	0.91 (0.80, 0.96)	0.74 (0.61, 0.84)	Classification accuracy (95% CI)
		+	+	169	0.85	0.82	0.89	0.87	CCCrm (95% CI)
H704_1535_030sN	0.33	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	-	34	0.38 (0.24, 0.55)	0.38 (0.24, 0.55)	0.74 (0.57, 0.86)	0.44 (0.29, 0.60)	Classification accuracy (95% CI)
		+	+	188	0.81 (0.69, 0.89)	0.78 (0.65, 0.87)	0.87 (0.8, 0.92)	0.84 (0.75, 0.9)	CCCrm (95% CI)
Breakthrough virus geometric mean^f	1.10	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	34	1.00 (0.90, 1.00)	1.00 (0.90, 1.00)	1.00 (0.90, 1.00)	0.97 (0.85, 0.99)	Classification accuracy (95% CI)
		+	+	188	0.70 (0.57, 0.8)	0.63 (0.48, 0.75)	0.83 (0.71, 0.91)	0.85 (0.74, 0.92)	CCCrm (95% CI)
PVO.4	1.74	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	122	0.77 (0.69, 0.84)	0.75 (0.67, 0.82)	0.75 (0.67, 0.82)	0.82 (0.74, 0.88)	Classification accuracy (95% CI)
		+	+	100	0.68 (0.46, 0.82)	0.71 (0.51, 0.84)	0.88 (0.79, 0.93)	0.90 (0.83, 0.94)	CCCrm (95% CI)
CH0505TF.gly4	0.005	−	−	1	NA^e	NA^e	NA^e	NA^e	NA^e
		−	+	23	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	0.96 (0.80, 0.99)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	+	222	0.84 (0.71, 0.91)	0.87 (0.77, 0.92)	0.89 (0.83, 0.93)	0.86 (0.78, 0.92)	CCCrm (95% CI)

*IC50 and IC80 values are for the clinical lot of VRC01, calculated as the geometric mean of all replicates.

^aDetectable serum concentration (+) = Serum concentration > 1.0 µg/mL;

No detectable serum concentration (-) = Serum concentration ≤ 1.0 µg/mL.

^bDetectable serum neutralization titer (+) = Serum neutralization titer > 10;

No detectable serum neutralization titer (-) = Serum neutralization titer ≤ 10.

^cClassification accuracy = percent of samples correctly predicted to have neutralization titer > 10 or ≤ 10 for samples with empirically measured neutralization titer > 10 or ≤ 10, respectively.

^dCCC_rm = Concordance Correlation Coefficient for repeated measurements, expressing how well the predicted and measured neutralization titers agree with each other. Only calculated for samples with detectable VRC01 serum concentration and VRC01 serum neutralization titer.

^eNA = Not applicable. Results are only shown in a row if n ≥ 10.

Appr. = Approach.

^fGeometric mean: predict the geometric mean titers based on the geometric mean of the observed titers across the 4 breakthrough viruses, and the geometric mean of the IC50 values (Panel A) or of the IC80 values (Panel B) of the 4 breakthrough viruses (as if the geometric mean corresponded to a single virus).

Figure 1. Plots of observed vs. predicted serum neutralization titer for each approach among double positives [category (4) samples]. (a) ID50, (b) ID80. Values in the lower right-hand corner of each plot correspond to CCCrm agreement values, with 95% confidence intervals shown in parentheses. Double positive = participant with detectable (>LLoQ) serum concentration of VRC01 and detectable (>LLoQ) serum VRC01-mediated neutralization. Breakthrough virus geometric mean: predict the geometric mean titer based on the geometric mean of the observed titers across the 4 breakthrough viruses, and the geometric mean of the IC50 values (Panel A) or of the IC80 values (Panel B) of the 4 breakthrough viruses (as if the geometric mean corresponded to a single virus).

Figure 2. Plots of fold difference (Fold diff.), relative fold difference (Rel. fold diff.), and relative mean squared error (Rel. MSE) for each approach predicting (a) serum ID50 neutralization titers and (b) serum ID80 neutralization titers. Serum neutralization titers were predicted among double positives [category (4) samples] for each of the six tested viruses and for the breakthrough virus geometric mean. Double positive = participant with detectable (>LLoQ) serum concentration of VRC01 and detectable (>LLoQ) neutralization of VRC01. Breakthrough virus geometric mean: predict the geometric mean titer based on the geometric mean of the observed titers across the 4 breakthrough viruses, and the geometric mean of the IC50 values (Panel A) or of the IC80 values (Panel B) of the 4 breakthrough viruses (as if the geometric mean corresponded to a single virus).

Pegu A, Borate B, Huang Y, Pauthner MG, Hessell AJ, Julg B, Doria-Rose NA, Schmidt SD, Carpp LN, Cully MD, et al. A meta-analysis of passive immunization studies shows that serum-neutralizing antibody titer associates with protection against SHIV challenge. Cell Host Microbe. 2019;26:336–46 e3. doi:10.1016/j.chom.2019.08.014.

 PubMed Web of Science ®Google Scholar