Foundations for improved vaccine correlate of risk analysis using positive-unlabeled learning

References

• World Health Organization. Correlates of vaccine- induced protection: method and implications. WHO/IVB/13.01. 2013.
   Google Scholar
• Qin L, Gilbert PB, Corey L, McElrath MJ, Self SG. A framework for assessing immunological correlates of protection in vaccine trials. J Infect Dis. 2007;196(9):1–11. doi:10.1086/522428.
   PubMed Web of Science ®Google Scholar
• Garber DA, Silvestri G, Feinberg MB. Prospects for an AIDS vaccine: three big questions, no easy answers. Lancet Infect Dis. 2004;4(7):397–413. doi:10.1016/S1473-3099(04)01056-4.
   PubMed Web of Science ®Google Scholar
• Haynes BF, Gilbert PB, McElrath MJ, Zolla-Pazner S, Tomaras GD, Alam SM, Evans DT, Montefiori DC, Karnasuta C, Sutthent R, et al. Immune-correlates analysis of an HIV-1 vaccine efficacy trial. N Engl J Med. 2012;366(14):1275–86. doi:10.1056/NEJMoa1113425.
   PubMed Web of Science ®Google Scholar
• Corey L, Gilbert PB, Tomaras GD, Haynes BF, Pantaleo G, Fauci AS. Immune correlates of vaccine protection against HIV-1 acquisition. Sci Transl Med. 2015;7(310):310rv7. doi:10.1126/scitranslmed.aac7732.
   PubMed Web of Science ®Google Scholar
• Mehta P, Bukov M, Wang CH, Day AGR, Richardson C, Fisher CK, Schwab DJ. A high-bias, low-variance introduction to machine learning for physicists. Phys Rep. 2019;810:1–124. doi:10.1016/j.physrep.2019.03.001.
   PubMed Web of Science ®Google Scholar
• Chicco D. Ten quick tips for machine learning in computational biology. BioData Min. 2017;10(1):35. doi:10.1186/s13040-017-0155-3.
   PubMedGoogle Scholar
• Chung AW, Kumar MP, Arnold KB, Yu WH, Schoen MK, Dunphy LJ, Suscovich T, Frahm N, Linde C, Mahan A, et al. Dissecting polyclonal vaccine-induced humoral immunity against HIV using systems serology. Cell. 2015;163(4):988–98. doi:10.1016/j.cell.2015.10.027.
   PubMed Web of Science ®Google Scholar
• Neidich SD, Fong Y, Li SS, Geraghty DE, Williamson BD, Young WC, Goodman D, Seaton KE, Shen X, Sawant S, et al. Antibody Fc effector functions and IgG3 associate with decreased HIV-1 risk. J Clin Invest. 2019;129(11):4838–49. doi:10.1172/JCI126391.
   PubMed Web of Science ®Google Scholar
• Ackerman ME, Das J, Pittala S, Broge T, Linde C, Suscovich TJ, Brown EP, Bradley T, Natarajan H, Lin S, et al. Route of immunization defines multiple mechanisms of vaccine-mediated protection against SIV. Nat Med. 2018;24(10):1590–8. doi:10.1038/s41591-018-0161-0.
   PubMed Web of Science ®Google Scholar
• Bradley T, Pollara J, Santra S, Vandergrift N, Pittala S, Bailey-Kellogg C, Shen X, Parks R, Goodman D, Eaton A, et al. Pentavalent HIV-1 vaccine protects against simian-human immunodeficiency virus challenge. Nat Commun. 2017;8(1):15711. doi:10.1038/ncomms15711.
   PubMedGoogle Scholar
• Pittala S, Bagley K, Schwartz JA, Brown EP, Weiner JA, Prado IJ, Zhang W, Xu R, Ota‐setlik A, Pal R, et al. Antibody Fab-Fc properties outperform titer in predictive models of SIV vaccine-induced protection. Mol Syst Biol. 2019;15(5):e8747. doi:10.15252/msb.20188747.
   PubMed Web of Science ®Google Scholar
• Vaccari M, Gordon SN, Fourati S, Schifanella L, Liyanage NP, Cameron M, Keele BF, Shen X, Tomaras GD, Billings E, et al. Corrigendum: adjuvant-dependent innate and adaptive immune signatures of risk of SIVmac251 acquisition. Nat Med. 2016;22(10):1192. doi:10.1038/nm1016-1192a.
   PubMed Web of Science ®Google Scholar
• Kazmin D, Nakaya HI, Lee EK, Johnson MJ, van der Most R, van den Berg RA, Ballou WR, Jongert E, Wille-Reece U, Ockenhouse C, et al. Systems analysis of protective immune responses to RTS,S malaria vaccination in humans. Proc Natl Acad Sci U S A. 2017;114(9):2425–30. doi:10.1073/pnas.1621489114.
   PubMed Web of Science ®Google Scholar
• Team H-CSP, Consortium H-I. Multicohort analysis reveals baseline transcriptional predictors of influenza vaccination responses. Sci Immunol. 2017;2(14). doi:10.1126/sciimmunol.aal4656.
   Web of Science ®Google Scholar
• Fourati S, Ribeiro SP, Blasco Tavares Pereira Lopes F, Talla A, Lefebvre F, Cameron M, Kaewkungwal J, Pitisuttithum P, Nitayaphan S, Rerks-Ngarm S, et al. Integrated systems approach defines the antiviral pathways conferring protection by the RV144 HIV vaccine. Nat Commun. 2019;10(1):863. doi:10.1038/s41467-019-08854-2.
   PubMedGoogle Scholar
• Lin L, Finak G, Ushey K, Seshadri C, Hawn TR, Frahm N, Scriba TJ, Mahomed H, Hanekom W, Bart P-A, et al. COMPASS identifies T-cell subsets correlated with clinical outcomes. Nat Biotechnol. 2015;33(6):610–6. doi:10.1038/nbt.3187.
   PubMed Web of Science ®Google Scholar
• Querec TD, Akondy RS, Lee EK, Cao W, Nakaya HI, Teuwen D, Pirani A, Gernert K, Deng J, Marzolf B, et al. Systems biology approach predicts immunogenicity of the yellow fever vaccine in humans. Nat Immunol. 2009;10(1):116–25. doi:10.1038/ni.1688.
   PubMed Web of Science ®Google Scholar
• Arevalillo JM, Sztein MB, Kotloff KL, Levine MM, Simon JK. Identification of immune correlates of protection in Shigella infection by application of machine learning. J Biomed Inform. 2017;74:1–9. doi:10.1016/j.jbi.2017.08.005.
   PubMed Web of Science ®Google Scholar
• Gitte Vanwinckelen HB. On estimating model accuracy with repeated cross-validation. BeneLearn 2012: Proceedings of the 21st Belgian-Dutch Conference on Machine Learning; 2012.
   Google Scholar
• Hawkins DM. The problem of overfitting. J Chem Inf Comput Sci. 2004;44(1):1–12. doi:10.1021/ci0342472.
   PubMedGoogle Scholar
• Subramanian J, Simon R. Overfitting in prediction models – is it a problem only in high dimensions? Contemp Clin Trials. 2013;36(2):636–41. doi:10.1016/j.cct.2013.06.011.
   PubMed Web of Science ®Google Scholar
• Han H, Jiang X. Overcome support vector machine diagnosis overfitting. Cancer Inform. 2014;13:145–58. doi:10.4137/CIN.S13875.
   PubMedGoogle Scholar
• Rolland M, Gilbert P. Evaluating immune correlates in HIV type 1 vaccine efficacy trials: what RV144 may provide. Aids Res Hum Retrov. 2012;28(4):400–4. doi:10.1089/aid.2011.0240.
   PubMed Web of Science ®Google Scholar
• Om K, Paquin-Proulx D, Montero M, Peachman K, Shen X, Wieczorek L, Beck Z, Weiner JA, Kim D, Li Y, et al. Adjuvanted HIV-1 vaccine promotes antibody-dependent phagocytic responses and protects against heterologous SHIV challenge. PLoS Pathog. 2020;16(9):e1008764. doi:10.1371/journal.ppat.1008764.
   PubMed Web of Science ®Google Scholar
• Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science. 2015;349(6245):255–60. doi:10.1126/science.aaa8415.
   PubMed Web of Science ®Google Scholar
• Batta M. Machine learning algorithms - a review. Int J Sci Res. 2020;9:381–6.
   Google Scholar
• Camargo G, Bugatti PH, Saito PTM. Active semi-supervised learning for biological data classification. PLoS One. 2020;15(8):e0237428. doi:10.1371/journal.pone.0237428.
   PubMed Web of Science ®Google Scholar
• Bekker J, Davis J. Learning from positive and unlabeled data: a survey. Mach Learn. 2020;109(4):719–60. doi:10.1007/s10994-020-05877-5.
   Web of Science ®Google Scholar
• Bing Liu WSL, Yu PS, Xiaoli L. Partially supervised classification of text documents. International Conference on Machine Learning; 2002;2:387–394.
   Google Scholar
• Team RC. R: a language and environment for statistical computing. Vienna (Austria): R Foundation for Statistical Computing; 2022.
   Google Scholar
• Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F. e1071: misc functions of the department of statistics, probability theory group (Formerly: e1071), TU Wien. R package version 1.7-11; 2022.
   Google Scholar
• Kuhn M. Building predictive models in R using the caret package. J Stat Softw. 2008;28(5):28. doi:10.18637/jss.v028.i05.
   Web of Science ®Google Scholar
• Chicco D, Totsch N, Jurman G. The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation. BioData Min. 2021;14(1). doi:10.1186/s13040-021-00244-z.
   Web of Science ®Google Scholar
• Torchiano M. effsize: efficient effect size computation. 2020.
   Google Scholar
• Bangzuo Zhang WZ. Reliable negative extracting based on kNN for learning from positive and unlabeled examples. J Comput. 2009;4:94–101.
   Google Scholar
• Karasavvas N, Billings E, Rao M, Williams C, Zolla-Pazner S, Bailer RT, Koup RA, Madnote S, Arworn D, Shen X, et al. The Thai phase III HIV type 1 vaccine trial (RV144) regimen induces antibodies that target conserved regions within the V2 loop of gp120. Aids Res Hum Retrov. 2012;28(11):1444–57. doi:10.1089/aid.2012.0103.
   PubMed Web of Science ®Google Scholar
• Hammer SM, Sobieszczyk ME, Janes H, Karuna ST, Mulligan MJ, Grove D, Koblin BA, Buchbinder SP, Keefer MC, Tomaras GD, et al. Efficacy trial of a DNA/rAd5 HIV-1 preventive vaccine. N Engl J Med. 2013;369(22):2083–92. doi:10.1056/NEJMoa1310566.
   PubMed Web of Science ®Google Scholar
• Orenstein WA, Bernier RH, Dondero TJ, Hinman AR, Marks JS, Bart KJ, Sirotkin B. Field evaluation of vaccine efficacy. Bull World Health Organ. 1985;63:1055–68.
   PubMed Web of Science ®Google Scholar
• Othus M, Zhang MJ, Gale RP. Clinical trials: design, endpoints and interpretation of outcomes. Bone Marrow Transplant. 2022;57(3):338–42. doi:10.1038/s41409-021-01542-0.
   PubMed Web of Science ®Google Scholar
• Kublin JG, Morgan CA, Day TA, Gilbert PB, Self SG, McElrath MJ, Corey L. HIV vaccine trials network: activities and achievements of the first decade and beyond. Clin Investig (Lond). 2012;2(3):245–54. doi:10.4155/cli.12.8.
   PubMedGoogle Scholar
• Follmann D, Duerr A, Tabet S, Gilbert P, Moodie Z, Fast P, Cardinali M, Self S. Endpoints and regulatory issues in HIV vaccine clinical trials: lessons from a workshop. J Acquir Immune Defic Syndr. 2007;44(1):49–60. doi:10.1097/01.qai.0000247227.22504.ce.
   PubMed Web of Science ®Google Scholar
• Gilbert PB, DeGruttola VG, Hudgens MG, Self SG, Hammer SM, Corey L. What constitutes efficacy for a human immunodeficiency virus vaccine that ameliorates viremia: issues involving surrogate end points in phase 3 trials. J Infect Dis. 2003;188(2):179–93. doi:10.1086/376449.
   PubMed Web of Science ®Google Scholar
• Rerks-Ngarm S, Paris RM, Chunsutthiwat S, Premsri N, Namwat C, Bowonwatanuwong C, Li SS, Kaewkungkal J, Trichavaroj R, Churikanont N, et al. Extended evaluation of the virologic, immunologic, and clinical course of volunteers who acquired HIV-1 infection in a phase III vaccine trial of ALVAC-HIV and AIDSVAX B/E. J Infect Dis. 2013;207(8):1195–205. doi:10.1093/infdis/jis478.
   PubMed Web of Science ®Google Scholar
• Rida W, Fast P, Hoff R, Fleming T. Intermediate-size trials for the evaluation of HIV vaccine candidates: a workshop summary. J Acq Immun Def Synd. 1997;16(3):195–203. doi:10.1097/00042560-199711010-00009.
   Web of Science ®Google Scholar