230
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Development and validation of machine-learning algorithms predicting retention, overdoses, and all-cause mortality among US military veterans treated with buprenorphine for opioid use disorder

, PharmD, PhD, MPHORCID Icon, , MS, , MA, PhDORCID Icon, , PharmD, PhD, , MD, MPHORCID Icon, , PhD, MPH, , PharmD, PhD & , PhD show all

References

  • Hedegaard H, Miniño AM, Spencer MR, Warner M. Drug overdose deaths in the United States, 1999–2020 (NCHS Data Brief. no 428). Hyattsville, MD: National Center for Health Statistics; 2021.
  • Jones CM, Einstein EB, Compton WM. Changes in synthetic opioid involvement in drug overdose deaths in the United States, 2010-2016. JAMA. 2018;319(17):1819–21. Doi:https://dx.doi.org/10.15620/cdc:112340.
  • O'Donnell J, Gladden RM, Mattson CL, Hunter CT, Davis NL. Vital signs: characteristics of drug overdose deaths involving opioids and stimulants—24 states and the District of Columbia, January–June 2019. MMWR Morb Mortal Wkly Rep. 2020;69(35):1189–97. doi:10.15585/MMWR.MM6935A1.
  • Drug Overdose Deaths | Drug Overdose | CDC Injury Center [Accessed 2023 Sep 7]. https://www.cdc.gov/drugoverdose/deaths/index.html.
  • JEC Analysis Finds Opioid Epidemic Cost U.S. Nearly $1.5 Trillion in 2020 - JEC Analysis Finds Opioid Epidemic Cost U.S. Nearly $1.5 Trillion in 2020 - United States Joint Economic Committee [Accessed 2023 Sep 7]. https://www.jec.senate.gov/public/index.cfm/democrats/2022/9/jec-analysis-finds-opioid-epidemic-cost-u-s-nearly-1-5-trillion-in-2020.
  • Gordon AJ, Trafton JA, Saxon AJ, Gifford AL, Goodman F, Calabrese VS, McNicholas L, Liberto J, Buprenorphine Work Group of the Substance Use Disorders Quality Enhancement Research Initiative. Implementation of buprenorphine in the Veterans Health Administration: results of the first 3 years. Drug Alcohol Depend. 2007;90(2-3):292–6. doi:10.1016/j.drugalcdep.2007.03.010.
  • Wyse JJ, Gordon AJ, Dobscha SK, Morasco BJ, Tiffany E, Drexler K, Sandbrink F, Lovejoy TI. Medications for opioid use disorder in the Department of Veterans Affairs (VA) health care system: historical perspective, lessons learned, and next steps. Subst Abus. 2018;39(2):139–44. doi:10.1080/08897077.2018.1452327.
  • Mattick RP, Breen C, Kimber J, Davoli M. Buprenorphine maintenance versus placebo or methadone maintenance for opioid dependence. Cochrane Database Syst Rev. 2014; 2014(2):CD002207. doi:10.1002/14651858.CD002207.pub4.
  • Schwartz RP, Gryczynski J, O'Grady KE, Sharfstein JM, Warren G, Olsen Y, Mitchell SG, Jaffe JH. Opioid agonist treatments and heroin overdose deaths in Baltimore, Maryland, 1995–2009. Am J Public Health. 2013;103(5):917–22. doi:10.2105/AJPH.2012.301049.
  • National Academies of Sciences, Engineering, and Medicine. Medications for opioid use disorder save lives. Washington, DC: The National Academies Press; 2019.
  • Timko C, Schultz NR, Cucciare MA, Vittorio L, Garrison-Diehn C. Retention in medication-assisted treatment for opiate dependence: a systematic review. J Addict Dis. 2016;35(1):22–35. doi:10.1080/10550887.2016.1100960.
  • Bart G. Maintenance medication for opiate addiction: the foundation of recovery. J Addict Dis. 2012;31(3):207–25. doi:10.1080/10550887.2012.694598.
  • Gordon AJ, Drexler K, Hawkins EJ, Burden J, Codell NK, Mhatre-Owens A, Dungan MT, Hagedorn H. Stepped Care for Opioid Use Disorder Train the Trainer (SCOUTT) initiative: expanding access to medication treatment for opioid use disorder within Veterans Health Administration facilities. Subst Abus. 2020;41(3):275–82. doi:10.1080/08897077.2020.1787299.
  • Hagedorn H, Kenny M, Gordon AJ, Ackland PE, Noorbaloochi S, Yu W, Harris AHS. Advancing pharmacological treatments for opioid use disorder (ADaPT-OUD): protocol for testing a novel strategy to improve implementation of medication-assisted treatment for veterans with opioid use disorders in low-performing facilities. Addict Sci Clin Pract. 2018;13(1):25. doi:10.1186/s13722-018-0127-z.
  • Wyse JJ, Shull S, Lindner S, Morasco BJ, Gordon AJ, Carlson KF, Korthuis PT, Ono SS, Liberto JG, Lovejoy TI. Access to medications for opioid use disorder in rural versus urban Veterans Health Administration facilities. J Gen Intern Med. 2023;38(8):1871–1876. doi:10.1007/s11606-023-08027-4.
  • Gordon AJ, Saxon AJ, Kertesz S, Wyse JJ, Manhapra A, Lin LA, Chen W, Hansen J, Pinnell D, Huynh T, et al. Buprenorphine use and courses of care for opioid use disorder treatment within the Veterans Health Administration. Drug Alcohol Depend. 2023;248:109902. doi:10.1016/J.DRUGALCDEP.2023.109902.
  • Teruya C, Schwartz RP, Mitchell SG, Hasson AL, Thomas C, Buoncristiani SH, Hser Y-I, Wiest K, Cohen AJ, Glick N, et al. Patient perspectives on buprenorphine/naloxone: a qualitative study of retention during the starting treatment with agonist replacement therapies (START) study. J Psychoactive Drugs. 2014;46(5):412–26. doi:10.1080/02791072.2014.921743.
  • National Quality Forum. NQF: Behavioral Health 2016-2017 Final Report; 2017 [Accessed 2019 Oct 8]. https://www.qualityforum.org/Publications/2017/08/Behavioral_Health_2016-2017_Final_Report.aspx
  • Oliva EM, Bowe T, Tavakoli S, Martins S, Lewis ET, Paik M, Wiechers I, Henderson P, Harvey M, Avoundjian T, et al. Development and applications of the Veterans Health Administration’s Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safety and prevent overdose and suicide. Psychol Serv. 2017;14(1):34–49. doi:10.1037/ser0000099.
  • REACH VET recovery engagement and coordination for health-veterans enhanced treatment predictive analytics for suicide prevention program overview; 2018 [Accessed 2019 Sep 5]. https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/3527-notes.pdf.
  • Lehnert EA, Wilt G, Flanagan B, Hallisey E. Spatial exploration of the CDC’s Social Vulnerability Index and heat-related health outcomes in Georgia. Int J Disaster Risk Reduct. 2020;46:101517. doi:10.1016/J.IJDRR.2020.101517.
  • Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HCW, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. doi:10.1136/BMJ.H5527.
  • Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015;162(1):55–63. doi:10.7326/M14-0697.
  • Lagisetty P, Garpestad C, Larkin A, Macleod C, Antoku D, Slat S, Thomas J, Powell V, Bohnert ASB, Lin LA, et al. Identifying individuals with opioid use disorder: validity of International Classification of Diseases diagnostic codes for opioid use, dependence and abuse. Drug Alcohol Depend. 2021;221:108583. doi:10.1016/J.DRUGALCDEP.2021.108583.
  • Fairley M, Humphreys K, Joyce VR, Bounthavong M, Trafton J, Combs A, Oliva EM, Goldhaber-Fiebert JD, Asch SM, Brandeau ML, et al. Cost-effectiveness of treatments for opioid use disorder. JAMA Psychiatry. 2021;78(7):767–77. doi:10.1001/JAMAPSYCHIATRY.2021.0247.
  • Olfson M, Zhang Shu V, Schoenbaum M, King M. Trends in buprenorphine treatment in the United States, 2009-2018. JAMA. 2020;323(3):276–7. doi:10.1001/JAMA.2019.18913.
  • Samples H, Williams AR, Olfson M, Crystal S. Risk factors for discontinuation of buprenorphine treatment for opioid use disorders in a multi-state sample of Medicaid enrollees. J Subst Abuse Treat. 2018;95:9–17. doi:10.1016/J.JSAT.2018.09.001.
  • USING DATA TO PREVENT VETERAN MORTALITY. A guide to accessing & understanding mortality data in the VA mortality data repository [Accessed 2023 Oct 2]. www.mentalhealth.va.gov/.
  • Williams AR, Samples H, Crystal S, Olfson M. Acute care, prescription opioid use, and overdose following discontinuation of long-term buprenorphine treatment for opioid use disorder. Am J Psychiatry. 2020;177(2):117–24. doi:10.1176/appi.ajp.2019.19060612.
  • Charlson ME, Charlson RE, Peterson JC, Marinopoulos SS, Briggs WM, Hollenberg JP. The Charlson comorbidity index is adapted to predict costs of chronic disease in primary care patients. J Clin Epidemiol. 2008;61(12):1234–40. doi:10.1016/j.jclinepi.2008.01.006.
  • Cacciola JS, Alterman AI, Dephilippis D, Drapkin ML, Valadez C, Fala NC, Oslin D, McKay JR. Development and initial evaluation of the Brief Addiction Monitor (BAM). J Subst Abuse Treat. 2013;44(3):256–63. doi:10.1016/j.jsat.2012.07.013.
  • Weiner SG, El Ibrahimi S, Hendricks MA, Hallvik SE, Hildebran C, Fischer MA, Weiss RD, Boyer EW, Kreiner PW, Wright DA, et al. Factors associated with opioid overdose after an initial opioid prescription. JAMA Netw Open. 2022;5(1):e2145691-e2145691. doi:10.1001/JAMANETWORKOPEN.2021.45691.
  • Lyons RM, Yule AM, Schiff D, Bagley SM, Wilens TE. Risk factors for drug overdose in young people: a systematic review of the literature. J Child Adolesc Psychopharmacol. 2019;29(7):487–97. doi:10.1089/CAP.2019.0013.
  • Duca JV, Rosenthal SS. Borrowing constraints and access to owner-occupied housing. Reg Sci Urban Econ. 1994;24(3):301–22. doi:10.1016/0166-0462(93)02041-Z.
  • Kaplan D, Venezky RL. Literacy and voting behavior: a bivariate probit model with sample selection. Soc Sci Res. 1994;23(4):350–67. doi:10.1006/ssre.1994.1014.
  • Alford DP, LaBelle CT, Kretsch N, Bergeron A, Winter M, Botticelli M, Samet JH. Collaborative care of opioid-addicted patients in primary care using buprenorphine. Arch Intern Med. 2011;171(5):425–31. doi:10.1001/archinternmed.2010.541.
  • Hui D, Weinstein ZM, Cheng DM, Quinn E, Kim H, Labelle C, Samet JH. Very early disengagement and subsequent re-engagement in primary care Office Based Opioid Treatment (OBOT) with buprenorphine. J Subst Abuse Treat. 2017;79:12–9. doi:10.1016/j.jsat.2017.05.010.
  • Cousins SJ, Radfar SR, Crèvecoeur-MacPhail D, Ang A, Darfler K, Rawson RA. Predictors of continued use of extended-released naltrexone (XR-NTX) for opioid-dependence: an analysis of heroin and non-heroin opioid users in Los Angeles County. J Subst Abuse Treat. 2016;63:66–71. doi:10.1016/J.JSAT.2015.12.004.
  • Sullivan MA, Rothenberg JL, Vosburg SK, Church SH, Feldman SJ, Epstein EM, Kleber HD, Nunes EV. Predictors of retention in naltrexone maintenance for opioid dependence: analysis of a Stage I trial. Am J Addict. 2006;15(2):150–9. doi:10.1080/10550490500528464.
  • VA Priority Groups [Accessed 2023 Nov 27]. https://veteran.com/va-priority-groups/.
  • Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, Saunders LD, Beck CA, Feasby TE, Ghali WA, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130–9. doi:10.1097/01.mlr.0000182534.19832.83.
  • Sharma N, Schwendimann R, Endrich O, Ausserhofer D, Simon M. Comparing Charlson and Elixhauser comorbidity indices with different weightings to predict in-hospital mortality: an analysis of national inpatient data. BMC Health Serv Res. 2021;21(1):13. doi:10.1186/S12913-020-05999-5/TABLES/3.
  • Mehta HB, Li S, An H, Goodwin JS, Alexander GC, Segal JB. Development and Validation of the Summary Elixhauser Comorbidity Score for Use With ICD-10-CM–Coded Data Among Older Adults. Ann Intern Med. 2022;175(10):1423–30. doi:10.7326/M21-4204.
  • USDA ERS - rural-urban commuting area codes [Accessed 2023 Oct 2]. https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes/
  • Finlay AK, Harris AHS, Timko C, Yu M, Smelson D, Stimmel M, Binswanger IA. Disparities in access to medications for opioid use disorder in the veterans health administration. J Addict Med. 2021;15(2):143–9. doi:10.1097/ADM.0000000000000719.
  • Chu A, Ahn H, Halwan B, Kalmin B, Artifon ELA, Barkun A, Lagoudakis MG, Kumar A. A decision support system to facilitate management of patients with acute gastrointestinal bleeding. Artif Intell Med. 2008;42(3):247–59. doi:10.1016/J.ARTMED.2007.10.003.
  • Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. Published online 2009; doi:10.1007/978-0-387-84858-7.
  • Lo-Ciganic W-H, Huang JL, Zhang HH, Weiss JC, Wu Y, Kwoh CK, Donohue JM, Cochran G, Gordon AJ, Malone DC, et al. Evaluation of machine-learning algorithms for predicting opioid overdose risk among medicare beneficiaries with opioid prescriptions. JAMA Netw Open. 2019;2(3):e190968. doi:10.1001/jamanetworkopen.2019.0968.
  • Radzi SFM, Karim MKA, Saripan MI, Rahman MAA, Isa INC, Ibahim MJ. Hyperparameter tuning and pipeline optimization via grid search method and tree-based autoML in breast cancer prediction. J Pers Med. 2021;11(10):978. doi:10.3390/JPM11100978.
  • Zhu W, Zeng NF, Wang N. Sensitivity, specificity, accuracy, associated confidence interval and ROC analysis with practical SAS. Published online 2010.
  • Parikh R, Mathai A, Parikh S, Sekhar GC, Thomas R. Understanding and using sensitivity, specificity and predictive values. Indian J Ophthalmol. 2008;56(1):45–50. doi:10.4103/0301-4738.37595.
  • Hasan MM, Young GJ, Shi J, Mohite P, Young LD, Weiner SG, Noor-E-Alam M. A machine learning based two-stage clinical decision support system for predicting patients’ discontinuation from opioid use disorder treatment: retrospective observational study. BMC Med Inform Decis Mak. 2021;21(1):331. doi:10.1186/S12911-021-01692-7.
  • Schisterman EF, Faraggi D, Reiser B, Hu J. Youden Index and the optimal threshold for markers with mass at zero. Stat Med. 2008;27(2):297–315. doi:10.1002/SIM.2993.
  • Bolón-Canedo V, Sánchez-Maroño N, Alonso-Betanzos A. Feature selection and classification in multiple class datasets: an application to KDD Cup 99 dataset. Expert Syst Appl. 2011;38(5):5947–57. doi:10.1016/j.eswa.2010.11.028.
  • Ashenden SK, Bartosik A, Agapow PM, Semenova E. Introduction to artificial intelligence and machine learning. The era of artificial intelligence, machine learning, and data science in the pharmaceutical industry. Published online January 1, 2021. p. 15–26. doi:10.1016/B978-0-12-820045-2.00003-9.
  • Lever J, Krzywinski M, Altman N. Principal component analysis. Nat Methods. 2017;14(7):641–2. https://www.nature.com/articles/nmeth.4346 doi:10.1038/nmeth.4346.
  • Elor Y, Averbuch-Elor H. To SMOTE, or not to SMOTE? Published online January 21, 2022 [Accessed 2024 Jan 17]. https://arxiv.org/abs/2201.08528v3
  • Manhapra A, Petrakis I, Rosenheck R. Three-year retention in buprenorphine treatment for opioid use disorder nationally in the Veterans Health Administration. Am J Addict. 2017;26(6):572–80. doi:10.1111/AJAD.12553.
  • Mintz CM, Presnall NJ, Xu KY, Hartz SM, Sahrmann JM, Bierut LJ, Grucza RA. An examination between treatment type and treatment retention in persons with opioid and co-occurring alcohol use disorders. Drug Alcohol Depend. 2021;226:108886. doi:10.1016/J.DRUGALCDEP.2021.108886.
  • Ware OD, Manuel JI, Huhn AS. Adults with opioid and methamphetamine co-use have lower odds of completing short-term residential treatment than other opioid co-use groups: a retrospective health services study. Front Psychiatry. 2021;12:784229. doi:10.3389/FPSYT.2021.784229/BIBTEX.
  • Manhapra A, Agbese E, Leslie DL, Rosenheck RA. Three-year retention in buprenorphine treatment for opioid use disorder among privately insured adults. Psychiatr Serv. 2018;69(7):768–76. doi:10.1176/APPI.PS.201700363/SUPPL_FILE/APPI.PS.201700363.DS001.PDF.
  • Mintz CM, Presnall NJ, Sahrmann JM, Borodovsky JT, Glaser PEA, Bierut LJ, Grucza RA. Age disparities in six-month treatment retention for opioid use disorder. Drug Alcohol Depend. 2020;216:108312. doi:10.1016/J.DRUGALCDEP.2020.108130.
  • Ker S, Hsu J, Balani A, Mukherjee SS, Rush AJ, Khan M, Elchehabi S, Huffhines S, DeMoss D, Rentería ME, et al. Factors that affect patient attrition in buprenorphine treatment for opioid use disorder: a retrospective real-world study using electronic health records. Neuropsychiatr Dis Treat. 2021;17:3229–44. doi:10.2147/NDT.S331442.
  • Stafford C, Marrero WJ, Naumann RB, Lich KH, Wakeman S, Jalali MS. Identifying key risk factors for premature discontinuation of opioid use disorder treatment in the United States: a predictive modeling study. Drug Alcohol Depend. 2022;237:109507. doi:10.1016/J.DRUGALCDEP.2022.109507.
  • Hosmer DW, Jr., Lemeshow S. Applied logistic regression. 2nd Edition. New York: John Wiley & Sons, Inc.; 2000.
  • Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38. doi:10.1097/EDE.0b013e3181c30fb2.
  • Ruhm CJ. Corrected US opioid-involved drug poisoning deaths and mortality rates, 1999–2015. Addiction. 2018;113(7):1339–44. doi:10.1111/ADD.14144.
  • Freijeiro-González L, Febrero-Bande M, González-Manteiga W. A critical review of LASSO and its derivatives for variable selection under dependence among covariates. Int Statistical Rev. 2022;90(1):118–45. doi:10.1111/insr.12469.
  • Breiman L. Random forests. Mach Learn. 2001;45(1):5–32. doi:10.1023/A:1010933404324.
  • Svetnik V, Liaw A, Tong C, Christopher Culberson J, Sheridan RP, and Feuston BP. Random forest: a classification and regression tool for compound classification and QSAR modeling. J Chem Inf Comput Sci. 2003;43(6):1947–58. Published online doi:10.1021/CI034160G.
  • Natekin A, Knoll A. Gradient boosting machines, a tutorial. Front Neurorobot. 2013;7(DEC):63623. doi:10.3389/FNBOT.2013.00021/BIBTEX.
  • Montavon G, Samek W, Müller KR. Methods for interpreting and understanding deep neural networks. Digit Signal Process. 2018;73:1–15. doi:10.1016/j.dsp.2017.10.011.
  • Cao XH, Stojkovic I, Obradovic Z. A robust data scaling algorithm to improve classification accuracies in biomedical data. BMC Bioinform. 2016;17(1):359. doi:10.1186/S12859-016-1236-X.
  • Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. jair. 2002;16:321–57. doi:10.1613/jair.953.
  • Liu XY, Wu J, Zhou ZH. Exploratory undersampling for class-imbalance learning. IEEE Trans Syst Man Cybern B Cybern. 2009;39(2):539–50. doi:10.1109/TSMCB.2008.2007853.
  • Lunardon N, Menardi G, Torelli N. ROSE: A package for binary imbalanced learning. R Journal. 2014;6(1):79–89. doi:10.32614/RJ-2014-008.
  • Wadekar AS. Understanding Opioid Use Disorder (OUD) using tree-based classifiers. Drug Alcohol Depend. 2020;208:107839. doi:10.1016/J.DRUGALCDEP.2020.107839.
  • Sharma B, Dligach D, Swope K, Salisbury-Afshar E, Karnik NS, Joyce C, Afshar M. Publicly available machine learning models for identifying opioid misuse from the clinical notes of hospitalized patients. BMC Med Inform Decis Mak. 2020;20(1):79. doi:10.1186/S12911-020-1099-Y.
  • Ellis RJ, Wang Z, Genes N, Ma’ayan A. Predicting opioid dependence from electronic health records with machine learning. BioData Min. 2019;12(1):3. doi:10.1186/S13040-019-0193-0.
  • Vunikili R, Glicksberg BS, Johnson KW, Dudley JT, Subramanian L, Shameer K. Predictive modelling of susceptibility to substance abuse, mortality and drug-drug interactions in opioid patients. Front Artif Intell. 2021;4:742723. doi:10.3389/FRAI.2021.742723.
  • Hasan MM, Young GJ, Patel MR, Modestino AS, Sanchez LD, Noor-E-Alam M. A machine learning framework to predict the risk of opioid use disorder. Mach Learn Appl. 2021;6:100144. doi:10.1016/j.mlwa.2021.100144.