Development of a classifier to identify patients with probable Lennox–Gastaut syndrome in health insurance claims databases via random forest methodology

Abstract

Objective: Describe the development of a claims-based classifier utilizing machine learning to identify patients with probable Lennox–Gastaut syndrome (LGS) from six state Medicaid programs.

Methods: Patients were included if they had ≥2 medical claims ≥30 days apart for specified or unspecified epilepsy, excluding those with ≥1 claim for petit mal status. The LGS classifier utilized a random forest algorithm, a compilation of thousands of binary decision trees in which machine-generated predictor variables split the data set into branches that predict the presence or absence of LGS. To construct the splitting rules, the importance of each candidate variable was determined by calculating the mean decrease in Gini impurity. Training and testing were performed on two data sets (30% and 70%) using a “true” LGS and non-LGS patient population. Performance was compared with logistic regression and single tree methodology.

Results: Using a 60% probability threshold, which yielded the highest sensitivity (97.3%) and specificity (95.6%), the classifier identified approximately 4% of patients with epilepsy as probable LGS. The most important input variables included number of distinct antiepileptic drugs received, epilepsy-related outpatient/inpatient visits, electroencephalogram procedures and claims for delayed development. The random forest methodology outperformed logistic regression and single tree methodology. Most of the important LGS predictor characteristics identified by the classifier were statistically significantly associated with LGS status (p < .05).

Conclusions: The claims-based LGS classifier showed high sensitivity and specificity, outperformed single tree and logistic regression methodologies and identified a prevalence of probable LGS that was similar to previously published estimates.

Keywords:

• Epilepsy
• Lennox–Gastaut syndrome
• LGS
• LGS classifier
• machine learning
• random forest methodology

Transparency

Declaration of funding

This study was funded by Lundbeck.

Author contributions: F.V., J.E.P.-G., W.Y.C., E.T., M.S.D., V.S. and J.I. were involved in the conception and design of this work. F.V., J.E.P.-G., W.Y.C., E.T., P.G.-D., A.O., J.D., M.S.D., V.S., T.B.S. and J.I. were involved in the analysis and interpretation of the data. F.V., W.Y.C., E.T., P.G.-D., A.O., J.D., M.S.D., V.S., T.B.S. and J.I. were involved in the drafting of the manuscript. F.V., J.E.P.-G., W.Y.C., E.T., P.G.-D., A.O., J.D., M.S.D., V.S., T.B.S. and J.I. were involved in the critical revision of the manuscript. All authors have approved the final version for publication of this manuscript and agree to be accountable for all aspects of this work.

Declaration of financial/other relationships

W.Y.C., E.T. and M.S.D. have disclosed that they are employees of Analysis Group Inc., which received research funding for this study. F.V., P.G.-D., A.O. and J.D. have disclosed that they were employees of Analysis Group Inc. at the time the study was conducted. J.E.P.-G. has disclosed that he has served as a consultant and/or speaker for Eisai Co. Ltd., Lundbeck, Sunovion Pharmaceuticals Inc., Supernus Pharmaceuticals and UCB, and has received research grants from Eisai Co. Ltd. and UCB. G.D.M. has disclosed that she has served as a consultant for Acorda Therapeutics Inc., Eisai Co. Ltd., Lundbeck, UCB and Upsher-Smith Laboratories. V.S. has disclosed that she is an employee of Lundbeck. J.I. and T.B.S. have disclosed that they were employees of Lundbeck at the time the study was conducted. CMRO peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgements

Manuscript preparation, including writing, editing and formatting the manuscript, incorporating author comments, and coordinating submission requirements, was provided by Prescott Medical Communications Group. Editorial support was also provided by CHC Group LLC (North Wales, PA), an ICON plc company. All editorial support was funded by Lundbeck. Some of these data have previously been published in abstract form at the following congresses: 2016 and 2017 American Academy of Neurology annual meetings, 2015 and 2016 American Epilepsy Society annual meetings, 2016 and 2017 International Society for Pharmacoeconomics and Outcomes annual international meetings.