Development and validation of a machine-learning ALS survival model lacking vital capacity (VC-Free) for use in clinical trials during the COVID-19 pandemic

Abstract

Introduction: Vital capacity (VC) is routinely used for ALS clinical trial eligibility determinations, often to exclude patients unlikely to survive trial duration. However, spirometry has been limited by the COVID-19 pandemic. We developed a machine-learning survival model without the use of baseline VC and asked whether it could stratify clinical trial participants and a wider ALS clinic population. Methods. A gradient boosting machine survival model lacking baseline VC (VC-Free) was trained using the PRO-ACT ALS database and compared to a multivariable model that included VC (VCI) and a univariable baseline %VC model (UNI). Discrimination, calibration-in-the-large and calibration slope were quantified. Models were validated using 10-fold internal cross validation, the VITALITY-ALS clinical trial placebo arm and data from the Emory University tertiary care clinic. Simulations were performed using each model to estimate survival of patients predicted to have a > 50% one year survival probability. Results. The VC-Free model suffered a minor performance decline compared to the VCI model yet retained strong discrimination for stratifying ALS patients. Both models outperformed the UNI model. The proportion of excluded vs. included patients who died through one year was on average 27% vs. 6% (VCI), 31% vs. 7% (VC-Free), and 13% vs. 10% (UNI). Conclusions. The VC-Free model offers an alternative to the use of VC for eligibility determinations during the COVID-19 pandemic. The observation that the VC-Free model outperforms the use of VC in a broad ALS patient population suggests the use of prognostic strata in future, post-pandemic ALS clinical trial eligibility screening determinations.

Keywords:

• Vital capacity
• COVID-19
• survival model
• machine learning prognostic model
• clinical trials

Acknowledgments

The machine learning models described here were created using the PRO-ACT database. We thank Cytokinetics for generously providing access to the VITALITY-ALS placebo dataset for use in these studies.

Declaration of interest

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The Emory Clinic Database Project is funded by a grant to Dr. Glass from the Muscular Dystrophy Association.