Using Differentiable Programming for Flexible Statistical Modeling

ABSTRACT

Differentiable programming has recently received much interest as a paradigm that facilitates taking gradients of computer programs. While the corresponding flexible gradient-based optimization approaches so far have been used predominantly for deep learning or enriching the latter with modeling components, we want to demonstrate that they can also be useful for statistical modeling per se, for example, for quick prototyping when classical maximum likelihood approaches are challenging or not feasible. In an application from a COVID-19 setting, we use differentiable programming to quickly build and optimize a flexible prediction model adapted to the data quality challenges at hand. Specifically, we develop a regression model, inspired by delay differential equations, that can bridge temporal gaps of observations in the central German registry of COVID-19 intensive care cases for predicting future demand. With this exemplary modeling challenge, we illustrate how differentiable programming can enable simple gradient-based optimization of the model by automatic differentiation. This allowed us to quickly prototype a model under time pressure that outperforms simpler benchmark models. We thus exemplify the potential of differentiable programming also outside deep learning applications to provide more options for flexible applied statistical modeling.

KEYWORDS:

• Differential equations
• Machine learning
• Optimization
• Workflow

Supplementary Materials

The supplementary material contains the code for the original Julia implementation of the loss function and model optimization as well as additional results on the comparison of the proposed model with benchmarks, sensitivity analysis with respect to model covariates and data subsets, and model validation.

Acknowledgments

The authors thank Gerta Rücker for her thoughtful comments on the article.

Notes

1 https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Projekte_RKI/SPoCK.html

Additional information

Funding

MH acknowledges funding by the DFG (German Research Foundation, grant no. 322977937/GRK2344).