A Stochastic Approximation-Langevinized Ensemble Kalman Filter Algorithm for State Space Models with Unknown Parameters

Abstract

Inference for high-dimensional, large scale and long series dynamic systems is a challenging task in modern data science. The existing algorithms, such as particle filter or sequential importance sampler, do not scale well to the dimension of the system and the sample size of the dataset, and often suffers from the sample degeneracy issue for long series data. The recently proposed Langevinized ensemble Kalman filter (LEnKF) addresses these difficulties in a coherent way. However, it cannot be applied to the case that the dynamic system contains unknown parameters. This article proposes the so-called stochastic approximation-LEnKF for jointly estimating the states and unknown parameters of the dynamic system, where the parameters are estimated on the fly based on the state variables simulated by the LEnKF under the framework of stochastic approximation Markov chain Monte Carlo (MCMC). Under mild conditions, we prove its consistency in parameter estimation and ergodicity in state variable simulations. The proposed algorithm can be used in uncertainty quantification for long series, large scale, and high-dimensional dynamic systems. Numerical results indicate its superiority over the existing algorithms. We employ the proposed algorithm in state-space modeling of the sea surface temperature with a long short term memory (LSTM) network, which indicates its great potential in statistical analysis of complex dynamic systems encountered in modern data science. Supplementary materials for this article are available online.

Keywords:

• Dynamic system
• Ensemble Kalman filter
• Long short term memory (LSTM) network
• Stochastic approximation
• Stochastic gradient MCMC

Disclosure Statement

The authors report there are no competing interests to declare.

Supplementary Materials

The supplementary materials describe the state-augmentation EnKF algorithm and the SA-LEnKF-Offline algorithm.

Acknowledgments

The authors thank the editor, associate editor, and referee for their constructive comments which have led to significant improvement of this article. The authors also thank Q. Song and Y. Sun for helpful discussions on the article.

Additional information

Funding

Liang’s research was supported in part by the grants DMS-2015498, R01-GM117597 and R01-GM126089.