A multilevel approach for stochastic nonlinear optimal control

Abstract

We consider a class of finite-time horizon nonlinear stochastic optimal control problem. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging task. We propose a new Monte Carlo approach that significantly improves upon existing methodology. We tackle the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state-space model, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce the cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}\left({ϵ}^2\right)$ mean squared error with a cost of $\mathcal{O}({ϵ}^{-2}\log (ϵ{)}^2)$. In contrast, a cost of $\mathcal{O}\left({ϵ}^{-3}\right)$ is required for the existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples.

Keywords:

• Optimal control
• multilevel Monte Carlo
• Markov chain Monte Carlo
• sequential Monte Carlo

Acknowledgments

A.J. and Y.X. were supported by an AcRF tier 2 [grant number R-155-000-161-112]. A.J. is affiliated with the Risk Management Institute, the Center for Quantitative Finance and the OR & Analytics cluster at NUS. A.J. was supported by a KAUST CRG4 grant ref: 2584.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

A.J. and Y.X. were supported by an Singapore Ministry of Education – AcRF tier 2 [grant number R-155-000-161-112]. A.J. is affiliated with the Risk Management Institute, the Center for Quantitative Finance and the OR & Analytics cluster at NUS. A.J. was supported by a KAUST CRG4 grant ref: 2584.