An efficient algorithm for stochastic optimal control problems by means of a least-squares Monte-Carlo method

Abstract

In this work, we provide discrete optimality conditions of the optimal control problems of stochastic differential equations. Euler and Runge–Kutta methods are used for discretization. A Lagrange multiplier method for a discrete-time stochastic optimal control problem is formulated. The discrete adjoint process $p_n$ is obtained in terms of conditional expectations $\mathbb{E}\left[p_{n+1}\right]$ and $\mathbb{E}\left[p_{n+1}\mathrm{\Delta }W\right]$ for both methods. To estimate these nested conditional expectations at each time step via simulation, we use a very powerful new approach, least-squares Monte-Carlo method, developed by Longstaff–Schwartz. This is the first time to solve a stochastic optimal control problem by calculating the nested conditional expectations numerically with the help of a least-squares Monte-Carlo method. Some examples are studied to test and demonstrate the efficiency of the Lagrange multiplier combined with the least-squares Monte-Carlo method.

Keywords:

• Stochastic optimal control
• discrete optimality conditions
• least-squares Monte-Carlo method
• Runge–Kutta method
• optimization

Disclosure statement

No potential conflict of interest was reported by the author(s).