On the policy improvement algorithm in continuous time

Abstract

We develop a general approach to the Policy Improvement Algorithm (PIA) for stochastic control problems for continuous-time processes. The main results assume only that the controls lie in a compact metric space and give general sufficient conditions for the PIA to be well-defined and converge in continuous time (i.e. without time discretization). the PIA in continuous time is applicable in the context of weak stochastic control. We give simple explicit examples of control problems demonstrating the need for the weak formulation as well as diffusion-based classes of problems where the PIA in continuous time is applicable.

Keywords:

• Stochastic control in continuous time
• policy improvement algorithm
• general state space
• general controls

Notes

No potential conflict of interest was reported by the authors.