Stochastic exit time problems arising in process control

Abstract

This paper concerns the problem of controlling a stochastic system, with small noise parameter, to prevent it leaving a safe region of the state space. Such problems arise in flow control and other areas. We consider a formulation of the problem, in which a control is sought, to maximize a cost which is related to the expected exit time, but modified to reduce the probability of an early exit, according to a specified level of risk aversion (‘risk sensitive’ stochastic control). Formally letting the noise parameter tend to zero, we find that the optimal control strategy for this problem coincides with the optimal feedback control strategy for a differential game. We identify a class of differential games arising in this way, the so called decomposable differential games, for which the optimal control strategy can be easily obtained and illustrate the proposed solution technique by applying it to a flow control problem arising in process systems engineering.

Keywords:

• stochastic control
• exit problems
• risk-sensitive control
• differential games