Overtaking optimality for controlled Markov-modulated diffusions

Abstract

This article concerns controlled Markov-modulated diffusions (also known as piecewise diffusions or switching diffusions or diffusions with Markovian switchings). Our main objective is to give conditions for the existence and characterization of overtaking optimal policies. To this end, first, we use fact that the average reward Hamilton–Jacobi–Bellman equation gives that the family of the so-called canonical control policies is nonempty. Then, within this family, we search policies with some special feature, for instance, canonical policies that in addition maximize the bias, which turn out to be overtaking optimal.

Keywords:

• average optimality criteria
• bias optimality
• controlled Markov-modulated diffusions
• ergodicity
• Hamilton–Jacobi–Bellman equations
• overtaking optimality

AMS Subject Classifications::

• 93E20
• 60J60

Acknowledgements

The research of B.A. Escobedo-Jrujiuo was supported by a CONACyT scholarship. The research of the O. Herńandez-Lerma was partially supported by a CONACyT grant 104001.