Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

ABSTRACT

The main goal of this paper is to study the infinite-horizon long run average continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. We provide conditions for the existence of a solution to an integro-differential optimality inequality, the so called Hamilton-Jacobi-Bellman (HJB) equation, and for the existence of a deterministic stationary optimal policy. These results are obtained by using the so-called vanishing discount approach, under some continuity and compactness assumptions on the parameters of the problem, as well as some non-explosive conditions for the process.

KEYWORDS:

• Average control
• continuous control
• Hamilton-Jacobi-Bellman inequality
• piecewise deterministic Markov process
• continuous-time Markov decision process

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

O. L. V. Costa  http://orcid.org/0000-0002-0875-8698

F. Dufour  http://orcid.org/0000-0001-6653-2024

Additional information

Funding

The first author received financial support from CNPq (Brazilian National Council for Scientific and Technological Development), grant 304091/2014-6, FAPESP (São Paulo Research Foundation)/ Shell through the Research Centre for Gas Innovation, FAPESP Grant Proc. 2014/50279-4, project INCT, grant CNPq 465755/2014-3, FAPESP 2014/50851-0 and FUSP (University of São Paulo Support Foundation).