Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

References

• C.D. Aliprantis and K.C. Border, Infinite Dimensional Analysis, 3rd ed., Springer, Berlin, 2006. A hitchhiker's guide.
   Google Scholar
• A. Almudevar, A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes, SIAM J. Control Optim. 40(2) (2001), pp. 525–539. doi: 10.1137/S0363012999364474
   Web of Science ®Google Scholar
• N. Bauerle and U. Rieder, Optimal control of piecewise deterministic Markov processes with finite time horizon, in Modern Trends in Controlled Stochastic Processes: Theory and Applications, Luniver Press, United Kingdom, 2010, pp. 123–143.
   Google Scholar
• R. Cavazos-Cadena and F. Salem-Silva, The discounted method and equivalence of average criteria for risk-sensitive Markov decision processes on Borel spaces, Appl. Math. Optim. 61 (2010), pp. 167–190. doi: 10.1007/s00245-009-9080-2
   Web of Science ®Google Scholar
• O.L.V. Costa and F. Dufour, The vanishing discount approach for the average continuous control of piecewise deterministic Markov processes, J. Appl. Probab. 46(4) (2009), pp. 1157–1183. doi: 10.1239/jap/1261670695
   Web of Science ®Google Scholar
• O.L.V Costa and F. Dufour, Continuous Average Control of Piecewise Deterministic Markov Processes, Springer Briefs in Mathematics, Springer, New York, 2013.
   Google Scholar
• O.L.V. Costa, F. Dufour, and A.B. Piunovskiy, Constrained and unconstrained optimal discounted control of piecewise deterministic Markov processes, SIAM J. Control Optim. 54(3) (2016), pp. 1444–1474. doi: 10.1137/140996380
   Web of Science ®Google Scholar
• M.H.A. Davis, Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models, J. R. Stat. Soc. (B) 46(3) (1984), pp. 353–388.
   Google Scholar
• M.H.A. Davis, Control of piecewise-deterministic processes via discrete-time dynamic programming, in Stochastic Differential Systems (Bad Honnef, 1985), volume 78 of Lecture Notes in Control and Inform. Sci., Springer, Berlin, 1986, pp. 140–150.
   Google Scholar
• M.H.A. Davis, Markov Models and Optimization, volume 49 of Monographs on Statistics and Applied Probability, Chapman & Hall, London, 1993.
   Google Scholar
• M.H.A Davis and M. Farid, A target recognition problem: Sequential analysis and optimal control, SIAM J. Control Optim. 34(6) (1996), pp. 2116–2132. doi: 10.1137/S0363012994273696
   Web of Science ®Google Scholar
• M.A.H. Dempster and J.J. Ye, Necessary and sufficient optimality conditions for control of piecewise deterministic processes, Stoch. Stoch. Rep. 40(3–4) (1992), pp. 125–145. doi: 10.1080/17442509208833785
   Google Scholar
• M.A.H. Dempster and J.J. Ye, Generalized Bellman-Hamilton-Jacobi optimality conditions for a control problem with boundary conditions, Appl. Math. Optim. 33(3) (1996), pp. 211–225. doi: 10.1007/BF01204702
   Web of Science ®Google Scholar
• L. Forwick, M. Schäl, and M. Schmitz, Piecewise deterministic Markov control processes with feedback controls and unbounded costs, Acta Appl. Math. 82(3) (2004), pp. 239–267. doi: 10.1023/B:ACAP.0000031200.76583.75
   Web of Science ®Google Scholar
• X. Guo and O. Hernández-Lerma, Continuous-Time Markov Decision Processes, volume 62 of Stochastic Modelling and Applied Probability, Springer-Verlag, Berlin, 2009. Theory and applications.
   Google Scholar
• X. Guo and A. Piunovskiy, Discounted continuous-time Markov decision processes with constraints: Unbounded transition and loss rates, Math. Oper. Res. 36(1) (2011), pp. 105–132. doi: 10.1287/moor.1100.0477
   Web of Science ®Google Scholar
• X. Guo and L. Ye, New discount and average optimality conditions for continuous-time Markov decision processes, Adv. Appl. Probab. 42(4) (2010), pp. 953–985. doi: 10.1239/aap/1293113146
   Web of Science ®Google Scholar
• X. Guo, O. Hernández-Lerma, T. Prieto-Rumeau, X.R. Cao, J. Zhang, Q. Hu, M.E. Lewis, and R. Vélez, A survey of recent results on continuous-time Markov decision processes, TOP 14(2) (2006), pp. 177–261. doi: 10.1007/BF02837562
   Web of Science ®Google Scholar
• O. Hernández-Lerma and J.-B. Lasserre, Discrete-Time Markov Control Processes, volume 30 of Applications of Mathematics (New York), Springer-Verlag, New York, 1996. Basic optimality criteria.
   Google Scholar
• J. Jacod, Multivariate point processes: Predictable projection, Radon-Nikodým derivatives, representation of martingales, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 31 (1974/75), pp. 235–253. doi: 10.1007/BF00536010
   Google Scholar
• J. Jacod, Calcul stochastique et problèmes de martingales, volume 714 of Lecture Notes in Mathematics, Springer, Berlin, 1979.
   Google Scholar
• A. Jaśkiewicz and A.S. Nowak, Zero-sum ergodic stochastic games with feller transition probabilities, SIAM J. Control Optim. 45 (2006), pp. 773–789. doi: 10.1137/S0363012904443257
   Web of Science ®Google Scholar
• A. Piunovskiy and Y. Zhang, The transformation method for continuous-time Markov decision processes, J. Optim. Theory Appl. 154(2) (2012), pp. 691–712. doi: 10.1007/s10957-012-0015-8
   Web of Science ®Google Scholar
• M. Schäl, Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 32(3) (1975), pp. 179–196. doi: 10.1007/BF00532612
   Google Scholar
• M. Schäl, Average optimality in dynamic programming with general state space, Math. Oper. Res. 18(1) (1993), pp. 163–172. doi: 10.1287/moor.18.1.163
   Web of Science ®Google Scholar
• M. Schäl, On piecewise deterministic Markov control processes: Control of jumps and of risk processes in insurance, Insurance Math. Econom. 22(1) (1998), pp. 75–91. doi: 10.1016/S0167-6687(98)00010-9
   Web of Science ®Google Scholar
• A.A. Yushkevich, On reducing a jump controllable Markov model to a model with discrete time, Theory Probab. Appl. 25(1) (1980), pp. 58–69. doi: 10.1137/1125005
   Web of Science ®Google Scholar
• A.A. Yushkevich, Bellman inequalities in Markov decision deterministic drift processes, Stochastics 23(1) (1987), pp. 25–77. doi: 10.1080/17442508708833481
   Google Scholar
• Y. Zhang, Average optimality for continuous-time Markov decision processes under weak continuity conditions, J. Appl. Probab. 51(4) (2014), pp. 954–970. doi: 10.1239/jap/1421763321
   Web of Science ®Google Scholar
• Q. Zhu, Average optimality for continuous-time Markov decision processes with a policy iteration approach, J. Math. Anal. Appl. 339(1) (2008), pp. 691–704. doi: 10.1016/j.jmaa.2007.06.071
   Web of Science ®Google Scholar