Denumerable continuous-time Markov decision processes with multiconstraints on average costs

Abstract

This article deals with multiconstrained continuous-time Markov decision processes in a denumerable state space, with unbounded cost and transition rates. The criterion to be optimised is the long-run expected average cost, and several kinds of constraints are imposed on some associated costs. The existence of a constrained optimal policy is ensured under suitable conditions by using a martingale technique and introducing an occupation measure. Furthermore, for the unichain model, we transform this multiconstrained problem into an equivalent linear programming problem, then construct a constrained optimal policy from an optimal solution to the linear programming. Finally, we use an example of a controlled queueing system to illustrate an application of our results.

Keywords:

• Markov decision processes
• average cost
• constrained optimal policy
• occupation measure

Acknowledgement

This research is supported by NSFC and GBUPS.