Approximation solution and suboptimality for discounted semi-markov decision problems with countable state space

Abstract

In this article we consider the approximate solution for semi-Markov decision problems with infinite horizon, countable state space, discounted cost function and finite action space. We present converging sequences of lower and upper bounds for the value function and, moreover, we derive a method for exclusion of suboptimal actions.

Keywords:

• Semi-Markov decision problems
• Markov renewal programming
• Mathematics Subject Classification 2000: 90C40