Optimality equations and sensitive optimality in bounded Markov decision processes ¹

¹Research supported by the Deutsche Forschungsgenleinschaft, Sonderforschungsbereich 72, a t the University of Bonn.

Abstract

This paper consider Markov decision processes with countable state space, compact action spaces and a bounded reward function. Under some recurrence and connectedness condition, including the simultaneous Döblin condition, we prove the existence of bounded solutions of the optimality equations which arise for the multichain case in connection with the average reward criterion and sensitive optimality criteria, and we give a characterization of the sets of n-average optimal decision rules.

Keywords:

• Markov decision process
• optimality equations
• average optimality
• bias optimality
• sensitive optimality criteria

ANS 1980 Subject Classifications:

• Primary: 90 C 39
• Secondary: 90 C 40

¹Research supported by the Deutsche Forschungsgenleinschaft, Sonderforschungsbereich 72, a t the University of Bonn.

¹Research supported by the Deutsche Forschungsgenleinschaft, Sonderforschungsbereich 72, a t the University of Bonn.

Notes

¹Research supported by the Deutsche Forschungsgenleinschaft, Sonderforschungsbereich 72, a t the University of Bonn.