Markov Decision Processes with Asymptotic Average Failure Rate Constraint

Abstract

In this paper, we introduce a constrained Markov Decision Process with absorbing states and periodic classes. The objective is to find a policy which maximises the infinite horizon expected average reward before the failure of the system, with a constraint on the asymptotic average failure rate. We propose an algorithm to find a stationary policy, ξ-optimal in the history dependent class. Finally, we apply the method to optimize the maintenance strategy of an aeronautical system.

Key Words:

• Constrained Markov decision processes
• Asymptotic failure rate
• Maintenance policy
• Fault tolerant aeronautical systems

Mathematics Subject Classification:

• 60J10
• 60K10
• 90B25

Acknowledgments

This work was financially supported by SNECMA Co. Villaroche Center. We are particularly indebted to Mrs J. C. Fillon, T. Bickard and G. Raffay for their particular interest to this work.