Control Limit Policy for Partially Observable Markov Decision Process Based on Stochastic Increasing Ordering

Abstract

We investigated an optimal maintenance problem for a condition monitoring system that is formulated as a partially observable Markov decision process. In almost all previous research, the optimal control limit policy with respect to totally positive of order 2 (TP₂) ordering of the state probability vectors was derived using the property of posterior probability. We call this approach “Approach PP.” Attempts to achieve an optimal control limit policy based on stochastic increasing (SI) ordering, which is weaker than TP2 ordering, ran into a difficulty. That is, the SI property of the posterior probability vectors could not be obtained, and this property is needed to derive results expected using Approach PP. We investigated the conditions for the SI property of posterior probability vectors and derived a necessary and sufficient condition that cannot be further relaxed when we try to derive an optimal control limit policy based on SI ordering using Approach PP. This condition is that the conditional probability matrix of the monitor observation given the state of the system is given as Type 1), the probabilities of the same monitoring output are the same whatever the true state, or Type 2), an identity matrix. Type 1) means that the monitoring output is independent of the true state.

Key Words:

• Condition monitoring maintenance
• optimal procedure
• totally positive of order 2

Additional information

Notes on contributors

Lu Jin

Lu Jin received a Ph.D. degree (2006) in systems engineering from the University of Electro-Communications (UEC). She is currently an Assistant Professor in the Department of Informatics, at UEC. She is a member of both the Japanese Society for Quality Control (JSQC) and the Reliability Engineering Association of Japan (REAJ). Her current research interest is optimal condition monitoring.

Kazuhiro Kumagai

Kazuhiro Kumagai received a B.S. degree (2005) in systems engineering from the University of Electro-Communications (UEC). He is currently working at Fuji Xerox Co., Ltd. in Japan.

Kazuyuki Suzuki

Kazuyuki Suzuki received a Ph.D. degree (1979) from the Tokyo Institute of Technology. He is currently a Professor in the Department of Informatics, at the University of Electro-Communications. His research interests include optimum conditioning monitoring, optimum reliability testing, and analysis of lifetime data. He was awarded the Wilcoxon Prize (1999), the Nikkei Quality Control Literature Prize (1977, 1986, 2005), Takagi Literature Prize (1995, 2003, 2006). He has been a member of the Deming Prize Committee since 1990. He has served as editor for both the Japanese Society for Quality Control (JSQC) and the Reliability Engineering Association of Japan (REAJ), and is on the board of both societies. He is currently a president of JSQC, also the vice-president of REAJ.