POMDP and MOMDP solutions for structural life-cycle cost minimization under partial and mixed observability

Abstract

Scheduling of inspection and maintenance policies during the life-cycle of operating infrastructure necessitates optimization of long-term objectives in stochastic environments. Modern answers to the problem should focus on quantitative decision-making techniques, taking advantage of informative but uncertain data that become available in time. As such, the problem is efficiently addressed within the framework of stochastic dynamic programming by means of Partially Observable Markov Decision Processes (POMDPs) and Mixed Observability Markov Decision Processes (MOMDPs). Although these methodologies can provide very sophisticated solutions with optimality guarantees, important computational challenges often emerge, mainly due to the continuity of the multidimensional belief space on the probability simplex. In response, recent value iteration algorithms based on point-based approaches have been suggested, focusing on reachable belief points that can support an accurate value function. In this work, several POMDP and MOMDP point-based algorithms, with various characteristics regarding the exploration of the belief space and the value function update procedures, are rigorously analyzed. The algorithms are compared and evaluated in terms of accuracy and performance in stationary and nonstationary problems of structural inspection and maintenance for life-cycle cost minimization. Results are thoroughly discussed and several insights along with practical suggestions for similar problems are provided.

Keywords:

• Decision-making
• Partially Observable Markov Decision Processes (POMDPs)
• Mixed Observability Markov Decision Processes (MOMDPs)
• point-based algorithms
• stochastic control
• structural life-cycle analysis
• inspection and maintenance