Explainable, Deep Reinforcement Learning–Based Decision Making for Operations and Maintenance

Abstract

This paper presents research that integrates condition monitoring and prognostics with decision making for nuclear power plant operations and maintenance aimed at reducing lifetime maintenance and repair costs. Additionally, a focal point of this research is to make the decisions explainable to operators, improving the trustworthiness of the decisions from what can be considered a black box model. In this work, we develop and evaluate an explainable, online asset management methodology to help reduce lifetime maintenance and repair costs. Using the latest advancements in condition monitoring, inventory management, deep reinforcement learning, and explainable artificial intelligence methods, we create a predictive maintenance methodology that can optimize the maintenance and spare part management of a repairable nuclear power plant system.

To demonstrate these methods, preliminary studies were conducted on a representative maintenance system undergoing a stochastic degradation process that requires repairs or replacement to continue operation. Using deep reinforcement learning, we were able to reduce maintenance spending by approximately 50% compared to optimized, time-based maintenance strategies for the chosen system. A key component of our methodology is the integration of Shapley values to quantify the contribution of various factors to the decision-making process. This addition enhances the explainability and trustworthiness of our decisions, providing operators with transparent and understandable insights into the rationale behind maintenance strategies. The robustness and resiliency of our decision policy against observation noise were also thoroughly evaluated, demonstrating its effectiveness in uncertain operational environments.

Keywords:

• Deep reinforcement learning
• operations and maintenance
• decision making
• explainability
• uncertainty

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported in part by the following projects: U.S. Nuclear Regulatory Commission grant number 31310018M0048, University of Pittsburgh Nuclear Engineering Graduate Fellowship Program, and U.S. Department of Energy Office of Nuclear Energy’s Nuclear Energy University Program DE-NE0008909 under the Nuclear Energy Enabling Technologies Advanced Sensors and Instrumentation Program.