Deep reinforcement learning for maintenance optimization of multi-component production systems considering quality and production plan

Abstract

In this article, the maintenance optimization of multi-component production systems is investigated by considering quality and production plan. On the one hand, the downtime determined by the production plan provides opportunities for reducing maintenance costs; on the other hand, the deterioration of product quality induced by poor health state leads to extra loss. The coupled relations between production plan, quality, and maintenance, as well as the dependence between multiple components, pose challenges for maintenance optimization. To overcome these challenges, a novel decision model and a deep reinforcement learning-based solving method are proposed. Specifically, in addition to the degradation states of all components, the remaining time of the current batch related to the production plan is also treated as the system state, and the quality loss related to the degradation states is added to the reward function. The deep Q-network algorithm is employed, solving the maintenance optimization problem that considers quality and production plan. The effectiveness of the proposed method is validated by a numerical experiment.

Keywords:

• Deep reinforcement learning
• maintenance optimization
• multi-component
• production plan
• quality

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Key Research and Development Program of China (No. 2022YFB3206904), and the National Natural Science Foundation of China (62033012, 62103394 and 62173317).

Notes on contributors

Ming Chen

Ming Chen received his BS degree from the University of Science and Technology of China (USTC), Hefei, China in 2021. He is currently pursuing the M.S. degree with the Department of Automation, USTC. His current research interests include maintenance optimization and reinforcement learning.

Yu Kang

Yu Kang received the PhD degree in control theory and control engineering from the University of Science and Technology of China, Hefei, China, in 2005. From 2005 to 2007, he was a Postdoctoral Fellow with the Academy of Mathe- matics and Systems Science, Chinese Academy of Sciences. He is currently a Professor with the Department of Automation and the Institute of Advanced Technology, University of Science and Technology of China. His current research interests in- clude monitoring of vehicle emissions, fault prediction and diagnosis and maintenance.

Kun Li

Kun Li received the PhD degree in control science and engineer- ing from University of Science and Technology of China (USTC), Hefei, China, in 2019. He is currently a Postdoctoral Fellow with Institute of Advanced Technology, USTC. His research interests include nonlinear control theory and its application, robot control system.

Pengfei Li

Pengfei Li received the PhD degree in control science and engineering from the University of Science and Technology of China (USTC), Hefei, China, in 2020. He is an Associate Research Fellow with the School of Information Science and Technology, USTC. His research interests include networked control systems, model predictive control, and learning based control.

Yun-Bo Zhao

Yun-Bo Zhao received his B.Sc. degree in mathematics from Shan- dong University, Jinan, China in 2003, M.Sc. degree in systems sciences from the Key Laboratory of Systems and Control, Chinese Academy of Sciences, Beijing, China in 2007, and Ph.D. degree in control engineering from the University of South Wales (formerly University of Glamorgan), Pontypridd, UK in 2008, respectively. He is cur- rently a Professor with University of Science and Technology of China, Hefei, China. He is mainly interested in AI-driven control and automation, specifically, AI-driven networked intelligent control, AI-driven humanmachine autonomies and AI-driven ma- chine gaming.