Leveraging deep reinforcement learning for design space exploration with multi-fidelity surrogate model

Abstract

Design automation is undergoing a new generation of changes caused by artificial intelligence technologies represented by deep learning and reinforcement learning. Notably, the advantages of deep reinforcement learning in addressing solution optimisation and decision-making tasks with cognitive automation functionality have garnered attention in design. In the context of surrogate model-driven engineering design optimisation, this paper addresses current research challenges such as reliance on domain knowledge for local development, shortcomings in the self-learning and adaptive capabilities of optimisation algorithms for global exploration, etc. Centred around the deep reinforcement learning model, Deep Q-learning, and complemented by self-organising maps and neural network technologies, we propose a methodology considering multi-fidelity simulation data for design space exploration. This approach effectively reduces sampling costs and enables the optimisation model to learn the optimal direction for high-precision predictions and achieve rapid, accurate optimisation. Finally, the effectiveness of the proposed method is comprehensively validated through four typical optimisation scenarios and a case study involving the optimisation of a wheeled robot's suspension swing arm structure. This work will be a crucial reference for applying deep reinforcement learning in simulation-driven engineering design optimisation.

KEYWORDS:

• Machining process design
• knowledge graph
• deep reinforcement learning
• knowledge representation and reuse

Acknowledgments

This paper is an outcome of the International Systems Realization Partnership between the Institute for Industrial Engineering The Beijing Institute of Technology, China, The Systems Realization Laboratory The University of Oklahoma, USA, the Design Engineering Laboratory Purdue, USA, and The Systems Realization Laboratory The University of Liverpool, UK. Thanks for the support of the Key Laboratory of Industry Knowledge & Data Fusion Technology and Application, Ministry of Industry and Information Technology, Beijing Institute of Technology.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Natural Science Foundation of China (NSFC 52105241), Beijing Institute of Technology Research Fund Program for Young Scholars, and also financial support from the National Ministries Projects of China (50923010101, D020101).