Cooperative zone-based rebalancing of idle overhead hoist transportations using multi-agent reinforcement learning with graph representation learning

Abstract

Due to the recent advances in manufacturing systems, the semiconductor FABs have become larger, and thus, more overhead hoist transporters (OHTs) need to be operated. In this article, we propose a cooperative zone-based rebalancing algorithm to allocate idle overhead hoist vehicles in a semiconductor FAB. The proposed model is composed of two parts: (i) a state representation learning part that extracts the localized embedding of each agent using a graph neural network; and (ii) a policy learning part that makes a rebalancing action using the constructed embedding. By conducting both representation learning and policy learning in a single framework, the proposed method can train the decentralized policy for agents to rebalance OHTs cooperatively. The experiments show that the proposed method can significantly reduce the average retrieval time while reducing the OHT utilization ratio. In addition, we investigated the transferable capability of the suggested algorithm by testing the policy on unseen dynamic scenarios without further training.

Keywords:

• Graph neural network
• multi-agent reinforcement learning
• semiconductor Manufacturing
• OHT system
• rebalancing

Additional information

Notes on contributors

Kyuree Ahn

Kyuree Ahn received her BS degree in mathematical sciences from the Korea Advanced Institute of Science and Technology (KAIST), South Korea, in 2011, and a MS degree in industrial and systems engineering from the KAIST, South Korea, in 2018. Currently, she is PhD candidate in the System Intelligence Laboratory at the Department of Industrial and Systems Engineering, KAIST, South Korea.

Jinkyoo Park

Jinkyoo Park received his BS degree in civil and architectural engineering from Seoul National University, Seoul, South Korea, in 2009, a M.S. degree in civil, architectural and environmental engineering from the University of Texas Austin, Texas, USA, in 2011, a MS degree in electrical engineering and a PhD degree in civil and environmental engineering from Stanford University, California, USA, in 2016. He is now an assistant professor in the Department of Industrial and Systems Engineering at KAIST, South Korea.