A multi-agent reinforcement learning model for inventory transshipments under supply chain disruption

Abstract

The COVID-19 pandemic has significantly disrupted global Supply Chains (SCs), emphasizing the importance of SC resilience, which refers to the ability of SCs to return to their original or more desirable state following disruptions. This study focuses on collaboration, a key component of SC resilience, and proposes a novel collaborative structure that incorporates a fictitious agent to manage inventory transshipment decisions between retailers in a centralized manner while maintaining the retailers’ autonomy in ordering. The proposed collaborative structure offers the following advantages from SC resilience and operational perspectives: (i) it facilitates decision synchronization for enhanced collaboration among retailers, and (ii) it allows retailers to collaborate without the need for information sharing, addressing the potential issue of information sharing reluctance. Additionally, this study employs non-stationary probability to capture the deeply uncertain nature of the ripple effect and the highly volatile customer demand caused by the pandemic. A new Reinforcement Learning (RL) algorithm is developed to handle non-stationary environments and to implement the proposed collaborative structure. Experimental results demonstrate that the proposed collaborative structure using the new RL algorithm achieves superior SC resilience compared with centralized inventory management systems with transshipment and decentralized inventory management systems without transshipment using traditional RL algorithms.

Keywords:

• Pandemic
• supply chain
• ripple effect
• deep uncertainty
• long-lasting crisis
• resilience
• collaboration
• transshipment
• multi-agent reinforcement learning

Data availability statement

The data that support the findings of this study can be generated by using a code openly available in GitHub at https://github.com/Byeongmok/multiagentRL (Kim, 2023).

Additional information

Notes on contributors

Byeongmok Kim

Byeongmok Kim is currently pursuing his PhD in the School of Industrial Engineering at Purdue University (West Lafayette, IN, USA). He earned his BS degree in Industrial Engineering from Hongik University (Seoul, South Korea) and his M.S. degree in Industrial and Management Engineering from POSTECH (Pohang, South Korea). Prior to entering Purdue University, he worked as a research engineer at LG Electronics and Hyundai Steel. His research interests encompass the application of operations research in manufacturing, logistics, supply chain management, and autonomous robotic delivery.

Jong Gwang Kim

Jong Gwang Kim is a PhD student in the School of Industrial Engineering at Purdue University. He received his Master's degree in Applied Mathematics from Columbia University and Bachelor's degrees in Business Administration and Economics from Yonsei University (Korea). His research focuses on the theory and computational aspects of algorithms for large-scale constrained optimization, with applications in game theory, operations research, and machine learning.

Seokcheon Lee

Seokcheon Lee received his BS and MS degrees in Industrial Engineering from Seoul National University (Seoul, South Korea) in 1991 and 1993, respectively, and his PhD degree in Industrial Engineering from Pennsylvania State University (PA, USA) in 2005. He is currently a professor in the School of Industrial Engineering at Purdue University (West Lafayette, IN, USA). His current research interests include optimization techniques from multidisciplinary perspectives and distributed control for logistics.