A multi-agent reinforcement learning model for inventory transshipments under supply chain disruption

References

• Afridi, M.T., Nieto-Isaza, S., Ehm, H., Ponsignon, T. and Hamed, A. (2020) A deep reinforcement learning approach for optimal replenishment policy in a vendor managed inventory setting for semiconductors, in 2020 Winter Simulation Conference (WSC), IEEE Press, Piscataway, NJ, pp. 1753–1764.
   Google Scholar
• Akbari, M. and Do, T.N.A. (2021) A systematic review of machine learning in logistics and supply chain management: Current trends and future directions. Benchmarking: An International Journal, 28(10), 2977–3005.
   Web of Science ®Google Scholar
• Ali, A., Mahfouz, A. and Arisha, A. (2017) Analysing supply chain resilience: Integrating the constructs in a concept mapping framework via a systematic literature review. Supply Chain Management, 22(1), 16–39.
   Web of Science ®Google Scholar
• Baryannis, G., Validi, S., Dani, S. and Antoniou, G. (2019) Supply chain risk management and artificial intelligence: State of the art and future research directions. International Journal of Production Research, 57(7), 2179–2202.
   Web of Science ®Google Scholar
• Bassiouni, M.M., Chakrabortty, R.K., Hussain, O.K. and Rahman, H.F. (2023) Advanced deep learning approaches to predict supply chain risks under COVID-19 restrictions. Expert Systems with Applications, 211, 118604.
   PubMed Web of Science ®Google Scholar
• Bechtsis, D., Tsolakis, N., Iakovou, E. and Vlachos, D. (2022) Data-driven secure, resilient and sustainable supply chains: gaps, opportunities, and a new generalised data sharing and data monetisation framework. International Journal of Production Research, 60(14), 4397–4417.
   Web of Science ®Google Scholar
• Belhadi, A., Kamble, S., Fosso Wamba, S. and Queiroz, M.M. (2022) Building supply-chain resilience: An artificial intelligence-based technique and decision-making framework. International Journal of Production Research, 60(14), 4487–4507.
   Web of Science ®Google Scholar
• Brintrup, A., Pak, J., Ratiney, D., Pearce, T., Wichmann, P., Woodall, P. and McFarlane, D. (2020) Supply chain data analytics for predicting supplier disruptions: A case study in complex asset manufacturing. International Journal of Production Research, 58(11), 3330–3341.
   Web of Science ®Google Scholar
• Chan, F.T. and Prakash, A. (2012) Inventory management in a lateral collaborative manufacturing supply chain: A simulation study. International Journal of Production Research, 50(16), 4670–4685.
   Web of Science ®Google Scholar
• Daniel, J.S.R. and Rajendran, C. (2005) A simulation-based genetic algorithm for inventory optimization in a serial supply chain. International Transactions in Operational Research, 12(1), 101–127.
   Google Scholar
• Dolgui, A., Ivanov, D. and Rozhkov, M. (2020) Does the ripple effect influence the bullwhip effect? An integrated analysis of structural and operational dynamics in the supply chain. International Journal of Production Research, 58(5), 1285–1301.
   Web of Science ®Google Scholar
• Dolgui, A., Ivanov, D. and Sokolov, B. (2018) Ripple effect in the supply chain: An analysis and recent literature. International Journal of Production Research, 56(1-2), 414–430.
   Web of Science ®Google Scholar
• Dong, L. and Rudi, N. (2004) Who benefits from transshipment? Exogenous vs. endogenous wholesale prices. Management Science, 50(5), 645–657.
   Web of Science ®Google Scholar
• Dubey, R., Gunasekaran, A., Childe, S.J., Fosso Wamba, S., Roubaud, D. and Foropon, C. (2021) Empirical investigation of data analytics capability and organizational flexibility as complements to supply chain resilience. International Journal of Production Research, 59(1), 110–128.
   Web of Science ®Google Scholar
• Dye, C.Y. (2020) Optimal joint dynamic pricing, advertising and inventory control model for perishable items with psychic stock effect. European Journal of Operational Research, 283(2), 576–587.
   Web of Science ®Google Scholar
• Ergun, O., Hopp, W.J. and Keskinocak, P. (2022) A structured overview of insights and opportunities for enhancing supply chain resilience. IISE Transactions, 55(1), 57–74.
   Web of Science ®Google Scholar
• Freeland, J.R. (1991) A survey of just-in-time purchasing practices in the United States. Production and Inventory Management Journal, 32(2), 43–50.
   Google Scholar
• Ganesan, V.K., Sundararaj, D. and Srinivas, A.P. (2021) Adaptive inventory replenishment for dynamic supply chains with uncertain market demand, in Industry 4.0 and Advanced Manufacturing: Proceedings of I-4AM 2019, Springer Singapore, pp. 325–335.
   Google Scholar
• Gholami-Zanjani, S.M., Jabalameli, M.S., Klibi, W. and Pishvaee, M.S. (2021) A robust location-inventory model for food supply chains operating under disruptions with ripple effects. International Journal of Production Research, 59(1), 301–324.
   Web of Science ®Google Scholar
• Güllü, R. and Tomsuk, D. (2018) Managing supply disruption in a two-echelon distribution network, in Proceedings of the IIE Annual Conference, IISE, Orlando, FL, pp. 1931–1936.
   Google Scholar
• Gupta, V. and Ivanov, D. (2020) Dual sourcing under supply disruption with risk-averse suppliers in the sharing economy. International Journal of Production Research, 58(1), 291–307.
   Web of Science ®Google Scholar
• Ha, A.Y., Tian, Q. and Tong, S. (2017) Information sharing in competing supply chains with production cost reduction. Manufacturing & Service Operations Management, 19(2), 246–262.
   Web of Science ®Google Scholar
• Hennelly, P.A., Srai, J.S., Graham, G. and Fosso Wamba, S. (2020) Rethinking supply chains in the age of digitalization. Production Planning & Control, 31(2-3), 93–95.
   Web of Science ®Google Scholar
• Hosseini, S. and Ivanov, D. (2022) A multi-layer Bayesian network method for supply chain disruption modelling in the wake of the COVID-19 pandemic. International Journal of Production Research, 60(17), 5258–5276.
   Web of Science ®Google Scholar
• Ivanov, D. (2018) Structural Dynamics and Resilience in Supply Chain Risk Management, Springer International Publishing, Berlin, Germany.
   Google Scholar
• Ivanov, D. (2020) Predicting the impacts of epidemic outbreaks on global supply chains: A simulation-based analysis on the coronavirus outbreak (COVID-19/SARS-CoV-2) case. Transportation Research Part E: Logistics and Transportation Review, 136, 101922.
   PubMed Web of Science ®Google Scholar
• Ivanov, D. (2021) Introduction to Supply Chain Resilience: Management, Modelling, Technology. Springer Nature, Cham, Switzerland.
   Google Scholar
• Ivanov, D. and Das, A. (2020) Coronavirus (COVID-19/SARS-CoV-2) and supply chain resilience: A research note. International Journal of Integrated Supply Management, 13(1), 90–102.
   Google Scholar
• Ivanov, D. and Dolgui, A. (2020) Viability of intertwined supply networks: Extending the supply chain resilience angles towards survivability. A position paper motivated by COVID-19 outbreak. International Journal of Production Research, 58(10), 2904–2915.
   Web of Science ®Google Scholar
• Ivanov, D. and Dolgui, A. (2021) OR-methods for coping with the ripple effect in supply chains during COVID-19 pandemic: Managerial insights and research implications. International Journal of Production Economics, 232, 107921.
   PubMedGoogle Scholar
• Ivanov, D., Dolgui, A. and Sokolov, B. (2019) Ripple effect in the supply chain: Definitions, frameworks and future research perspectives in Handbook of Ripple Effects in the Supply Chain, Springer, Cham, pp. 1–33.
   Google Scholar
• Ivanov, D., Pavlov, A. and Sokolov, B. (2014) Optimal distribution (re) planning in a centralized multi-stage supply network under conditions of the ripple effect and structure dynamics. European Journal of Operational Research, 237(2), 758–770.
   Web of Science ®Google Scholar
• Ivanov, D., Sokolov, B. and Dolgui, A. (2014) The ripple effect in supply chains: Trade-off ‘efficiency-flexibility-resilience’ in disruption management. International Journal of Production Research, 52(7), 2154–2172.
   Web of Science ®Google Scholar
• Ivanov, D., Tsipoulanidis, A. and Schönberger, J. (2021) Supply chain risk management and resilience, in Global Supply Chain and Operations Management, Springer, Cham, pp. 485–520.
   Google Scholar
• Jain, V., Kumar, S., Soni, U. and Chandra, C. (2017) Supply chain resilience: Model development and empirical analysis. International Journal of Production Research, 55(22), 6779–6800.
   Web of Science ®Google Scholar
• Jauhar, S.K., Jani, S.M., Kamble, S.S., Pratap, S., Belhadi, A. and Gupta, S. (2023) How to use no-code artificial intelligence to predict and minimize the inventory distortions for resilient supply chains. International Journal of Production Research (in press and online available).
   PubMed Web of Science ®Google Scholar
• Jin, M. and Agirbas, G. (2013) Reducing expiry of slow-moving critical medical items with transshipment. IIE Transactions on Healthcare Systems Engineering, 3(3), 193–206.
   Google Scholar
• Kara, A. and Dogan, I. (2018) Reinforcement learning approaches for specifying ordering policies of perishable inventory systems. Expert Systems with Applications, 91, 150–158.
   Web of Science ®Google Scholar
• Karabağ, O. and Tan, B. (2019) Purchasing, production, and sales strategies for a production system with limited capacity, fluctuating sales and purchasing prices. IISE Transactions, 51(9), 921–942.
   Web of Science ®Google Scholar
• Katok, E. and Villa, S. (2022) Centralized or decentralized transfer prices: A behavioral approach for improving supply chain coordination. Manufacturing & Service Operations Management, 24(1), 143–158.
   Google Scholar
• Katsaliaki, K., Galetsi, P. and Kumar, S. (2021) Supply chain disruptions and resilience: A major review and future research agenda. Annals of Operations Research, 319, 965–1002.
   PubMedGoogle Scholar
• Khan, K., Su, C.W. and Zhu, M.N. (2022) Examining the behaviour of energy prices to COVID-19 uncertainty: A quantile on quantile approach. Energy, 239, 122430.
   PubMed Web of Science ®Google Scholar
• Kim, B. (2023) GitHub. Available at https://github.com/Byeongmok/multiagentRL (accessed 6 March 2023).
   Google Scholar
• Klibi, W., Martel, A. and Guitouni, A. (2010) The design of robust value-creating supply chain networks: A critical review. European Journal of Operational Research, 203(2), 283–293.
   Web of Science ®Google Scholar
• Kranenburg, A.A. and Van Houtum, G.J. (2009) A new partial pooling structure for spare parts networks. European Journal of Operational Research, 199(3), 908–921.
   Web of Science ®Google Scholar
• Li, R. (2020) Reinvent retail supply chain: Ship-from-store-to-store. Production and Operations Management, 29(8), 1825–1836.
   Web of Science ®Google Scholar
• Li, G., Xue, J., Li, N. and Ivanov, D. (2022) Blockchain-supported business model design, supply chain resilience, and firm performance. Transportation Research Part E: Logistics and Transportation Review, 163, 102773.
   Web of Science ®Google Scholar
• Littman, M.L. (1994) Markov games as a framework for multi-agent reinforcement learning, in Machine Learning Proceedings 1994, Morgan Kaufmann, New Brunswick, NJ, pp. 157–163.
   Google Scholar
• Littman, M.L. (2001) Friend-or-foe Q-learning in general-sum games, in Proceedings of the 18th International Conference on Machine Learning (ICML‘01), Morgan Kaufmann, Williamstown, MA, pp. 322–328.
   Google Scholar
• Lohmer, J., Bugert, N. and Lasch, R. (2020) Analysis of resilience strategies and ripple effect in blockchain-coordinated supply chains: An agent-based simulation study. International Journal of Production Economics, 228, 107882.
   PubMed Web of Science ®Google Scholar
• Mariappan, M.B., Devi, K. and Venkataraman, Y. (2023) Predicting order processing times in e-pharmacy supply chains during COVID pandemic using machine learning—A real-world study, in Proceedings of International Conference on Data Science and Applications (ICDSA 2022), Springer Nature, Singapore, pp. 175–197.
   Google Scholar
• Mehrotra, S., Rahimian, H., Barah, M., Luo, F. and Schantz, K. (2020) A model of supply-chain decisions for resource sharing with an application to ventilator allocation to combat COVID-19. Naval Research Logistics (NRL), 67(5), 303–320.
   PubMed Web of Science ®Google Scholar
• Min, H. (2010) Artificial intelligence in supply chain management: Theory and applications. International Journal of Logistics: Research and Applications, 13(1), 13–39.
   Web of Science ®Google Scholar
• Mohammadi, M., Dehghan, M., Pirayesh, A. and Dolgui, A. (2022) Bi-objective optimization of a stochastic resilient vaccine distribution network in the context of the COVID-19 pandemic. Omega, 113, 102725.
   PubMed Web of Science ®Google Scholar
• Mortazavi, A., Khamseh, A.A. and Azimi, P. (2015) Designing of an intelligent self-adaptive model for supply chain ordering management system. Engineering Applications of Artificial Intelligence, 37, 207–220.
   Web of Science ®Google Scholar
• Naderi, S., Kilic, K. and Dasci, A. (2020) A deterministic model for the transshipment problem of a fast fashion retailer under capacity constraints. International Journal of Production Economics, 227, 107687.
   Web of Science ®Google Scholar
• Nakandala, D., Lau, H. and Shum, P.K. (2017) A lateral transshipment model for perishable inventory management. International Journal of Production Research, 55(18), 5341–5354.
   Web of Science ®Google Scholar
• Nguyen, H., Sharkey, T.C., Mitchell, J.E. and Wallace, W.A. (2020) Optimizing the recovery of disrupted single-sourced multi-echelon assembly supply chain networks. IISE Transactions, 52(7), 703–720.
   Web of Science ®Google Scholar
• Onar, S.C., Kahraman, C. and Oztaysi, B. (2020) Multi-criteria spherical fuzzy regret based evaluation of healthcare equipment stocks. Journal of Intelligent & Fuzzy Systems, 39(5), 5987–5997.
   Web of Science ®Google Scholar
• Paul, S.K. and Chowdhury, P. (2020) A production recovery plan in manufacturing supply chains for a high-demand item during COVID-19. International Journal of Physical Distribution & Logistics Management, 51(2), 104–125.
   Web of Science ®Google Scholar
• Porteus, E.L. (1990) Stochastic inventory theory. Handbooks in Operations Research and Management Science, 2, 605–652.
   Google Scholar
• Raj Sinha, P., Whitman, L.E. and Malzahn, D. (2004) Methodology to mitigate supplier risk in an aerospace supply chain. Supply Chain Management, 9(2), 154–168.
   Web of Science ®Google Scholar
• Rolf, B., Jackson, I., Müller, M., Lang, S., Reggelin, T. and Ivanov, D. (2022) A review on reinforcement learning algorithms and applications in supply chain management. International Journal of Production Research (in press and online available).
   Web of Science ®Google Scholar
• Rudi, N., Kapur, S. and Pyke, D.F. (2001) A two-location inventory model with transshipment and local decision making. Management Science, 47(12), 1668–1680.
   Web of Science ®Google Scholar
• Russell, S.J. and Norvig, P. (2016) Artificial Intelligence: A Modern Approach, Pearson Education, Limited, Malaysia.
   Google Scholar
• Schneiderman, A.M. (1996) Metrics for the order fulfillment process (part 1). Journal of Cost Management, 10, 30–42.
   Google Scholar
• Seifert, R.W., Thonemann, U.W. and Sieke, M.A. (2006) Relaxing channel separation: Integrating a virtual store into the supply chain via transshipments. IIE Transactions, 38(11), 917–931.
   Web of Science ®Google Scholar
• Shanker, S., Barve, A., Muduli, K., Kumar, A., Garza-Reyes, J.A. and Joshi, S. (2022) Enhancing resiliency of perishable product supply chains in the context of the COVID-19 outbreak. International Journal of Logistics Research and Applications, 25(9), 1219–1243.
   Web of Science ®Google Scholar
• Shen, Z.M. and Sun, Y. (2021) Strengthening supply chain resilience during COVID-19: A case study of JD.com. Journal of Operations Management, 69(3), 359–383.
   Web of Science ®Google Scholar
• Sheth, J. (2020) Impact of Covid-19 on consumer behavior: Will the old habits return or die? Journal of Business Research, 117, 280–283.
   PubMed Web of Science ®Google Scholar
• Shih, W.C. (2020) Global supply chains in a post-pandemic world, Harvard Business Review, Available at https://hbr.org/2020/09/global-supply-chains-in-a-post-pandemic-world (accessed 8 June 2022).
   Google Scholar
• Shokouhifar, M. and Ranjbarimesan, M. (2022) Multivariate time-series blood donation/demand forecasting for resilient supply chain management during COVID-19 pandemic. Cleaner Logistics and Supply Chain, 5, 100078.
   Google Scholar
• Simatupang, T.M. and Sridharan, R. (2008) Design for supply chain collaboration. Business Process Management Journal, 14(3), 401–418.
   Google Scholar
• Singh, S., Kumar, R., Panchal, R. and Tiwari, M.K. (2021) Impact of COVID-19 on logistics systems and disruptions in food supply chain. International Journal of Production Research, 59(7), 1993–2008.
   Web of Science ®Google Scholar
• Slikker, M., Fransoo, J. and Wouters, M. (2005) Cooperation between multiple news-vendors with transshipments. European Journal of Operational Research, 167(2), 370–380.
   Web of Science ®Google Scholar
• Smirnov, D. and Gerchak, Y. (2014) Inventory sharing via circular unidirectional chaining. European Journal of Operational Research, 237(2), 474–486.
   Web of Science ®Google Scholar
• Sutton, R.S. and Barto, A.G. (2018) Reinforcement Learning: An Introduction. MIT press, Cambridge, MA.
   Google Scholar
• Snyder, L.V., Atan, Z., Peng, P., Rong, Y., Schmitt, A.J. and Sinsoysal, B. (2016) OR/MS models for supply chain disruptions: A review. IIE Transactions, 48(2), 89–109.
   Web of Science ®Google Scholar
• Tagaras, G. (1999) Pooling in multi-location periodic inventory distribution systems. Omega, 27(1), 39–59.
   Web of Science ®Google Scholar
• Taylor, C. (2020) Sales of hand sanitizer are skyrocketing due to the coronavirus, leading to rationing and price hikes. CNBC. CNBC, March, 4, Available at https://www.cnbc.com/2020/03/03/coronavirus-hand-sanitizer-sales-surge-leading-to-price-hikes.html (accessed 17 November 2022).
   Google Scholar
• Tucker, E.L., Daskin, M.S., Sweet, B.V. and Hopp, W.J. (2020) Incentivizing resilient supply chain design to prevent drug shortages: Policy analysis using two-and multi-stage stochastic programs. IISE Transactions, 52(4), 394–412.
   Web of Science ®Google Scholar
• Tukamuhabwa, B.R., Stevenson, M., Busby, J. and Zorzini, M. (2015) Supply chain resilience: Definition, review and theoretical foundations for further study. International Journal of Production Research, 53(18), 5592–5623.
   Web of Science ®Google Scholar
• Vali-Siar, M.M. and Roghanian, E. (2022) Sustainable, resilient and responsive mixed supply chain network design under hybrid uncertainty with considering COVID-19 pandemic disruption. Sustainable Production and Consumption, 30, 278–300.
   PubMed Web of Science ®Google Scholar
• Vicente, J.J., Relvas, S. and Póvoa, A.P.B. (2015) Multi-period and multi-product inventory management model with lateral transshipments in Operational Research, Springer, Cham, pp. 425–444.
   Google Scholar
• Watkins, C.J.C.H. (1989) Learning from delayed rewards. PhD Thesis, University of Cambridge, England.
   Google Scholar
• Watkins, C.J. and Dayan, P. (1992) Q-learning. Machine Learning, 8(3-4), 279–292.
   Web of Science ®Google Scholar
• Wong, T.E. and Keller, K. (2017) Deep uncertainty surrounding coastal flood risk projections: A case study for New Orleans. Earth's Future, 5(10), 1015–1026.
   Web of Science ®Google Scholar
• Wong, T.E., Klufas, A., Srikrishnan, V. and Keller, K. (2018) Neglecting model structural uncertainty underestimates upper tails of flood hazard. Environmental Research Letters, 13(7), 074019.
   Web of Science ®Google Scholar
• Yan, Y., Chow, A.H., Ho, C.P., Kuo, Y.H., Wu, Q. and Ying, C. (2022) Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities. Transportation Research Part E: Logistics and Transportation Review, 162, 102712.
   Web of Science ®Google Scholar
• Yao, D.D., Zhou, S.X. and Zhuang, W. (2016) Joint initial stocking and transshipment—Asymptotics and bounds. Production and Operations Management, 25(2), 273–289.
   Web of Science ®Google Scholar
• Yu, H., Zeng, A.Z. and Zhao, L. (2009) Single or dual sourcing: Decision-making in the presence of supply chain disruption risks. Omega, 37(4), 788–800.
   Web of Science ®Google Scholar
• Yu, C., Zhou, Y. and Zhang, Z. (2020) Multi-agent reinforcement learning for dynamic spare parts inventory control, in 2020 Global Reliability and Prognostics and Health Management, IEEE Press, Piscataway, NJ, pp. 1–6.
   Google Scholar
• Zhang, J. (2005) Transshipment and its impact on supply chain members’ performance. Management Science, 51(10), 1534–1539.
   Web of Science ®Google Scholar
• Zhao, H., Deshpande, V. and Ryan, J.K. (2005) Inventory sharing and rationing in decentralized dealer networks. Management Science, 51(4), 531–547.
   Web of Science ®Google Scholar