Real-time AGV scheduling optimisation method with deep reinforcement learning for energy-efficiency in the container terminal yard

References

• Bartošek, A., and O. Marek. 2013. “Quay Cranes in Container Terminals.” Transactions on Transport Sciences 6 (1): 9–18. https://doi.org/10.2478/v10158-012-0027-y.
   Google Scholar
• Chen, C., Z. H. Hu, and L. Wang. 2021. “Scheduling of AGVs in Automated Container Terminal Based on the Deep Deterministic Policy Gradient (DDPG) Using the Convolutional Neural Network (CNN).” Journal of Marine Science and Engineering 9 (12): 1439. https://doi.org/10.3390/jmse9121439.
   Web of Science ®Google Scholar
• Drungilas, D., M. Kurmis, A. Senulis, Z. Lukosius, A. Andziulis, J. Januteniene, M. Bogdevicius, V. Jankunas, and M. Voznak. 2023. “Deep Reinforcement Learning Based Optimization of Automated Guided Vehicle Time and Energy Consumption in a Container Terminal.” Alexandria Engineering Journal 67: 397–407. https://doi.org/10.1016/J.AEJ.2022.12.057.
   Web of Science ®Google Scholar
• Elsisi, M., and Minh-Quang Tran. 2021. “Development of An IoT Architecture Based on a Deep Neural Network Against Cyber Attacks for Automated Guided Vehicles.” Sensors 21 (24): 8467. https://doi.org/10.3390/s21248467.
   PubMed Web of Science ®Google Scholar
• Fan, H. M., Z. F. Guo, L. J. Yue, and M. Z. Ma. 2021. “Joint Configuration and Scheduling Optimization of Dual-trolley Quay Crane and AGV for Container Terminal with Considering Energy Saving.” Acta Automatica Sinica 47 (10): 2412–2426. https://doi.org/10.16383/j.aas.c190626.
   Google Scholar
• Guruji, A. K., H. Agarwal, and D. K. Parsediya. 2016. “Time-efficient A* Algorithm for Robot Path Planning.” Procedia Technology 23: 144–149. https://doi.org/10.1016/j.protcy.2016.03.010.
   Google Scholar
• Hu, H., X. Jia, Q. He, S. Fu, and K. Liu. 2020. “Deep Reinforcement Learning Based AGVs Real-time Scheduling with Mixed Rule for Flexible Shop Floor in Industry 4.0.” Computers & Industrial Engineering 149: 106749. https://doi.org/10.1016/j.cie.2020.106749.
   Web of Science ®Google Scholar
• Hu, H. T., X. R. Yang, S. C. Xiao, and F. Y. Wang. 2023. “Anti-conflict AGV Path Planning in Automated Container Terminals Based on Multi-agent Reinforcement Learning.” International Journal of Production Research 61 (1): 65–80. https://doi.org/10.1080/00207543.2021.1998695.
   Web of Science ®Google Scholar
• Huang, D., Z. F. Mao, K. Fang, and L. Chen. 2023. “Solving the Shortest Path Interdiction Problem Via Reinforcement Learning.” International Journal of Production Research 61 (1): 31–48. https://doi.org/10.1080/00207543.2021.2002962.
   Web of Science ®Google Scholar
• Ihsan, S., and D. L. Hommertzheim. 1992. “Experimental Investigation of FMS Machine and AGV Scheduling Rules Against the Mean Flow-time Criterion.” International Journal of Production Research30 (7): 1617–1635. https://doi.org/10.1080/00207549208948110.
   Web of Science ®Google Scholar
• Jin, J., and X. H. Zhang. 2016. “MULTI AGV SCHEDULING PROBLEM IN AUTOMATED CONTAINER TERMINAL.” Journal of Marine Science & Technology 24 (1): 32–38.
   Web of Science ®Google Scholar
• Kim, J., R. Choe, and K. R. Ryu. 2013. “Multi-objective Optimization of Dispatching Strategies for Situation-Adaptive AGV Operation in An Automated Container Terminal.” In Research in Adaptive & Convergent Systems, 1–6. New York, NY: ACM.
   Google Scholar
• Li, Linman, Yuqing Li, Ran Liu, Yaoming Zhou, and Ershun Pan. 2023. “A Two-stage Stochastic Programming for AGV Scheduling with Random Tasks and Battery Swapping in Automated Container Terminals.” Transportation Research Part E 174: 103110. https://doi.org/10.1016/J.TRE.2023.103110.
   Google Scholar
• Lombard, A., F. Perronnet, A. Abbas-Turki, and A. E. Moudni. 2016. “Decentralized Management of Intersections of Automated Guided Vehicles.” IFAC-PapersOnLine 49 (12): 497–502. https://doi.org/10.1016/j.ifacol.2016.07.669.
   Google Scholar
• Lowe, R., Y. Wu, A. Tamar, and J. Harb. 2017. “Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments.” arXiv:1706.02275.
   Google Scholar
• Lu, Z., E. P. Chew, and L. H. Lee. 2011. “An Integrated Model for Berth Template and Yard Template Planning in Transshipment Hubs.” Transportation Science 45 (4): 483–504. https://doi.org/10.1287/trsc.1100.0364.
   Web of Science ®Google Scholar
• Marcel, P., and B. Bender. 2022. “Deep Reinforcement Learning in Production Systems: a Systematic Literature Review.” International Journal of Production Research 60: 13. https://doi.org/10.1080/00207543.2021.1973138.
   Web of Science ®Google Scholar
• Rashidi, H., and E. Tsang. 2011. “A Complete and An Incomplete Algorithm for Automated Guided Vehicle Scheduling in Container Terminals.” Computers & Mathematics with Applications: An International Journal 61 (3): 630–641. https://doi.org/10.1016/j.camwa.2010.12.009.
   Web of Science ®Google Scholar
• Wang, C., C. Jin, and Z. Li. 2019. “Bilevel Programming Model of Low Energy Consumption AGV Scheduling Problem at Automated Container Terminal.” In 2019 IEEE International Conference on Smart Manufacturing, Industrial & Logistics Engineering (SMILE), IEEE, 2019.
   Google Scholar
• Wang, Z., and Q. Zeng. 2022. “A Branch-and-bound Approach for AGV Dispatching and Routing Problems in Automated Container Terminals.” Computers & Industrial Engineering 166: 107968. https://doi.org/10.1016/j.cie.2022.107968.
   Web of Science ®Google Scholar
• Wen, X., Y. G. Sun, H. Ma, and S. Chun. 2022. “Green Smart Manufacturing: Energy-efficient Robotic Job Shop Scheduling Models.” International Journal of Production Research 61 (17): 5791–5805. https://doi.org/10.1080/00207543.2022.2112989.
   Web of Science ®Google Scholar
• Wu, T. H., M. Z. Jiang, and L. Zhang. 2020. “Cooperative Multiagent Deep Deterministic Policy Gradient (CoMADDPG) for Intelligent Connected Transportation with Unsignalized Intersection.” Mathematical Problems in Engineering 2020. https://doi.org/10.1155/2020/1820527.
   Google Scholar
• Xiang, Xi, and Changchun Liu. 2021. “Modeling and Analysis for An Automated Container Terminal Considering Battery Management.” Computers & Industrial Engineering 156: 107258. https://doi.org/10.1016/J.CIE.2021.107258.
   Web of Science ®Google Scholar
• Xiang, X., C. Liu, L. H. Lee, and E. P. Chew. 2022. “Performance Estimation and Design Optimization of a Congested Automated Container Terminal.” IEEE Transactions on Automation Science and Engineering 19 (3): 2437–2449. https://doi.org/10.1109/TASE.2021.3085329.
   Web of Science ®Google Scholar
• Xin, J., R. R. Negenborn, and G. Lodewijks. 2014. “Energy-aware Control for Automated Container Terminals Using Integrated Flow Shop Scheduling and Optimal Control.” Transportation Research Part C 44: 214–230. https://doi.org/10.1016/j.trc.2014.03.014.
   Web of Science ®Google Scholar
• Xu, B., J. J. Wang, Z. M. Zhang, and Y. N. Jiang. 2023. “Research on Distributed Algorithm of AGV Dispatching with Pooled Dispatching Strategy in Automated Container Terminal.” Journal of Research in Science and Engineering1). https://doi.org/10.53469/JRSE.2023.05(01).04.
   Google Scholar
• Yang, Y., M. Zhong, Y. Dessouky, and O. Postolache. 2018. “An Integrated Scheduling Method for AGV Routing in Automated Container Terminals.” Computers & Industrial Engineering 126 (12): 482–493. https://doi.org/10.1016/j.cie.2018.10.007.
   Google Scholar
• Yue, L. J., and H. M. Fan. 2022. “Dynamic Scheduling and Path Planning of Automated Guided Vehicles in Automatic Container Terminal.” IEEE/CAA Journal of Automatica Sinica 9 (11): 2005–2019. https://doi.org/10.1109/JAS.2022.105950.
   Google Scholar
• Zhang, H. C., Q. Liang, W. J. Luan, and H. J. Ma. 2022. “Double-Cycling AGV Scheduling Considering Uncertain Crane Operational Time At Container Terminals.” Applied Sciences 12 (10): 4820. https://doi.org/10.3390/APP12104820.
   Google Scholar
• Zhang, L., C. Yang, Y. Yan, and Y. Hu. 2022. “Distributed Real-time Scheduling in Cloud Manufacturing by Deep Reinforcement Learning.” IEEE Transactions on Industrial Informatics 18 (12): 8999–9007. https://doi.org/10.1109/TII.2022.3178410.
   Web of Science ®Google Scholar
• Zhang, Y., Y. Zhou, H. Lu, and H. Fujita. 2021. “Cooperative Multi-agent Actor-critic Control of Traffic Network Flow Based on Edge Computing.” Future Generation Computer Systems 123: 128–141. https://doi.org/10.1016/j.future.2021.04.018.
   Web of Science ®Google Scholar
• Zheng, F., Y. Li, F. Chu, M. Liu, and Y. Xu. 2018. “Integrated Berth Allocation and Quay Crane Assignment with Maintenance Activities.” International Journal of Production Research 57 (11): 3478–3503. https://doi.org/10.1080/00207543.2018.1539265.
   Web of Science ®Google Scholar
• Zhong, M., Y. Yang, Y. Dessouky, and O. Postolache. 2020. “Multi-AGV Scheduling for Conflict-free Path Planning in Automated Container Terminals.” Computers & Industrial Engineering 142 (4): 106371.1–106371.11.
   Google Scholar