Multi-resource constrained dynamic workshop scheduling based on proximal policy optimisation

References

• Aydin, M. E., and E. Öztemel. 2000. “Dynamic Job-shop Scheduling Using Reinforcement Learning Agents.” Robotics and Autonomous Systems 33 (2-3): 169–178.
   Web of Science ®Google Scholar
• Bellemare, M. G., W. Dabney, and R. Munos. 2017. “A Distributional Perspective on Reinforcement Learning.” In Paper presented at 34th International Conference on Machine Learning, Sydney, August 6–11.
   Google Scholar
• Berner, C., G. Brockman, B. Chan, V. Cheung, P. Dębiak, C. Dennison, D. Farhi, et al. 2019. “Dota 2 With Large Scale Deep Reinforcement Learning.” arXiv preprint arXiv:1912.06680.
   Google Scholar
• Brockman, G., V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. 2016. “Openai Gym.” arXiv preprint arXiv:1606.01540.
   Google Scholar
• Cao, ZhengCai, ChengRan Lin, and MengChu Zhou. 2019. “A Knowledge-based Cuckoo Search Algorithm to Schedule a Flexible Job Shop with Sequencing Flexibility.” IEEE Transactions on Automation Science and Engineering 18 (1): 56–69.
   Web of Science ®Google Scholar
• Cao, ZhengCai, ChengRan Lin, MengChu Zhou, and Ran Huang. 2018. “Scheduling Semiconductor Testing Facility by Using Cuckoo Search Algorithm with Reinforcement Learning and Surrogate Modeling.” IEEE Transactions on Automation Science and Engineering 16 (2): 825–837.
   Web of Science ®Google Scholar
• Cao, ZhengCai, ChengRan Lin, MengChu Zhou, and JiaQi Zhang. 2020. “Surrogate-assisted Symbiotic Organisms Search Algorithm for Parallel Batch Processor Scheduling.” IEEE/ASME Transactions on Mechatronics 25 (5): 2155–2166.
   Web of Science ®Google Scholar
• Carrol, D. 1965. “Heuristic Sequencing of Jobs With Single and Multiple Components.” PhD diss., Sloan School of Management, Massachusetts Institute of Technology.
   Google Scholar
• Dou, Jianping, Jun Li, Dan Xia, and Xia Zhao. 2020. “A Multi-objective Particle Swarm Optimisation for Integrated Configuration Design and Scheduling in Reconfigurable Manufacturing System.” International Journal of Production Research 59 (5): 1–21.
   Google Scholar
• Fu, Yaping, MengChu Zhou, Xiwang Guo, and Liang Qi. 2019. “Scheduling Dual-objective Stochastic Hybrid Flow Shop with Deteriorating Jobs Via Bi-population Evolutionary Algorithm.” IEEE Transactions on Systems, Man, and Cybernetics: Systems 50 (12): 5037–5048.
   Web of Science ®Google Scholar
• Gao, Kaizhou, Zhiguang Cao, Le Zhang, Zhenghua Chen, Yuyan Han, and Quanke Pan. 2019. “A Review on Swarm Intelligence and Evolutionary Algorithms for Solving Flexible Job Shop Scheduling Problems.” IEEE/CAA Journal of Automatica Sinica 6 (4): 904–916.
   Google Scholar
• Hessel, M., J. Modayil, H. Van Hasselt, T. Schaul, G. Ostrovski, W. Dabney, D. Horgan, B. Piot, M. Azar, and D. Silver. 2018. “Rainbow: Combining Improvements in Deep Reinforcement Learning.” In Paper presented at 32nd AAAI Conference on Artificial Intelligence, New Orleans, February 02–07.
   Google Scholar
• Kauten, Christian. 2018. “Super Mario Bros for OpenAI Gym.” GitHub. https://github.com/Kautenja/gym-super-mario-bros.
   Google Scholar
• Komaki, G. M., Shaya Sheikh, and Behnam Malakooti. 2019. “Flow Shop Scheduling Problems with Assembly Operations: a Review and New Trends.” International Journal of Production Research 57 (10): 2926–2955.
   Web of Science ®Google Scholar
• Lamothe, Jacques, François Marmier, Matthieu Dupuy, Paul Gaborit, and Lionel Dupont. 2012. “Scheduling Rules to Minimize Total Tardiness in a Parallel Machine Problem with Setup and Calendar Constraints.” Computers & Operations Research 39 (6): 1236–1244.
   Web of Science ®Google Scholar
• Li, Yingli, Xinyu Li, Liang Gao, Biao Zhang, Quan-Ke Pan, M. Fatih Tasgetiren, and Leilei Meng. 2021. “A Discrete Artificial Bee Colony Algorithm for Distributed Hybrid Flowshop Scheduling Problem with Sequence-dependent Setup Times.” International Journal of Production Research 59 (13): 3880–3899.
   Web of Science ®Google Scholar
• Mahadevan, Sridhar. 1996. “Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results.” Machine Learning 22 (1-3): 159–195.
   Web of Science ®Google Scholar
• Mao, Hongzi, Mohammad Alizadeh, Ishai Menache, and Srikanth Kandula. 2016. “Resource Management With Deep Reinforcement Learning.” In Paper presented at Proceedings of the 15th ACM Workshop on Hot Topics in Networks, Atlanta, November 9–10.
   Google Scholar
• Mnih, Volodymyr, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. “Asynchronous Methods for Deep Reinforcement Learning.” In Paper Presented at International Conference on Machine Learning, New York City, June 19–24.
   Google Scholar
• Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. “Playing Atari With Deep Reinforcement Learning.” In Paper presented at Neural Information Processing Systems, Lake Tahoe, December 5–10.
   Google Scholar
• Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, and Alex Graves, et al. 2015. “Human-level Control Through Deep Reinforcement Learning.” Nature 518 (7540): 529–533.
   PubMed Web of Science ®Google Scholar
• Oukil, Amar, and Ahmed El-Bouri. 2021. “Ranking Dispatching Rules in Multi-objective Dynamic Flow Shop Scheduling: A Multi-faceted Perspective.” International Journal of Production Research 59 (2): 388–411.
   Web of Science ®Google Scholar
• Park, Junyoung, Jaehyeong Chun, Sang Hun Kim, Youngkook Kim, and Jinkyoo Park. 2021. “Learning to Schedule Job-shop Problems: Representation and Policy Learning Using Graph Neural Network and Reinforcement Learning.” International Journal of Production Research 59 (11): 3360–3377.
   Web of Science ®Google Scholar
• Paszke, Adam, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, and Trevor Killeen, et al. 2019. “Pytorch: An Imperative Style, High-Performance Deep Learning Library.” In Advances in Neural Information Processing Systems, 8026–8037.
   Google Scholar
• Schaul, Tom, John Quan, Ioannis Antonoglou, and David Silver. 2015. “Prioritized Experience Replay.” In Paper presented at 4th International Conference on Learning Representations, San Juan, May 2–4.
   Google Scholar
• Schulman, John, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. “Trust Region Policy Optimization.” In Paper Presented at International Conference on Machine Learning, Lille, July 6–15.
   Google Scholar
• Schulman, John, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. 2015. “High-Dimensional Continuous Control Using Generalized Advantage Estimation.” In Paper Presented at 4th International Conference on Learning Representations, San Juan, May 2–4.
   Google Scholar
• Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. “Proximal Policy Optimization Algorithms.” arXiv preprint arXiv:1707.06347.
   Google Scholar
• Silver, David, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, and Thomas Hubert, et al. 2017. “Mastering the Game of Go Without Human Knowledge.” Nature 550 (7676): 354–359.
   PubMed Web of Science ®Google Scholar
• Sutton, Richard S., David A. McAllester, Satinder P. Singh, and Yishay Mansour. 2000. “Policy Gradient Methods for Reinforcement Learning With Function Approximation.” In Advances in Neural Information Processing Systems, 1057–1063.
   Google Scholar
• Touzout, Faycal A., and Lyes Benyoucef. 2019. “Multi-objective Multi-unit Process Plan Generation in a Reconfigurable Manufacturing Environment: a Comparative Study of Three Hybrid Metaheuristics.” International Journal of Production Research 57 (24): 7520–7535.
   Web of Science ®Google Scholar
• Van Hasselt, Hado, Arthur Guez, and David Silver. 2016. “Deep Reinforcement Learning With Double Q-Learning.” In Paper Presented at 30th AAAI Conference on Artificial Intelligence, Phoenix, February 12–17.
   Google Scholar
• Vepsalainen, Ari P. J., and Thomas E. Morton. 1987. “Priority Rules for Job Shops with Weighted Tardiness Costs.” Management Science 33 (8): 1035–1047.
   Web of Science ®Google Scholar
• Volgenant, A., and E. Teerhuis. 1999. “Improved Heuristics for the N-job Single-machine Weighted Tardiness Problem.” Computers & Operations Research 26 (1): 35–44.
   Web of Science ®Google Scholar
• Wang, Jufeng, Chunfeng Liu, and MengChu Zhou. 2020. “Improved Bacterial Foraging Algorithm for Cell Formation and Product Scheduling Considering Learning and Forgetting Factors in Cellular Manufacturing Systems.” IEEE Systems Journal 14 (2): 3047–3056.
   Web of Science ®Google Scholar
• Wang, Ling, and Jiawen Lu. 2019. “A Memetic Algorithm with Competition for the Capacitated Green Vehicle Routing Problem.” IEEE/CAA Journal of Automatica Sinica 6 (2): 516–526.
   Google Scholar
• Wang, Ziyu, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. 2016. “Dueling Network Architectures for Deep Reinforcement Learning.” In Paper Presented at International Conference on Machine Learning, New York City, June 19–24.
   Google Scholar
• Wang, Yi-Chi, and John M. Usher. 2005. “Application of Reinforcement Learning for Agent-based Production Scheduling.” Engineering Applications of Artificial Intelligence 18 (1): 73–82.
   Web of Science ®Google Scholar
• Watkins, Christopher J. C. H., and Peter Dayan. 1992. “Q-learning.” Machine Learning 8 (3-4): 279–292.
   Web of Science ®Google Scholar
• Wei, Yingzi, and Mingyang Zhao. 2004. “Composite Rules Selection Using Reinforcement Learning for Dynamic Job-Shop Scheduling.” In Paper presented at IEEE Conference on Robotics, Automation and Mechatronics, Singapore, Dec 1–3.
   Google Scholar
• Yang, Shengluo, and Zhigang Xu. 2021. “The Distributed Assembly Permutation Flowshop Scheduling Problem with Flexible Assembly and Batch Delivery.” International Journal of Production Research 59 (13): 4053–4071.
   Web of Science ®Google Scholar
• Ye, Yufei, Xiaoqin Ren, Jin Wang, Lingxiao Xu, Wenxia Guo, Wenqiang Huang, and Wenhong Tian. 2018. “A New Approach for Resource Scheduling With Deep Reinforcement Learning.” arXiv preprint arXiv:1806.08122.
   Google Scholar
• Zhang, Zhicong, Li Zheng, Na Li, Weiping Wang, Shouyan Zhong, and Kaishun Hu. 2012. “Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling with Reinforcement Learning.” Computers & Operations Research 39 (7): 1315–1324.
   Web of Science ®Google Scholar
• Zhang, Zhicong, Li Zheng, and Michael X. Weng. 2007. “Dynamic Parallel Machine Scheduling with Mean Weighted Tardiness Objective by Q-Learning.” The International Journal of Advanced Manufacturing Technology 34 (9–10): 968–980.
   Web of Science ®Google Scholar