Deep reinforcement learning for solving steelmaking-continuous casting scheduling problems under time-of-use tariffs

References

• Arviv, Kfir, Helman Stern, and Yael Edan. 2016. “Collaborative Reinforcement Learning for a two-Robot job Transfer Flow-Shop Scheduling Problem.” International Journal of Production Research 54 (4): 1196–1209. https://doi.org/10.1080/00207543.2015.1057297.
   Web of Science ®Google Scholar
• Burachik, R. S., C. Y. Kaya, and M. M. Rizvi. 2014. “A New Scalarization Technique to Approximate Pareto Fronts of Problems with Disconnected Feasible Sets.” Journal of Optimization Theory and Applications 162 (2): 428–446. https://doi.org/10.1007/s10957-013-0346-0.
   Web of Science ®Google Scholar
• Cai, Jingcao, Deming Lei, Jing Wang, and Lei Wang. 2022. “A Novel Shuffled Frog-Leaping Algorithm with Reinforcement Learning for Distributed Assembly Hybrid Flow Shop Scheduling.” International Journal of Production Research 61 (4): 1233–1251.
   Web of Science ®Google Scholar
• Che, Ada, Yizeng Zeng, and Ke Lyu. 2016. “An Efficient Greedy Insertion Heuristic for Energy-Conscious Single Machine Scheduling Problem Under Time-of-use Electricity Tariffs.” Journal of Cleaner Production 129: 565–577. https://doi.org/10.1016/j.jclepro.2016.03.150.
   Web of Science ®Google Scholar
• Chen, Shengkai, Shuiliang Fang, and Renzhong Tang. 2019. “A Reinforcement Learning Based Approach for Multi-Projects Scheduling in Cloud Manufacturing.” International Journal of Production Research 57 (10): 3080–3098. https://doi.org/10.1080/00207543.2018.1535205.
   Web of Science ®Google Scholar
• Dai, Min, Dunbing Tang, Adriana Giret, Miguel A Salido, and Wei Dong Li. 2013. “Energy-efficient Scheduling for a Flexible Flow Shop Using an Improved Genetic-Simulated Annealing Algorithm.” Robotics and Computer-Integrated Manufacturing 29 (5): 418–429. https://doi.org/10.1016/j.rcim.2013.04.001.
   Web of Science ®Google Scholar
• Deng, W., J. Xu, X. Z. Gao, and H. Zhao. 2022. ““An Enhanced MSIQDE Algorithm With Novel Multiple Strategies for Global Optimization Problems.” IEEE Transactions on Systems, Man, and Cybernetics: Systems 52 (3): 1578–1587. https://doi.org/10.1109/TSMC.2020.3030792.
   Web of Science ®Google Scholar
• Deng, W., J. Xu, H. Zhao, and Y. Song. 2022. “A Novel Gate Resource Allocation Method Using Improved PSO-Based QEA.” IEEE Transactions on Intelligent Transportation Systems 23 (3): 1737–1745. https://doi.org/10.1109/TITS.2020.3025796.
   Web of Science ®Google Scholar
• Deng, W., X. Zhang, Y. Zhou, Y. Liu, X. Zhou, H. Chen, and H. Zhao. 2022. “An Enhanced Fast non-Dominated Solution Sorting Genetic Algorithm for Multi-Objective Problems.” Information Sciences 585: 441–453. https://doi.org/10.1016/j.ins.2021.11.052.
   Web of Science ®Google Scholar
• Fang, K., N. A. Uhan, F. Zhao, and J. W. Sutherland. 2016. “Scheduling on a Single Machine Under Time-of-use Electricity Tariffs.” Annals of Operations Research 238 (1-2): 199–227. https://doi.org/10.1007/s10479-015-2003-5.
   Web of Science ®Google Scholar
• Gahm, Christian, Florian Denz, Martin Dirr, and Axel Tuma. 2016. “Energy-efficient Scheduling in Manufacturing Companies: A Review and Research Framework.” European Journal of Operational Research 248 (3): 744–757. https://doi.org/10.1016/j.ejor.2015.07.017.
   Web of Science ®Google Scholar
• Geng, Kaifeng, Chunming Ye, Zhen Hua Dai, and Li Liu. 2020. “Bi-Objective Re-Entrant Hybrid Flow Shop Scheduling Considering Energy Consumption Cost Under Time-of-Use Electricity Tariffs.” Complexity 2020: 8565921.
   Web of Science ®Google Scholar
• Han, Bao-An, and Jian-Jun Yang. 2020. “Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN.” IEEE Access 8: 186474–186495. https://doi.org/10.1109/Access.6287639.
   Web of Science ®Google Scholar
• Heger, Jens, and Thomas Voss. 2023. “Dynamically Adjusting the k-Values of the ATCS Rule in a Flexible Flow Shop Scenario with Reinforcement Learning.” International Journal of Production Research 61 (1): 147–161. https://doi.org/10.1080/00207543.2021.1943762.
   Web of Science ®Google Scholar
• Ho, Minh Hung, Faicel Hnaien, and Frederic Dugardin. 2021. “Electricity Cost Minimisation for Optimal Makespan Solution in Flow Shop Scheduling Under Time-of-use Tariffs.” International Journal of Production Research 59 (4): 1041–1067. https://doi.org/10.1080/00207543.2020.1715504.
   Web of Science ®Google Scholar
• Huang, C., X. Zhou, X. Ran, Y. Liu, W. Deng, and W. Deng. 2023. “Co-evolutionary Competitive Swarm Optimizer with Three-Phase for Large-Scale Complex Optimization Problem.” Information Sciences 619: 2–18. https://doi.org/10.1016/j.ins.2022.11.019.
   Web of Science ®Google Scholar
• Kara, Sami, Suphunnika Manmek, and Christoph Herrmann. 2010. “Global Manufacturing and the Embodied Energy of Products.” CIRP Annals 59 (1): 29–32. https://doi.org/10.1016/j.cirp.2010.03.004.
   Web of Science ®Google Scholar
• Lee, Jun-Ho, and Hyun-Jung Kim. 2022. “Reinforcement Learning for Robotic Flow Shop Scheduling with Processing Time Variations.” International Journal of Production Research 60 (7): 2346–2368. https://doi.org/10.1080/00207543.2021.1887533.
   Web of Science ®Google Scholar
• Li, Yuxin, Wenbin Gu, Minghai Yuan, and Yaming Tang. 2022. “Real-Time Data-Driven Dynamic Scheduling for Flexible Job Shop with Insufficient Transportation Resources Using Hybrid Deep Q Network.” Robotics and Computer-Integrated Manufacturing 74: 102283. https://doi.org/10.1016/j.rcim.2021.102283.
   Web of Science ®Google Scholar
• Li, Wen, André Zein, Sami Kara, and Christoph Herrmann. 2011. “An Investigation Into Fixed Energy Consumption of Machine Tools. In Glocalized Solutions for Sustainability in Manufacturing, 268–273. Berlin, Heidelberg: Springer.
   Google Scholar
• Liu, Renke, Rajesh Piplani, and Carlos Toro. 2022. “Deep Reinforcement Learning for Dynamic Scheduling of a Flexible job Shop.” International Journal of Production Research 60 (13): 4049–4069. https://doi.org/10.1080/00207543.2022.2058432.
   Web of Science ®Google Scholar
• Luo, Peng Cheng, Huan Qian Xiong, Bo Wen Zhang, Jie Yang Peng, and Zhao Feng Xiong. 2022. “Multi-resource Constrained Dynamic Workshop Scheduling Based on Proximal Policy Optimisation.” International Journal of Production Research 60 (19): 5937–5955. https://doi.org/10.1080/00207543.2021.1975057.
   Web of Science ®Google Scholar
• Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, et al. 2015. “Human-level Control Through Deep Reinforcement Learning.” Nature 518 (7540): 529–533. https://doi.org/10.1038/nature14236.
   PubMed Web of Science ®Google Scholar
• Moon, J. Y., H. Na, B. K. Choi, and J. Park. 2011. “Modeling and Optimization of Unrelated Parall Machine Scheduling Problem With Timedependent and Machine-Dependent Electricity Cost.” Paper presented at the APMS 2011 international conference, Stavanger (Norway), 2011.
   Google Scholar
• Moon, Joon-Yung, and Jinwoo Park. 2014. “Smart Production Scheduling with Time-Dependent and Machine-Dependent Electricity Cost by Considering Distributed Energy Resources and Energy Storage.” International Journal of Production Research 52 (13): 3922–3939. https://doi.org/10.1080/00207543.2013.860251.
   Web of Science ®Google Scholar
• Mori, M., M. Fujishima, Y. Inamasu, and Y. Oda. 2011. “A Study on Energy Efficiency Improvement for Machine Tools.” Cirp Annals-Manufacturing Technology 60 (1): 145–148. https://doi.org/10.1016/j.cirp.2011.03.099.
   Web of Science ®Google Scholar
• Pan, Ruilin, Zhenghong Li, Jianhua Cao, Hongliang Zhang, and Xue Xia. 2019. “Electrical Load Tracking Scheduling of Steel Plants Under Time-of-use Tariffs.” Computers & Industrial Engineering 137: 106049. https://doi.org/10.1016/j.cie.2019.106049.
   Web of Science ®Google Scholar
• Pan, Ruilin, Qiong Wang, Zhenghong Li, Jianhua Cao, and Yongjin Zhang. 2022. “Steelmaking-continuous Casting Scheduling Problem with Multi-Position Refining Furnaces Under Time-of-use Tariffs.” Annals of Operations Research 310 (1): 119–151. https://doi.org/10.1007/s10479-021-04217-7.
   Web of Science ®Google Scholar
• Park, Junyoung, Jaehyeong Chun, Sang Hun Kim, Youngkook Kim, and Jinkyoo Park. 2021. “Learning to Schedule job-Shop Problems: Representation and Policy Learning Using Graph Neural Network and Reinforcement Learning.” International Journal of Production Research 59 (11): 3360–3377. https://doi.org/10.1080/00207543.2020.1870013.
   Web of Science ®Google Scholar
• Rahimifard, Shahin, Yingying Seow, and Thomas Childs. 2010. “Minimising Embodied Product Energy to Support Energy Efficient Manufacturing.” CIRP Annals 59 (1): 25–28. https://doi.org/10.1016/j.cirp.2010.03.048.
   Web of Science ®Google Scholar
• Sabri, Abderrazzak, Hamid Allaoui, and Omar Souissi. 2023. “Reinforcement Learning and Stochastic Dynamic Programming for Jointly Scheduling Jobs and Preventive Maintenance on a Single Machine to Minimise Earliness-Tardiness.” International Journal of Production Research 1–15. https://doi.org/10.1080/00207543.2023.2172472
   Web of Science ®Google Scholar
• Shahrabi, J., M. A. Adibi, and M. Mahootchi. 2017. “A Reinforcement Learning Approach to Parameter Estimation in Dynamic job Shop Scheduling.” Computers & Industrial Engineering 110: 75–82. https://doi.org/10.1016/j.cie.2017.05.026.
   Web of Science ®Google Scholar
• Sun, Liangliang, Hang Jin, and Ye Li. 2018. “Research on Scheduling of Iron and Steel Scrap Steelmaking and Continuous Casting Process Aiming at Power Saving and Carbon Emissions Reducing.” IEEE Robotics and Automation Letters 3 (4): 3105–3112. https://doi.org/10.1109/LRA.2018.2849500.
   Web of Science ®Google Scholar
• Tan, Yuanyuan, and Shixin Liu. 2014. “Models and Optimisation Approaches for Scheduling Steelmaking-Refining-Continuous Casting Production Under Variable Electricity Price.” International Journal of Production Research 52 (4): 1032–1049. https://doi.org/10.1080/00207543.2013.828179.
   Web of Science ®Google Scholar
• Tavakoli, Arash, Fabio Pardo, and Petar Kormushev. 2018. “Action Branching Architectures for Deep Reinforcement Learning.” Paper presented at the 32nd AAAI Conference on Artificial Intelligence / 30th Innovative Applications of Artificial Intelligence Conference / 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, LA, 2018 Feb 02-07.
   Google Scholar
• Wang, Ling. 2020a. “Multi-objective Optimization Based on Decomposition for Flexible job Shop Scheduling Under Time-of-use Electricity Prices.” Knowledge-Based Systems 204: 0950–7051.
   Web of Science ®Google Scholar
• Wang, Yu-Fang. 2020b. “Adaptive job Shop Scheduling Strategy Based on Weighted Q-Learning Algorithm.” Journal of Intelligent Manufacturing 31 (2): 417–432. https://doi.org/10.1007/s10845-018-1454-3.
   Web of Science ®Google Scholar
• Wang, Guirong, and Qiqiang Li. 2017. “Solving the Steelmaking-continuous Casting Production Scheduling Problem with Uncertain Processing Time under the TOU Electricity Price.” Paper presented at the Chinese Automation Congress (CAC), Jinan, PEOPLES R CHINA, 2017 Oct 20-22.
   Google Scholar
• Wang, Shijin, Xiaodong Wang, Feng Chu, and Jianbo Yu. 2020. “An Energy-Efficient two-Stage Hybrid Flow Shop Scheduling Problem in a Glass Production.” International Journal of Production Research 58 (8): 2283–2314. https://doi.org/10.1080/00207543.2019.1624857.
   Web of Science ®Google Scholar
• Wang, Shijin, Zhanguo Zhu, Kan Fang, Feng Chu, and Chengbin Chu. 2018. “Scheduling on a two-Machine Permutation Flow Shop Under Time-of-use Electricity Tariffs.” International Journal of Production Research 56 (9): 3173–3187. https://doi.org/10.1080/00207543.2017.1401236.
   Web of Science ®Google Scholar
• Xu, J., Y. Zhao, H. Chen, and W. Deng. 2023. “ABC-GSPBFT: PBFT with Grouping Score Mechanism and Optimized Consensus Process for Flight Operation Data-Sharing.” Information Sciences 624: 110–127. https://doi.org/10.1016/j.ins.2022.12.068.
   Web of Science ®Google Scholar
• Yang, Shengluo, and Zhigang Xu. 2022. “Intelligent Scheduling and Reconfiguration via Deep Reinforcement Learning in Smart Manufacturing.” International Journal of Production Research 60 (16): 4936–4953. https://doi.org/10.1080/00207543.2021.1943037.
   Web of Science ®Google Scholar
• Zeng, Y. Z., A. Che, and X. Q. Wu. 2018. “Bi-objective Scheduling on Uniform Parallel Machines Considering Electricity Cost.” Engineering Optimization 50 (1): 19–36. https://doi.org/10.1080/0305215X.2017.1296437.
   Web of Science ®Google Scholar
• Zeng, Zhiqiang, Mengna Hong, Yi Man, Jigeng Li, Yanzhong Zhang, and Huanbin Liu. 2018. “Multi-object Optimization of Flexible Flow Shop Scheduling with Batch Process - Consideration Total Electricity Consumption and Material Wastage.” Journal of Cleaner Production 183: 925–939. https://doi.org/10.1016/j.jclepro.2018.02.224.
   Web of Science ®Google Scholar
• Zhang, Wei, and Thomas G Dietterich. 1995. “A Reinforcement Learning Approach to job-Shop Scheduling.” Paper Presented at the IJCAI.
   Google Scholar
• Zheng, Xu, Shengchao Zhou, Rui Xu, and Huaping Chen. 2020. “Energy-efficient Scheduling for Multi-Objective two-Stage Flow Shop Using a Hybrid ant Colony Optimisation Algorithm.” International Journal of Production Research 58 (13): 4103–4120. https://doi.org/10.1080/00207543.2019.1642529.
   Web of Science ®Google Scholar