Shovel allocation and scheduling for open-pit mining using deep reinforcement learning

References

• R. Noriega and Y. Pourrahimian, A systematic review of artificial intelligence and data-driven approaches in strategic open-pit mine planning, Resour. Policy. 77 (2022), pp. 102727. doi:10.1016/j.resourpol.2022.102727.
   Web of Science ®Google Scholar
• P. Chimunhu, E. Topal, A. Duany, and W. Asad, A review of machine learning applications for underground mine planning and scheduling, Resour. Policy 77 (2022), pp. 102693. doi:10.1016/j.resourpol.2022.102693.
   Web of Science ®Google Scholar
• R. Rai, M.K. Tiwari, D. Ivanov, and A. Dolgui, Machine learning in manufacturing and industry 4.0 applications, Int. J. Prod. Res 59 (16) (2021), pp. 4773–4778. doi:10.1080/00207543.2021.1956675.
   Web of Science ®Google Scholar
• M. Panzer, B. Bender, and N. Gronau, Deep reinforcement learning in production planning and control: A systematic literature review, in Proc. 2nd Conf. Prod. Syst. Logist. CPSL 2021, D. Herberger and M. Hubner, eds., Leibniz Universitat Hannover, Hannover, 2021, pp. 535–545.
   Google Scholar
• B. Rolf, I. Jackson, M. Müller, S. Lang, T. Reggelin, and D. Ivanov, A review on reinforcement learning algorithms and applications in supply chain management, Int. J. Prod. Res. 61 (20) (2022), pp. 7151–7179. doi:10.1080/00207543.2022.2140221.
   Web of Science ®Google Scholar
• C. Li, L. Bai, L. Yao, S.T. Waller, and W. Liu, A bibliometric analysis and review on reinforcement learning for transportation applications, Transp B Transp Dyn 11 (1) (2023), pp. 1095–1135. doi:10.1080/21680566.2023.2179461.
   Web of Science ®Google Scholar
• R. Nian, J. Liu, and B. Huang, A review on reinforcement learning: Introduction and applications in industrial process control, Comput. Chem. Eng. 139 (2020), pp. 106886. doi:10.1016/j.compchemeng.2020.106886.
   Web of Science ®Google Scholar
• M. Blom, A.R. Pearce, and P.J. Stuckey, Short-term planning for open pit mines: A review, Int. J. Mining, Reclam. Environ 33 (5) (2019), pp. 318–339. doi:10.1080/17480930.2018.1448248.
   Web of Science ®Google Scholar
• H. Eivazy and H. Askari-Nasab, A mixed integer linear programming model for short-term open pit mine production scheduling, Min. Technol. 121 (2) (2012), pp. 97–108. doi:10.1179/1743286312Y.0000000006.
   Google Scholar
• G. L’Heureux, M. Gamache, and F. Soumis, Mixed integer programming model for short term planning in open-pit mines, Min. Technol. 122 (2) (2013), pp. 101–109. doi:10.1179/1743286313Y.0000000037.
   Google Scholar
• E. Kozan and S.Q. Liu, A new open-pit multi-stage mine production timetabling model for drilling, blasting and excavating operations, Min. Technol. 125 (1) (2015), pp. 47–53. doi:10.1179/1743286315Y.0000000031.
   Google Scholar
• M. Blom, P.J.S. Short-Term, A.R. Pearce, A.R. Pearce, P.J. Stuckey, M. Blom, Short-term scheduling of an open-pit mine with multiple objectives multiple objectives, Eng. Optim. 49 (5) (2016), pp. 777–795. doi:10.1080/0305215X.2016.1218002.
   Web of Science ®Google Scholar
• M. Blom, A.R. Pearce, and P.J. Stuckey, A Decomposition-Based Algorithm for the scheduling of open-pit networks over multiple time periods, Manage. Sci. 62 (10) (2016), pp. 3059–3084. doi:10.1287/mnsc.2015.2284.
   Web of Science ®Google Scholar
• C. Both and R. Dimitrakopoulos, Joint stochastic short-term production scheduling and fleet management optimization for mining complexes, Optim. Eng. 21 (4) (2020), pp. 1717–1743. doi:10.1007/s11081-020-09495-x.
   Web of Science ®Google Scholar
• S.P. Upadhyay and H. Askari-Nasab, Dynamic shovel allocation approach to short-term production planning in open-pit mines, Int. J. Mining, Reclam. Environ 33 (1) (2017), pp. 1–20. doi:10.1080/17480930.2017.1315524.
   Web of Science ®Google Scholar
• S.P. Upadhyay and H. Askari-Nasab, Simulation and optimization approach for uncertainty-based short-term planning in open pit mines, Int. J. Min. Sci. Technol. 28 (2018), pp. 153–166. doi:10.1016/j.ijmst.2017.12.003.
   Web of Science ®Google Scholar
• M.S. Shishvan and J. Benndorf, Simulation-based optimization approach for material dispatching in continuous mining systems, Eur. J. Oper. Res. 275 (3) (2019), pp. 1108–1125. doi:10.1016/j.ejor.2018.12.015.
   Web of Science ®Google Scholar
• R. Sutton and A. Barto, Reinforcement Learning an Introduction, 2nd ed., MIT Press, Cambridge, Massachusetts, 2018.
   Google Scholar
• M. Naeem, S. Tahir, H. Rizvi, and A. Coronato, A gentle introduction to reinforcement learning and its application in different fields, IEEE. Access 8 (2020), pp. 209320–209344. doi:10.1109/ACCESS.2020.3038605.
   Web of Science ®Google Scholar
• L. Alzubaidi, J. Zhang, A.J. Humaidi, A.A. Dujaili, Y. Duan, O.A. Shamma, Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions, J. Big. Data. 8 (1) (2021), pp. 8. doi:10.1186/s40537-021-00444-8.
   Google Scholar
• V. Mnih, K. Kavukcuoglu, D. Silver, A.A. Rusu, J. Veness, M.G. Bellemare, Human-level control through deep reinforcement learning, Nature 518 (7540) (2015), pp. 529–533. doi:10.1038/nature14236.
   PubMed Web of Science ®Google Scholar
• J.N. Tsitsiklis and B. Van Roy, An analysis of temporal-difference learning with function approximation, IEEE Trans. Automat. Contr. 42 (5) (1997), pp. 674–690. doi:10.1109/9.580874.
   Web of Science ®Google Scholar
• H. Wang, N. Liu, Y. Zhang, D. Feng, F. Huang, D. Li, Y.-M. Zhang, et al., Deep reinforcement learning: A survey, Front. Inf. Technol. Electron. Eng. 21 (12) (2020), pp. 1726–1744. doi:10.1631/FITEE.1900533.
   Web of Science ®Google Scholar
• H. Askari-Nasab and K. Awuah-Offei, Open pit optimisation using discounted economic block values, Min. Technol. 118 (1) (2009), pp. 1–12. doi:10.1179/037178409x12450752943243.
   Google Scholar
• H. Askari-Nasab, S. Frimpong, and J. Szymanski, Modelling open pit dynamics using discrete simulation, Int. J. Mining, Reclam. Environ 21 (1) (2007), pp. 35–49. doi:10.1080/17480930600720206.
   Google Scholar
• C. Paduraru and R. Dimitrakopoulos, Adaptive policies for short-term material flow optimization in a mining complex, Min. Technol. 127 (1) (2017), pp. 56–63. doi:10.1080/14749009.2017.1341142.
   Google Scholar
• C. Paduraru and R. Dimitrakopoulos, Responding to new information in a mining complex: Fast mechanisms using machine learning, Min. Technol. 128 (3) (2019), pp. 129–142. doi:10.1080/25726668.2019.1577596.
   Google Scholar
• A. Kumar, R. Dimitrakopoulos, and M. Maulen, Adaptive self-learning mechanisms for updating short-term production decisions in an industrial mining complex, J Intell Manuf 31 (7) (2020), pp. 1795–1811. doi:10.1007/s10845-020-01562-5.
   Web of Science ®Google Scholar
• A. Kumar and R. Dimitrakopoulos, Production scheduling in industrial mining complexes with incoming new information using tree search and deep reinforcement learning, Appl. Soft Comput. 110 (2021), pp. 15. doi:10.1016/j.asoc.2021.107644.
   Web of Science ®Google Scholar
• S. Avalos and J.M. Ortiz, Multivariate geostatistical simulation and deep Q-Learning to optimize mining decisions, Math Geosci 55 (5) (2023), pp. 673–692. doi:10.1007/s11004-023-10049-x.
   Web of Science ®Google Scholar
• Z. Levinson, R. Dimitrakopoulos, and J. Keutchayan, Simultaneous stochastic optimization of an open-pit mining complex with preconcentration using reinforcement learning, Appl. Soft Comput. 138 (2023), pp. 110180. doi:10.1016/j.asoc.2023.110180.
   Web of Science ®Google Scholar
• E. Goris Cervantes, S. Upadhyay, and H. Askari-Nasab, Improvements to production planning in oil sands mining through analysis and simulation of truck cycle times, CIM J. 10 (1) (2019), pp. 39–52. doi:10.15834/cimj.2019.1.
   Google Scholar
• S. Upadhyay, M. Tabesh, M. Badiozamani, A. Moradi Afrapoli, and H. Askari-Nasab, A simulation-based algorithm for solving surface mines’ equipment selection and sizing problem under uncertainty, CIM J. 12 (1) (2021), pp. 36–46. doi:10.1080/19236026.2021.1872995.
   Google Scholar
• M. Hessel, J. Modayil, H. van Hasselt, T. Schaul, G. Ostrovski, and W. Dabney, Rainbow: Combining improvements in deep reinforcement learning. Proc. Thirty-Second AAAI Conf. Artif. Intell., New Orleans, Louisiana, USA, 2018.
   Google Scholar
• R.S. Sutton, Learning to predict by the methods of temporal differences, Mach Learn 3 (1) (1988), pp. 9–44. doi:10.1007/BF00115009.
   Google Scholar
• H. van Hasselt, A. Guez, and D. Silver Deep reinforcement learning with double Q-Learning. Proc. AAAI Conf. Artif. Intell, Phoenix, AZ: 2016, p. 2094–20100. 10.1609/aaai.v30i1.10295.
   Google Scholar
• M. Fortunato, A. Gheshlaghi, B. Piot, J. Menick, M. Hessel, and I. Osband, Noisy networks for exploration. Proc. 6th Int. Conf. Learn. Represent. ICLR 2018, Vancouver, Canada: 2018.
   Google Scholar
• Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lactot, and N. de Freites, Dueling network architectures for deep reinforcement learning, in Proc. 33rd Int. Conf. Mach. Learn Vol. 48, 2016, pp. 1995–2003.
   Google Scholar
• J.S. Obando-Ceron and P.S. Castro, Revisiting rainbow: Promoting more insightful and inclusive deep reinforcement learning research, Proc. 38th Int. Conf. Mach. Learn. PMLR Vol. 139, 2021, pp. 1373–1383.
   Google Scholar
• J. Zhang, T. He, S. Sra, and A. Jadbabaie, Why gradient clipping accelerates training: A theoretical justification for adaptivity, Eighth Int. Conf. Learn. Represent. ICLR 2020.
   Google Scholar