Reinforcement learning based optimal decision making towards product lifecycle sustainability

References

• Abadi, M., P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, et al. 2016. “TensorFlow: A System for Large-Scale Machine Learning.” In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, 265–283. OSDI’16. USA: USENIX Association.
   Google Scholar
• Andriotis, C. P., and K. G. Papakonstantinou. 2020. “Deep Reinforcement Learning Driven Inspection and Maintenance Planning under Incomplete Information and Constraints.” ArXiv, July. arXiv. http://arxiv.org/abs/2007.01380.
   Google Scholar
• Badurdeen, F., R. Aydin, and A. Brown. November 2018. “A Multiple Lifecycle-Based Approach to Sustainable Product Configuration Design.” Journal of Cleaner Production 200: 756–769. Elsevier Ltd. 10.1016/j.jclepro.2018.07.317.
   Web of Science ®Google Scholar
• Bishop, C. M. 2006 Pattern Recognition and Machine Learning. New York: Springer.
   Google Scholar
• Bosch-Mauchand, M., F. Belkadi, M. Bricogne, and B. Eynard. 2013. “Knowledge-Based Assessment of Manufacturing Process Performance: Integration of Product Lifecycle Management and Value-Chain Simulation Approaches.” International Journal of Computer Integrated Manufacturing 26 (5): 453–473. doi:10.1080/0951192X.2012.731611. Taylor and Francis Ltd.
   Web of Science ®Google Scholar
• Cai, B., L. Huang, and M. Xie. 2017. “Bayesian Networks in Fault Diagnosis.“ IEEE Transactions on Industrial Informatics 13 ( 5): 2227–2240. doi:10.1109/TII.2017.2695583.
   Web of Science ®Google Scholar
• Cai, H., L. D. Xu, B. Xu, C. Xie, S. Qin, and L. Jiang. 2014. “IoT-Based Configurable Information Service Platform for Product Lifecycle Management.” IEEE Transactions on Industrial Informatics 10 (2): 1558–1567. doi:10.1109/TII.2014.2306391.
   Web of Science ®Google Scholar
• Carpenter, B., A. Gelman, M. D. Hoffman, D. Lee, B. Goodrich, M. Betancourt, M. Brubaker, J. Guo, P. Li, and A. Riddell. 2017. “Stan: A Probabilistic Programming Language.” Journal of Statistical Software 76 (1): 1–32. doi:10.18637/jss.v076.i01.
   PubMed Web of Science ®Google Scholar
• Deisenroth, M. P., D. Fox, and C. E. Rasmussen. 2015. “Gaussian Processes for Data-Efficient Learning in Robotics and Control.“ IEEE Transactions on Pattern Analysis and Machine Intelligence 37 ( 2): 408–423. doi:10.1109/TPAMI.2013.218.
   PubMed Web of Science ®Google Scholar
• Ding, R. X., I. Palomares, X. Wang, G. R. Yang, B. Liu, Y. Dong, E. Herrera-Viedma, and F. Herrera. July 2020. “Large-Scale Decision Making: Characterisation, Taxonomy, Challenges and Future Directions from an Artificial Intelligence and Applications Perspective.” Information Fusion 59: 84–102. Elsevier BV. 10.1016/j.inffus.2020.01.006.
   Web of Science ®Google Scholar
• Duan, C., C. Deng, A. Gharaei, J. Wu, and B. Wang. 2018. “Selective Maintenance Scheduling under Stochastic Maintenance Quality with Multiple Maintenance Actions.” International Journal of Production Research 56 (23): 7160–7178. doi:10.1080/00207543.2018.1436789. Taylor and Francis Ltd.
   Web of Science ®Google Scholar
• Duan, Y., J. S. Edwards, and Y. K. Dwivedi. 2019. “Artificial Intelligence for Decision Making in the Era of Big Data – Evolution, Challenges and Research Agenda.” International Journal of Information Management 48 (October): 63–71. doi:10.1016/j.ijinfomgt.2019.01.021. Elsevier Ltd.
   Web of Science ®Google Scholar
• Ferreira, F., J. Faria, A. Azevedo, and L. A. Marques. 2017. “Product Lifecycle Management in Knowledge Intensive Collaborative Environments: An Application to Automotive Industry.” International Journal of Information Management 37 (1): 1474–1487. doi:10.1016/j.ijinfomgt.2016.05.006. Elsevier Ltd: 1474–1487.
   Web of Science ®Google Scholar
• Froger, A., M. Gendreau, J. E. Mendoza, É. Pinson, and M. L. Rousseau. 2016. “Maintenance Scheduling in the Electricity Industry: A Literature Review.” European Journal of Operational Research 251 (3): 695–706. doi:10.1016/j.ejor.2015.08.045. Elsevier B.V.
   Web of Science ®Google Scholar
• Gao, J. X., H. Aziz, P. G. Maropoulos, and W. M. Cheung. 2003. “Application of Product Data Management Technologies for Enterprise Integration.” International Journal of Computer Integrated Manufacturing 16 (7–8): 7–8. doi:10.1080/0951192031000115813. Taylor & Francis Group: 491–500.
   Web of Science ®Google Scholar
• Geissinger, A., C. Laurell, C. Öberg, and C. Sandström. January 2019. “How Sustainable Is the Sharing Economy? on the Sustainability Connotations of Sharing Economy Platforms.” Journal of Cleaner Production 206: 419–429. Elsevier Ltd. 10.1016/j.jclepro.2018.09.196.
   Web of Science ®Google Scholar
• Gelman, A., J. B. Carlin, H. S. Stern, D. B. Dunson, A. Vehtari, and D. B. Rubin. 2013. Bayesian Data Analysis, 3rd Edition. 675. New York: Chapman and Hall/CRC.
   Google Scholar
• Goodfellow, I., Y. Bengio, and A. Courville. 2016. Deep Learning. Cambridge, Massachusetts: MIT Press. http://www.deeplearningbook.org
   Google Scholar
• Govindan, K., A. Jafarian, and V. Nourbakhsh. October 2019. “Designing A Sustainable Supply Chain Network Integrated with Vehicle Routing: A Comparison of Hybrid Swarm Intelligence Metaheuristics.” Computers & Operations Research 110: 220–235. Pergamon. 10.1016/J.COR.2018.11.013.
   Web of Science ®Google Scholar
• Guo, Z., Y. Zhang, J. Lv, Y. Liu, and Y. Liu. 2021a. “An Online Learning Collaborative Method for Traffic Forecasting and Routing Optimization.“ IEEE Transactions on Intelligent Transportation Systems 22 ( 10): 6634–6645. doi: 10.1109/TITS.2020.2986158.
   Web of Science ®Google Scholar
• Guo, Z., Y. Zhang, X. Zhao, and X. Song. 2021b. “CPS-Based Self-Adaptive Collaborative Control for Smart Production-Logistics Systems.“ IEEE Transactions on Cybernetics 51 ( 1): 188–198. doi:10.1109/TCYB.2020.2964301.
   PubMed Web of Science ®Google Scholar
• Gustavsson, E., M. Patriksson, A. B. Strömberg, A. Wojciechowski, and M. Önnheim. 2014. “Preventive Maintenance Scheduling of Multi-Component Systems with Interval Costs.” Computers and Industrial Engineering 76 (October): 390–400. doi:10.1016/j.cie.2014.02.009. Elsevier Ltd.
   Google Scholar
• Hatim, Q. Y., C. Saldana, G. Shao, D. B. Kim, K. C. Morris, P. Witherell, S. Rachuri, and S. Kumara. 2020. “A Decision Support Methodology for Integrated Machining Process and Operation Plans for Sustainability and Productivity Assessment.” The International Journal of Advanced Manufacturing Technology 107 (7): 3207–3230. 10.1007/S00170-019-04268-Y.
   Web of Science ®Google Scholar
• Hwang, H. T., S. H. Lee, H. G. Chi, N. K. Kang, H. B. Kong, J. Lu, and H. Ohk. 2019. “An Evaluation Methodology for 3D Deep Neural Networks Using Visualization in 3D Data Classification.” Journal of Mechanical Science and Technology 33 (3): 3. doi:10.1007/s12206-019-0233-1. Korean Society of Mechanical Engineers: 1333–1339.
   Web of Science ®Google Scholar
• Jardine, A. K. S., D. Lin, and D. Banjevic. 2006. “A Review on Machinery Diagnostics and Prognostics Implementing Condition-Based Maintenance.” Mechanical Systems and Signal Processing 20 (7): 1483–1510. doi:10.1016/j.ymssp.2005.09.012.
   Web of Science ®Google Scholar
• Jarrahi, M. H. 2018. “Artificial Intelligence and the Future of Work: Human-AI Symbiosis in Organizational Decision Making.” Business Horizons 61 (4): 577–586. doi:10.1016/J.BUSHOR.2018.03.007. Elsevier.
   Web of Science ®Google Scholar
• Kaewunruen, S., and Q. Lian. August 2019. “Digital Twin Aided Sustainability-Based Lifecycle Management for Railway Turnout Systems.” Journal of Cleaner Production 228: 1537–1551. Elsevier Ltd. 10.1016/j.jclepro.2019.04.156.
   Web of Science ®Google Scholar
• Kermany, D. S., M. Goldbaum, W. Cai, C. C. S. Valentim, H. Liang, S. L. Baxter, A. McKeown, et al. 2018. “Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning.” Cell 172 (5): 1122–1131.e9. doi:10.1016/j.cell.2018.02.010. Cell Press.
   PubMed Web of Science ®Google Scholar
• Kiritsis, D., A. Bufardi, and P. Xirouchakis. 2003. “Research Issues on Product Lifecycle Management and Information Tracking Using Smart Embedded Systems.” Advanced Engineering Informatics 17 (3–4): 3–4. doi:10.1016/j.aei.2004.09.005. Elsevier BV: 189–202.
   Web of Science ®Google Scholar
• Knowles, M., D. Baglee, and S. Wermter. 2011. In Reinforcement Learning for Scheduling of Maintenance, edited by Bramer, M., Petridis, M., and Hopgood, A. Research and Development in Intelligent Systems, Vol. XXVII, 409–422. London: Springer. doi:10.1007/978-0-85729-130-1_31.
   Google Scholar
• Längkvist, M., L. Karlsson, and A. Loutfi. 2014. “A Review of Unsupervised Feature Learning and Deep Learning for Time-Series Modeling.” Pattern Recognition Letters 42 (1): 11–24. doi:10.1016/j.patrec.2014.01.008. North-Holland.
   Google Scholar
• Lee, S. G., Y. S. Ma, G. L. Thimm, and J. Verstraeten. 2008. “Product Lifecycle Management in Aviation Maintenance, Repair and Overhaul.” Computers in Industry 59 (2–3): 2–3. doi:10.1016/j.compind.2007.06.022. Elsevier: 296–303.
   Web of Science ®Google Scholar
• Li, B. H., B. C. Hou, W. T. Yu, X. B. Lu, and C. W. Yang. 2017. “Applications of Artificial Intelligence in Intelligent Manufacturing: A Review.” Frontiers of Information Technology and Electronic Engineering. Zhejiang University. doi:10.1631/FITEE.1601885.
   Google Scholar
• Li, J., F. Tao, Y. Cheng, and L. Zhao. 2015. “Big Data in Product Lifecycle Management.” The International Journal of Advanced Manufacturing Technology 81: 667–684. doi:10.1007/s00170-015-7151-x. .
   Web of Science ®Google Scholar
• Li, Z., S. Zhong, and L. Lin. 2019. “An Aero-Engine Life-Cycle Maintenance Policy Optimization Algorithm: Reinforcement Learning Approach.” Chinese Journal of Aeronautics 32 (9): 2133–2150. doi:10.1016/j.cja.2019.07.003. Chinese Journal of Aeronautics.
   Web of Science ®Google Scholar
• Lim, K. Y. H., P. Zheng, and C. H. Chen. 2020. “A State-of-the-Art Survey of Digital Twin: Techniques, Engineering Product Lifecycle Management and Business Innovation Perspectives.” Journal of Intelligent Manufacturing 31 (6): 1313–1337. doi:10.1007/s10845-019-01512-w.
   Web of Science ®Google Scholar
• Litjens, G., T. Kooi, B. Ehteshami Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sánchez. 2017. “A Survey on Deep Learning in Medical Image Analysis.” Medical Image Analysis 42: 60–88. doi:10.1016/j.media.2017.07.005.
   PubMed Web of Science ®Google Scholar
• Liu, X. L., W. M. Wang, H. Guo, A. V. Barenji, Z. Li, and G. Q. Huang. June 2020a. “Industrial Blockchain Based Framework for Product Lifecycle Management in Industry 4.0.” Robotics and Computer-Integrated Manufacturing 63: 101897. Elsevier Ltd: 101897. 10.1016/j.rcim.2019.101897.
   Web of Science ®Google Scholar
• Liu, Y., Y. Zhang, S. Ren, M. Yang, Y. Wang, and D. Huisingh. March 2020b. “How Can Smart Technologies Contribute to Sustainable Product Lifecycle Management?” Journal of Cleaner Production 249: 119423. Elsevier Ltd. 10.1016/j.jclepro.2019.119423.
   Web of Science ®Google Scholar
• Lou, S., Y. Feng, H. Zheng, Y. Gao, and J. Tan. 2018. “Data-Driven Customer Requirements Discernment in the Product Lifecycle Management via Intuitionistic Fuzzy Sets and Electroencephalogram.” Journal of Intelligent Manufacturing 31 (7): 1–16. doi:10.1007/s10845-018-1395-x. Springer New York LLC.
   Web of Science ®Google Scholar
• Ma, Y., K. Rong, D. Mangalagiu, T. F. Thornton, and D. Zhu. July 2018. “Co-Evolution between Urban Sustainability and Business Ecosystem Innovation: Evidence from the Sharing Mobility Sector in Shanghai.” Journal of Cleaner Production 188: 942–953. Elsevier Ltd. 10.1016/j.jclepro.2018.03.323.
   Web of Science ®Google Scholar
• MacCarthy, B. L., and R. C. Pasley. 2021. “Group Decision Support for Product Lifecycle Management.” International Journal of Production Research 59 (16): 5050–5067. June. Taylor and Francis Ltd. doi:10.1080/00207543.2020.1779372.
   Web of Science ®Google Scholar
• Matsokis, A., and D. Kiritsis. 2010. “An Ontology-Based Approach for Product Lifecycle Management.” Computers in Industry 61 (8): 787–797. doi:10.1016/J.COMPIND.2010.05.007. Elsevier.
   Web of Science ®Google Scholar
• Meng, K., Y. Cao, X. Peng, V. Prybutok, and K. Youcef-Toumi. November 2020. “Smart Recovery Decision Making for End-of-Life Products in the Context of Ubiquitous Information and Computational Intelligence.” Journal of Cleaner Production 272: 122804. Elsevier. 10.1016/J.JCLEPRO.2020.122804.
   Web of Science ®Google Scholar
• Nezhad, M. Z., N. Sadati, K. Yang, and D. Zhu. January 2019. “A Deep Active Survival Analysis Approach for Precision Treatment Recommendations: Application of Prostate Cancer.” Expert Systems with Applications 115: 16–26. Elsevier Ltd. 10.1016/j.eswa.2018.07.070.
   Web of Science ®Google Scholar
• Nishant, R., M. Kennedy, and J. Corbett. August 2020. “Artificial Intelligence for Sustainability: Challenges, Opportunities, and a Research Agenda.” International Journal of Information Management 53: 102104. Pergamon: 102104. 10.1016/J.IJINFOMGT.2020.102104.
   Web of Science ®Google Scholar
• O’Leary, D. E. 2013. “Artificial Intelligence and Big Data.” IEEE Intelligent Systems 28 (2): 96–99. doi:10.1109/MIS.2013.39.
   PubMed Web of Science ®Google Scholar
• Pomerol, J. C. 1997. “Artificial Intelligence and Human Decision Making.” European Journal of Operational Research 99 (1): 3–25. doi:10.1016/S0377-2217(96)00378-5. Elsevier.
   Web of Science ®Google Scholar
• Poole, D. L., and A. K. Mackworth. 2017. Artificial Intelligence: Foundations of Computational Agents. 2nd ed. Cambridge: Cambridge University Press.
   Google Scholar
• Powell, W. B., and H. Topaloglu. 2006. Approximate Dynamic Programming for Large-Scale Resource Allocation Problems Models, Methods, and Applications for Innovative Decision Making (Informs)doi:10.1287/educ.1063.0027
   Google Scholar
• Ren, S., Y. Zhang, Y. Liu, T. Sakao, D. Huisingh, and C. M. V. B. Almeida. 2019. “A Comprehensive Review of Big Data Analytics Throughout Product Lifecycle to Support Sustainable Smart Manufacturing: A Framework, Challenges and Future Research Directions.” Journal of Cleaner Production 210 ( February). Elsevier Ltd: 1343–1365. doi:10.1016/j.jclepro.2018.11.025.
   Web of Science ®Google Scholar
• Rifkin, J. 2000. The Age of Access: The New Culture of Hypercapitalism, Where All of Life Is a Paid-For Experience. New York: JP Tarcher/Putnam.
   Google Scholar
• Rondini, A., F. Tornese, M. G. Gnoni, G. Pezzotta, and R. Pinto. 2017. “Hybrid Simulation Modelling as A Supporting Tool for Sustainable Product Service Systems: A Critical Analysis.” International Journal of Production Research 55 (23): 6932–6945. doi:10.1080/00207543.2017.1330569. Taylor and Francis Ltd.
   Web of Science ®Google Scholar
• Sina Tayarani-Bathaie, S., Z. N. Sadough Vanini, and K. Khorasani. February 2014. “Dynamic Neural Network-Based Fault Diagnosis of Gas Turbine Engines.” Neurocomputing 125: 153–165. Elsevier. 10.1016/j.neucom.2012.06.050.
   Web of Science ®Google Scholar
• Spacagna, G. 2018. “Deep Time-to-Failure: Survival Analysis and Time-to-Failure Predictive Modeling Using Weibull Distributions and Recurrent Neural Networks in Keras.”
   Google Scholar
• Sutton, R. S., and A. G. Barto. 2018. Reinforcement Learning, Second Edition: An Introduction. Cambridge, Massachusetts: MIT press.
   Google Scholar
• Talhi, A., V. Fortineau, J. C. Huet, and S. Lamouri. 2019. “Ontology for Cloud Manufacturing Based Product Lifecycle Management.” Journal of Intelligent Manufacturing 30 (5): 2171–2192. doi:10.1007/s10845-017-1376-5. Springer New York LLC.
   Web of Science ®Google Scholar
• Tao, F., J. Cheng, Q. Qi, M. Zhang, H. Zhang, and F. Sui. 2018. “Digital Twin-Driven Product Design, Manufacturing and Service with Big Data.” The International Journal of Advanced Manufacturing Technology 94: 3563–3576. doi:10.1007/s00170-017-0233-1.
   Web of Science ®Google Scholar
• Wang, L., Z. Zhang, H. Long, X. Jia, and R. Liu. 2017. “Wind Turbine Gearbox Failure Identification with Deep Neural Networks.“ IEEE Transactions on Industrial Informatics 13 (3): 1360–1368. doi:10.1109/TII.2016.2607179.
   Web of Science ®Google Scholar
• Wang, N., S. Ren, Y. Liu, M. Yang, J. Wang, and D. Huisingh. August 2020. “An Active Preventive Maintenance Approach of Complex Equipment Based on a Novel Product-Service System Operation Mode.” Journal of Cleaner Production 277: 123365. Elsevier BV. 10.1016/j.jclepro.2020.123365.
   Web of Science ®Google Scholar
• Wingfield, N. 2017. “Automakers Race to Get Ahead of Silicon Valley on Car-Sharing.” New York Times.
   Google Scholar
• Woyke, E. 2017. “General Electric Builds an AI Workforce.” MIT Technology Review 1: 1–3.
   Google Scholar
• Yao, L., Q. Dong, J. Jiang, and F. Ni. 2020. “Deep Reinforcement Learning for Long‐term Pavement Maintenance Planning.” Computer-Aided Civil and Infrastructure Engineering 35 (11): 1230–1245. doi:10.1111/mice.12558. Blackwell Publishing Inc.
   Web of Science ®Google Scholar
• Yousefi, N., S. Tsianikas, and D. W. Coit. 2020. “Reinforcement Learning for Dynamic Condition-Based Maintenance of a System with Individually Repairable Components.” Quality Engineering 32 (3): 388–408. doi:10.1080/08982112.2020.1766692. Taylor and Francis Inc.
   Web of Science ®Google Scholar
• Zhang, Y., S. Ren, Y. Liu, and S. Si. January 2017b. “A Big Data Analytics Architecture for Cleaner Manufacturing and Maintenance Processes of Complex Products.” Journal of Cleaner Production 142: 626–641. Elsevier Ltd. 10.1016/j.jclepro.2016.07.123.
   Web of Science ®Google Scholar
• Zhang, Y., S. Ren, Y. Liu, T. Sakao, and D. Huisingh. August 2017a. “A Framework for Big Data Driven Product Lifecycle Management.” Journal of Cleaner Production 159: 229–240. Elsevier Ltd. 10.1016/j.jclepro.2017.04.172.
   Web of Science ®Google Scholar
• Zhang, Y., Z. Guo, L. Jingxiang, and Y. Liu. 2018. “A Framework for Smart Production-Logistics Systems Based on CPS and Industrial IoT.“ IEEE Transactions on Industrial Informatics 14 (9): 4019–4032. doi:10.1109/TII.2018.2845683.
   Web of Science ®Google Scholar
• Zhao, Z. Q., P. Zheng, X. Shou Tao, and W. Xindong 2019. “Object Detection with Deep Learning: A Review.“ IEEE Transactions on Neural Networks and Learning Systems 30 (11): 3212–3232. doi:10.1109/TNNLS.2018.2876865.
   PubMed Web of Science ®Google Scholar