Smart manufacturing under limited and heterogeneous data: a sim-to-real transfer learning with convolutional variational autoencoder in thermoforming

References

• Albanese, D., S. Riccadonna, C. Donati, and P. Franceschi. 2018. “A Practical Tool for Maximal Information Coefficient Analysis.” Gigascience 7 (4): giy032. https://doi.org/10.1093/gigascience/giy032.
   Web of Science ®Google Scholar
• Bayoudh, K., R. Knani, F. Hamdaoui, and A. Mtibaa. 2021. “A Survey on Deep Multimodal Learning for Computer Vision: Advances, Trends, Applications, and Datasets.” The Visual Computer 38 (8): 2939–2970. https://doi.org/10.1007/s00371-021-02166-7.
   PubMed Web of Science ®Google Scholar
• Cheng, H.-T., L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, and M. Ispir. 2016. “Wide & Deep Learning for Recommender Systems.” In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, Boston, USA, 7–10.
   Google Scholar
• De, S., J. Britton, M. Reynolds, R. Skinner, K. Jansen, and A. Doostan. 2020. “On Transfer Learning of Neural Networks Using Bi-Fidelity Data for Uncertainty Propagation.” International Journal for Uncertainty Quantification 10 (6): 543–573. https://doi.org/10.1615/Int.J.UncertaintyQuantification.2020033267.
   Web of Science ®Google Scholar
• Dwivedi, K., and G. Roig. 2019. “Representation Similarity Analysis for Efficient Task Taxonomy & Transfer Learning.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 12387–12396.
   Google Scholar
• Feng, L., Y.-S. Ong, S. Jiang, and A. Gupta. 2017. “Autoencoding Evolutionary Search with Learning Across Heterogeneous Problems.” IEEE Transactions on Evolutionary Computation 21 (5): 760–772. https://doi.org/10.1109/TEVC.2017.2682274.
   Web of Science ®Google Scholar
• Friedman, J. H. 2001. “Greedy Function Approximation: A Gradient Boosting Machine.” The Annals of Statistics 29 (5): 1189–1232. https://doi.org/10.1214/aos/1013203451.
   Web of Science ®Google Scholar
• Géron, A. 2019. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Sebastopol, CA, USA: O’Reilly Media.
   Google Scholar
• Guo, Q., J. Hang, S. Wang, W. Hui, and Z. Xie. 2021. “Design Optimization of Variable Stiffness Composites by Using Multi-Fidelity Surrogate Models.” Structural and Multidisciplinary Optimization 63 (1): 439–461. https://doi.org/10.1007/s00158-020-02684-3.
   Web of Science ®Google Scholar
• Hanneke, S., and S. Kpotufe. 2019. “On the Value of Target Data in Transfer Learning.” Advances in Neural Information Processing Systems 32.
   Google Scholar
• Ha, D., and J. Schmidhuber. 2018. “World Models.” ArXiv Preprint ArXiv:1803.10122.
   Google Scholar
• Ho, T. K. 1998. “The Random Subspace Method for Constructing Decision Forests.” IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (8): 832–844. https://doi.org/10.1109/34.709601.
   Web of Science ®Google Scholar
• Huang, X. 2020. “Intelligent Remote Monitoring and Manufacturing System of Production Line Based on Industrial Internet of Things.” Computer Communications 150:421–428. https://doi.org/10.1016/j.comcom.2019.12.011.
   Web of Science ®Google Scholar
• Jolliffe, I. T., and J. Cadima. 2016. “Principal Component Analysis: A Review and Recent Developments.” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 374 (2065): 20150202. https://doi.org/10.1098/rsta.2015.0202.
   PubMed Web of Science ®Google Scholar
• Jung, H., J. Jeon, D. Choi, and J.-Y. Park. 2021. “Application of Machine Learning Techniques in Injection Molding Quality Prediction: Implications on Sustainable Manufacturing Industry.” Sustainability 13 (8): 4120. https://doi.org/10.3390/su13084120.
   Web of Science ®Google Scholar
• Khan, W. Z., M. H. Rehman, H. Mohammed Zangoti, M. Khalil Afzal, N. Armi, and K. Salah. 2020. “Industrial Internet of Things: Recent Advances, Enabling Technologies and Open Challenges.” Computers & Electrical Engineering 81:106522. https://doi.org/10.1016/j.compeleceng.2019.106522.
   Web of Science ®Google Scholar
• Kustowski, B., J. A. Gaffney, B. K. Spears, G. J. Anderson, R. Anirudh, P.-T. Bremer, J. J. Thiagarajan, K. G. K. Michael, and R. C. Nora. 2022. “Suppressing Simulation Bias in Multi-Modal Data Using Transfer Learning.” Machine Learning: Science and Technology 3 (1): 15035. https://doi.org/10.1088/2632-2153/ac5e3e.
   Google Scholar
• Leite, W. D. O., J. Carlos Campos Rubio, F. Mata Cabrera, A. Carrasco, and I. Hanafi. 2018. “Vacuum Thermoforming Process: An Approach to Modeling and Optimization Using Artificial Neural Networks.” Polymers 10 (2): 143. https://doi.org/10.3390/polym10020143.
   PubMed Web of Science ®Google Scholar
• Li, Z.-Z., T.-H. Cheng, Y.-D. Shen, and D.-J. Xuan. 2015. “Optimal Heater Control with Technology of Fault Tolerance for Compensating Thermoforming Preheating System.” Advances in Materials Science and Engineering 2015:1–5. https://doi.org/10.1155/2015/403730.
   Web of Science ®Google Scholar
• Li, Q., Q. Tang, I. Chan, H. Wei, P. Yudi, H. Jiang, L. Jun, and J. Zhou. 2018. “Smart Manufacturing Standardization: Architectures, Reference Models and Standards Framework.” Computers in Industry 101:91–106. https://doi.org/10.1016/j.compind.2018.06.005.
   Web of Science ®Google Scholar
• Liu, B., Y. Zhang, L. Jingxiang, A. Majeed, C.-H. Chen, and D. Zhang. 2021. “A Cost-Effective Manufacturing Process Recognition Approach Based on Deep Transfer Learning for CPS Enabled Shop-Floor.” Robotics and Computer-Integrated Manufacturing 70:102128. https://doi.org/10.1016/j.rcim.2021.102128.
   Web of Science ®Google Scholar
• Lundberg, S. M., and S.-I. Lee. 2017. “A Unified Approach to Interpreting Model Predictions.” In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4768–4777.
   Google Scholar
• Meng, X., and G. Em Karniadakis. 2020. “A Composite Neural Network That Learns from Multi-Fidelity Data: Application to Function Approximation and Inverse PDE Problems.” Journal of Computational Physics 401:109020. https://doi.org/10.1016/j.jcp.2019.109020.
   Web of Science ®Google Scholar
• Murphy, K. P. 2012. Machine Learning: A Probabilistic Perspective. Cambridge, MA, USA: MIT press.
   Google Scholar
• Neyshabur, B., H. Sedghi, and C. Zhang. 2020. “What is Being Transferred in Transfer Learning?” arXiv Preprint arXiv August. http://arxiv.org/abs/2008.11687.
   Google Scholar
• Pan, S. J., and Q. Yang. 2010. “A Survey on Transfer Learning.” IEEE Transactions on Knowledge and Data Engineering 22 (10): 1345–1359. https://doi.org/10.1109/TKDE.2009.191.
   Web of Science ®Google Scholar
• Peng, X. B., M. Andrychowicz, W. Zaremba, and P. Abbeel. 2018. “Sim-To-Real Transfer of Robotic Control with Dynamics Randomization.” In 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 1–8. IEEE.
   Google Scholar
• Ramezankhani, M., B. Crawford, H. Khayyam, M. Naebe, R. Seethaler, and A. S. Milani. 2019. “A Multi-Objective Gaussian Process Approach for Optimization and Prediction of Carbonization Process in Carbon Fiber Production Under Uncertainty.” Advanced Composites and Hybrid Materials 2 (3): 444–455. https://doi.org/10.1007/s42114-019-00107-6.
   Google Scholar
• Ramezankhani, M., B. Crawford, A. Narayan, H. Voggenreiter, R. Seethaler, and A. S. Milani. 2021. “Making Costly Manufacturing Smart with Transfer Learning Under Limited Data: A Case Study on Composites Autoclave Processing.” Journal of Manufacturing Systems 59 (April): 345–354. https://doi.org/10.1016/j.jmsy.2021.02.015.
   Google Scholar
• Ramezankhani, M., A. Narayan, R. Seethaler, and A. S. Milani. 2021. “An Active Transfer Learning (ATL) Framework for Smart Manufacturing with Limited Data: Case Study on Material Transfer in Composites Processing.” In 2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), 277–282. IEEE. https://doi.org/10.1109/ICPS49255.2021.9468145.
   Google Scholar
• Ramezankhani, M., A. Nazemi, A. Narayan, H. Voggenreiter, M. Harandi, R. Seethaler, and A. S. Milani. 2022. “A Data-Driven Multi-Fidelity Physics-Informed Learning Framework for Smart Manufacturing: A Composites Processing Case Study,” February. http://arxiv.org/abs/2202.06139.
   Google Scholar
• Reshef, D. N., Y. A. Reshef, H. K. Finucane, S. R. Grossman, G. McVean, P. J. Turnbaugh, E. S. Lander, M. Mitzenmacher, and P. C. Sabeti. 2011. “Detecting Novel Associations in Large Data Sets.” Science 334 (6062): 1518–1524. https://doi.org/10.1126/science.1205438.
   PubMed Web of Science ®Google Scholar
• Romero, M., W. Guédria, H. Panetto, and B. Barafort. 2020. “Towards a Characterisation of Smart Systems: A Systematic Literature Review.” Computers in Industry 120:103224. https://doi.org/10.1016/j.compind.2020.103224.
   Web of Science ®Google Scholar
• Shorten, C., and T. M. Khoshgoftaar. 2019. “A Survey on Image Data Augmentation for Deep Learning.” Journal of Big Data 6 (1): 1–48. https://doi.org/10.1186/s40537-019-0197-0.
   Google Scholar
• Tercan, H., A. Guajardo, and T. Meisen. 2019. “Industrial Transfer Learning: Boosting Machine Learning in Production.” In 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), Helsinki, Finland, 1:274–279. IEEE.
   Google Scholar
• Throne, J. L. 2008. Understanding Thermoforming. http://www.christiani.es/pdf/84439_probe.pdf.
   Google Scholar
• Tschannen, M., O. Bachem, and M. Lucic. 2018. “Recent Advances in Autoencoder-Based Representation Learning.” ArXiv Preprint ArXiv:1812.05069.
   Google Scholar
• Van Der Maaten, L. 2009. “Learning a Parametric Embedding by Preserving Local Structure.” In Artificial Intelligence and Statistics, edited by David van Dyk, and Max Welling, 384–391. Clearwater Beach, FL, USA: PMLR.
   Google Scholar
• Vapnik, V., S. E. Golowich, and A. J. Smola. 1997. “Support Vector Method for Function Approximation, Regression Estimation and Signal Processing.” Proceedings of the 9th International Conference on Neural Information Processing Systems, Denver, Colorado, 281–287. https://doi.org/10.5555/2998981.2999021.
   Google Scholar
• Wang, P., Z. Liu, R. X. Gao, and Y. Guo. 2019. “Heterogeneous Data-Driven Hybrid Machine Learning for Tool Condition Prognosis.” CIRP Annals 68 (1): 455–458. https://doi.org/10.1016/j.cirp.2019.03.007.
   Web of Science ®Google Scholar
• Wang, J., M. Yulin, L. Zhang, R. X. Gao, and W. Dazhong. 2018. “Deep Learning for Smart Manufacturing: Methods and Applications.” Journal of Manufacturing Systems 48 (1): 144–156. https://doi.org/10.1016/j.jmsy.2018.01.003.
   Google Scholar
• Wei, Z., Y. Feng, Z. Hong, Q. Rongxia, and J. Tan. 2017. “Product Quality Improvement Method in Manufacturing Process Based on Kernel Optimisation Algorithm.” International Journal of Production Research 55 (19): 5597–5608. https://doi.org/10.1080/00207543.2017.1324223.
   Web of Science ®Google Scholar
• Wellmer, Z. 2020. “WorldModels.” https://github.com/zacwellmer/WorldModels.
   Google Scholar
• Xiao, Y., H. Shao, S. Han, Z. Huo, and J. Wan. 2022. “Novel Joint Transfer Network for Unsupervised Bearing Fault Diagnosis from Simulation Domain to Experimental Domain.” IEEE/ASME Transactions on Mechatronics 27 (6): 5254–5263. https://doi.org/10.1109/TMECH.2022.3177174.
   Web of Science ®Google Scholar
• Xie, C. 2012. “Interactive Heat Transfer Simulations for Everyone.” The Physics Teacher 50 (4): 237–240. https://doi.org/10.1119/1.3694080.
   Google Scholar
• Xu, K., L. Yingguang, C. Liu, X. Liu, X. Hao, J. Gao, and P. G. Maropoulos. 2020. “Advanced Data Collection and Analysis in Data-Driven Manufacturing Process.” Chinese Journal of Mechanical Engineering 33:1–21. https://doi.org/10.1186/s10033-020-00459-x.
   Web of Science ®Google Scholar
• Yosinski, J., J. Clune, Y. Bengio, and H. Lipson. 2014. “How Transferable are Features in Deep Neural Networks?” Advances in Neural Information Processing Systems 4 (January): 3320–3328.
   Google Scholar
• Zhang, S., C. Kang, Z. Liu, W. Juan, and M. Chunmin. 2020. “A Product Quality Monitor Model with the Digital Twin Model and the Stacked Auto Encoder.” IEEE Access 8:113826–113836. https://doi.org/10.1109/ACCESS.2020.3003723.
   Web of Science ®Google Scholar
• Zhao, W., J. P. Queralta, and T. Westerlund. 2020. “Sim-To-Real Transfer in Deep Reinforcement Learning for Robotics: A Survey.” In 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, Australia, 737–744. IEEE.
   Google Scholar
• Zoph, B., G. Ghiasi, T.-Y. Lin, Y. Cui, H. Liu, E. D. Cubuk, and Q. V. Le. 2020. “Rethinking Pre-Training and Self-Training.” ArXiv Preprint ArXiv:2006.06882.
   Google Scholar