Repetitive assembly basic action detection and standard work measurement based on deep learning

References

• Aehnelt, M., E. Gutzeit, and B. Urban. 2014. “Using Activity Recognition for the Tracking of Assembly Processes: Challenges and Requirements.” 2014 (2014): 12–21.
   Google Scholar
• Al-Amin, M., W. J. Tao, D. Doell, R. Lingard, Z. Z. Yin, M. C. Leu, and R. W. Qin. 2019. “Action Recognition in Manufacturing Assembly Using Multimodal Sensor Fusion.” 25th International Conference on Production Research Manufacturing Innovation: Cyber Physical Manufacturing, 158–167 39. https://doi.org/10.1016/j.promfg.2020.01.288.
   Google Scholar
• Albert, J. A., V. Owolabi, A. Gebel, C. M. Brahms, U. Granacher, and B. Arnrich. 2020. “Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect V2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study.“ Sensors 20 (18): 5104. https://doi.org/10.3390/s20185104
   PubMed Web of Science ®Google Scholar
• Antico, M., N. Balletti, G. Laudato, A. Lazich, M. Notarantonio, R. Oliveto, S. Ricciardi, S. Scalabrino, and J. Simeone. 2021. “Postural Control Assessment via Microsoft Azure Kinect DK: An Evaluation Study.” Computer Methods and Programs in Biomedicine 209 (2021): 106324. https://doi.org/10.1016/j.cmpb.2021.106324.
   PubMedGoogle Scholar
• Barata, J., J. C. S. Cardoso, and P. R. Cunha. 2023. “Mass Customization and Mass Personalization Meet at the Crossroads of Industry 4.0: A Case of Augmented Digital Engineering.” Systems Engineering. https://doi.org/10.1002/sys.21682.
   Web of Science ®Google Scholar
• Ben-Shabat, Y., X. Yu, F. Saleh, D. Campbell, C. Rodriguez-Opazo, H. Li, and S. Gould. 2021. “The Ikea Asm Dataset: Understanding People Assembling Furniture Through Actions, Objects and Pose.” Paper Presented at the Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.
   Google Scholar
• Berg, J., T. Reckordt, C. Richter, and G. Reinhart. 2018. “Action Recognition in Assembly for Human-Robot-Cooperation Using Hidden Markov Models.” Procedia CIRP 76 (2018): 205–210. https://doi.org/10.1016/j.procir.2018.02.029
   Google Scholar
• Brehar, R. D., M. P. Muresan, T. Mariţa, C. C. Vancea, M. Negru, and S. Nedevschi. 2021. “Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences.” IEEE Access Nedevschi 9 (2021): 74302–74324. https://doi.org/10.1109/ACCESS.2021.3080822.
   Google Scholar
• Buch, S., V. Escorcia, C. Shen, B. Ghanem, and J. Carlos Niebles. 2017. “Sst: Single-Stream Temporal Action Proposals.” Paper Presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
   Google Scholar
• Carreira, J., and A. Zisserman. 2017. “Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset.” Paper presented at the proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
   Google Scholar
• Chen, C., T. Wang, D. Li, and J. Hong. 2020. “Repetitive Assembly Action Recognition Based on Object Detection and Pose Estimation.” Journal of Manufacturing Systems Hong 55 (2020): 325–333. https://doi.org/10.1016/j.jmsy.2020.04.018.
   Google Scholar
• Feichtenhofer, C., H. Fan, J. Malik, and K. He. 2019. “Slowfast Networks for Video Recognition.” Paper Presented at the Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South).
   Google Scholar
• Gkournelos, C., C. Konstantinou, P. Angelakis, E. Tzavara, and S. Makris. 2023. “Praxis: A Framework for AI-Driven Human Action Recognition in Assembly.” Journal of Intelligent Manufacturing (2023): 1–15. https://doi.org/10.1007/s10845-023-02228-8.
   Google Scholar
• González, L., J. C. Alvarez, A. M. López, and D. Alvarez. 2020. “Evaluation of Optical Motion Capture System Performance in Humanrobot Collaborative Cells.” Paper Presented at the 2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Karlsruhe, Germany.
   Google Scholar
• Han, S., M. Achar, S. H. Lee, and F. Peña-Mora. 2013. “Empirical Assessment of a RGB-D Sensor on Motion Capture and Action Recognition for Construction Worker Monitoring.” Visualization in Engineering Volume 1 (1): 1–13. https://doi.org/10.1186/2213-7459-1-6.
   Google Scholar
• Heyde, GC. 1996. “Modular Arrangement of Predetermined Time Standards. Woodbridge, VA.”
   Google Scholar
• Hinton, G., O. Vinyals, and J. Dean. 2015. “Distilling the Knowledge in a Neural Network.”
   Google Scholar
• Hsieh, H. Y., D. J. Chen, and T. L. Liu. 2020. “Temporal Action Proposal Generation via Deep Feature Enhancement.” Proceedings of the 2020 IEEE International Conference on Image Processing (Icip), Abu Dhabi, United Arab Emirates, 1391–1395.
   Google Scholar
• Huangjun. 2021. Pp-Tsm Video Classification Model. https://aistudio.baidu.com/projectdetail/3399656.
   Google Scholar
• Kong, Y., and Y. Fu. 2022. “Human Action Recognition and Prediction: A Survey.” International Journal of Computer Vision 130 (5): 1366–1401.
   Web of Science ®Google Scholar
• Li, X., H. Qian, W. Zhijian, H. Zhenjie, and T. Yang. 2021. “Sequentialpointnet: A Strong Parallelized Point Cloud Sequence Network for 3d Action Recognition.”
   Google Scholar
• Ligorio, G., E. Bergamini, L. Truppa, M. Guaitolini, M. Raggi, A. Mannini, A. M. Sabatini, G. Vannozzi, and P. Garofalo. 2020. “A Wearable Magnetometer-Free Motion Capture System: Innovative Solutions for Real-World Applications.” IEEE Sensors Journal 20 (15): 8844–8857.
   Web of Science ®Google Scholar
• Lin, J., C. Gan, and S. Han. 2019. “Tsm: Temporal Shift Module for Efficient Video Understanding.” Paper Presented at the Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South).
   Google Scholar
• Lin, T., X. Zhao, and Z. Shou. 2017. “Single Shot Temporal Action Detection.” Paper Presented at the Proceedings of the 25th ACM International Conference on Multimedia, Mountain View California USA.
   Google Scholar
• Liu, M., H. Liu, and C. Chen. 2017. “Enhanced Skeleton Visualization for View Invariant Human Action Recognition.” Pattern Recognition 68 (2017) :346–362. https://doi.org/10.1016/j.patcog.2017.02.030.
   Google Scholar
• Liu, W., D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. 2016. “Ssd: Single Shot Multibox Detector.” Paper Presented at the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. October 11–14, 2016, Proceedings, Part I 14.
   Google Scholar
• Liu, X., Q. Wang, Y. Hu, X. Tang, S. Zhang, S. Bai, and X. Bai. 2022. “End-To-End Temporal Action Detection with Transformer.” IEEE Transactions on Image Processing 31 (2022): 5427–5441. https://doi.org/10.1109/TIP.2022.3195321.
   PubMedGoogle Scholar
• Liu, Z., H. Zhang, Z. Chen, Z. Wang, and W. Ouyang. 2020. “Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition.” Paper Presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.
   Google Scholar
• Ragusa, F., A. Furnari, S. Livatino, and G. Maria Farinella. 2021. “The Meccano Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-Like Domain.” Paper Presented at the Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.
   Google Scholar
• Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2016. “You Only Look Once: Unified, Real-Time Object Detection.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), 779–788. https://doi.org/10.1109/Cvpr.2016.91.
   Google Scholar
• Sener, F., D. Chatterjee, D. Shelepov, K. He, D. Singhania, R. Wang, and A. Yao. 2022. “Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities.” Paper Presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.
   Google Scholar
• Shun’Ichi, N., K. Nakayama, and H. Nakayama. 2002. “A Study on Setting Standard Time Using Work Achievement Quotient.” International Journal of Production Research 40 (15): 3945–3953. https://doi.org/10.1080/00207540210159581.
   Web of Science ®Google Scholar
• Simonyan, K., and A. Zisserman. 2014. “Two-Stream Convolutional Networks for Action Recognition in Videos.” Advances in Neural Information Processing Systems 27 (Nips 2014) 27 (2014): 568–576. https://doi.org/10.48550/arXiv.1406.2199.
   Google Scholar
• Sooksatra, S., and S. Watcharapinchai. 2022. “A Comprehensive Review on Temporal-Action Proposal Generation.” Journal of Imaging 8 (8): 207.
   PubMed Web of Science ®Google Scholar
• Telsang, M. T. 1998. Industrial Engineering and Production Management. Beijing, China: Machinery Industry Publishing House.
   Google Scholar
• Tran, D., L. Bourdev, R. Fergus, L. Torresani, and M. Paluri. 2015. “Learning Spatiotemporal Features with 3d Convolutional Networks.” Paper Presented at the Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.
   Google Scholar
• Unzeitig, W., M. Wifling, A. Stocker, and M. Rosenberger. 2015. “Industrial Challenges in Human-Centred Production.” Paper Presented at the Proceedings of the MOTSP 2015-International Conference Management of Technology, Porec, Croatia.
   Google Scholar
• Wang, J., Z. Liu, Y. Wu, and J. Yuan. 2012. “Mining Actionlet Ensemble for Action Recognition with Depth Cameras.” Paper Presented at the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.
   Google Scholar
• Wang, J., S. Tan, X. Zhen, S. Xu, F. Zheng, Z. He, and L. Shao. 2021. “Deep 3D Human Pose Estimation: A Review.” Computer Vision and Image Understanding 210 (2021): 103225. https://doi.org/10.1016/j.cviu.2021.103225.
   Google Scholar
• Wang, L., R. Gao, J. Váncza, J. Krüger, X. Vincent Wang, S. Makris, and G. Chryssolouris. 2019. “Symbiotic Human-Robot Collaborative Assembly.” CIRP Annals 68 (2): 701–726.
   Web of Science ®Google Scholar
• Wang, L. M., Y. J. Xiong, Z. Wang, Y. Qiao, D. H. Lin, X. O. Tang, and L. Van Gool. 2016. “Temporal Segment Networks: Towards Good Practices for Deep Action Recognition.” Computer Vision - ECCV 9912:20–36. https://doi.org/10.1007/978-3-319-46484-8_2.
   Google Scholar
• Xia, H., and Y. Zhan. 2020. “A Survey on Temporal Action Localization.” IEEE Access Zhan 8 (2020): 70477–70487. https://doi.org/10.1109/ACCESS.2020.2986861.
   Google Scholar
• Zhang, J., P. Byvshev, and Y. Xiao. 2020. “A Video Dataset of a Wooden Box Assembly Process: Dataset.” Paper Presented at the Proceedings of the Third Workshop on Data: Acquisition To Analysis, New York, NY, United States.
   Google Scholar