1,468
Views
1
CrossRef citations to date
0
Altmetric
Research Article

Towards gestured-based technologies for human-centred Smart Factories

, , , , & ORCID Icon
Pages 110-127 | Received 22 Jul 2021, Accepted 31 Aug 2022, Published online: 13 Sep 2022

References

  • Ansari, F., P. Hold, W. Mayrhofer, S. Schlund, and W. Sihn. 2018. AUTODIDACT: Introducing the Concept of Mutual Learning into a Smart Factory Industry 4.0. International Association for Development of the Information Society. 978-989-8533-81-4. http://www.iadisportal.org/digital-library/autodidact-introducing-the-concept-of-mutual-learning-into-a-smart-factory-industry-40
  • Atre, P., S. Bhagat, N. Pooniwala, and P. Shah. 2018. “Efficient and Feasible Gesture Controlled Robotic Arm.” In 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), 14.06.2018 - 15.06.2018, 1–6. Madurai, India: IEEE. doi:10.1109/ICCONS.2018.8662943.
  • Auquilla, A. R., H. T. Salamea, O. Alvarado-Cando, J. K. Molina, and P. A. S. Cedillo. 2019. “Implementation of a Telerobotic System Based on the Kinect Sensor for the Inclusion of People with Physical Disabilities in the Industrial Sector.” In 2019 IEEE 4th Colombian Conference on Automatic Control (CCAC), 15.10.2019 - 18.10.2019, 1–6. Medellín, Colombia: IEEE. doi:10.1109/CCAC.2019.8921359.
  • Battini, D., A. Persona, and F. Sgarbossa. 2014. “Innovative Real-Time System to Integrate Ergonomic Evaluations into Warehouse Design and Management.” Computers & Industrial Engineering 77: 1–10. doi:10.1016/j.cie.2014.08.018.
  • Boenzi, F., S. Digiesi, F. Facchini, and G. Mummolo. 2016. “Ergonomic Improvement Through Job Rotations in Repetitive Manual Tasks in Case of Limited Special- Ization and Differentiated Ergonomic Requirements.” IFAC-PapersOnline 49 (12): 1667–1672. doi:10.1016/j.ifacol.2016.07.820.
  • Bortolini, M., M. Faccio, M. Gamberi, and F. Pilati. 2018a. “Motion Analysis System (MAS) for Production and Ergonomics Assessment in the Manufacturing Processes.” Computers & Industrial Engineering 139. doi:10.1016/j.cie.2018.10.046.
  • Bortolini, M., M. Gamberi, F. Pilati, and A. Regattieri. 2018b. “Automatic Assessment of the Ergonomic Risk for Manual Manufacturing and Assembly Activities Through Optical Motion Capture Technology.” Procedia CIRP 72: 81–86. doi:10.1016/j.procir.2018.03.198.
  • Chen, S., H. Ma, C. Yang, and M. Fu. 2015. “Hand Gesture Based Robot Control System Using Leap Motion.” In Intelligent Robotics and Applications, Lecture Notes in Computer Science, edited by H. Liu, N. Kubota, X. Zhu, R. Dillmann, and D. Zhou, 581–591. Vol. 9244. Cham: Springer International Publishing. doi:10.1007/978-3-319-22879-2_53.
  • Chuma, E. L., and Y. Iano. 2021. “A Movement Detection System Using Continuous-Wave Doppler Radar Sensor and Convolutional Neural Network to Detect Cough and Other Gestures.” IEEE Sensors Journal 21 (3, February 1): 2921–2928. doi:10.1109/JSEN.2020.3028494.3.
  • Coban, M., and G. Gelen. 2018. “Wireless Teleoperation of an Industrial Robot by Using Myo Arm Band.” In 2018 International Conference on Artificial Intelligence and Data Processing (IDAP), 28.09.2018 - 30.09.2018, 1–6. Malatya, Turkey: IEEE. doi:10.1109/IDAP.2018.8620789.
  • Colombini, D. 1998. “An Observational Method for Classifying Exposure to Repetitive Movements of the Upper Limbs.” Ergonomics 41 (9): 1261–1289. doi:10.1080/001401398186306.
  • Cruz, L., D. Lucio, and L. Velho. 2012. “Kinect and Rgbd Images: Challenges and Applications.” In Proceedings of the 2012 25th SIBGRAPI Conference on Graphics, Pat- terns and Images Tutorials. IEEE. doi:10.1109/SIBGRAPI-T.2012.13.
  • Cueva, C. W. F., S. H. M. Torres, and M. J. Kern. 2017. “7 DOF Industrial Robot Controlled by Hand Gestures Using Microsoft Kinect V2.” In 2017 IEEE 3rd Colombian Conference on Automatic Control (CCAC), 18.10.2017 - 20.10.2017, 1–6. Cartagena: IEEE. doi:10.1109/CCAC.2017.8276455.
  • Dimitropoulos, N., T. Togias, N. Zacharaki, G. Michalos, and S. Makris. 2021. “Seamless Human–robot Collaborative Assembly Using Artificial Intelligence and Wearable Devices.” Applied Sciences 11 (12): 5699. doi:10.3390/app11125699.
  • Eurofund (European Foundation for the Improvement of Living and Working Conditions). 2019. “Sixth European Working Conditions Survey: Working Conditions and Workers’ Health”.
  • Eurostat (European Statistical Office). 2019. “Population Structure and Ageing.”
  • Facchini, F., G. Mummolo, G. Mossa, S. Digiesi, F. Boenzi, and R. Verriello. 2016. “Minimizing the Carbon Footprint of Material Handling Equipment: Comparison of Electric and LPG Forklifts.” Journal of Industrial Engineering and Management 9 (5): 1035–1046. doi:10.3926/jiem.2082.
  • Fan, Y., C. Yang, and X. Wu. 2019. “Improved Teleoperation of an Industrial Robot Arm System Using Leap Motion and MYO Armband.” In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), 06.12.2019 - 08.12.2019, 1670–1675. Dali, China: IEEE. doi:10.1109/ROBIO49542.2019.8961758.
  • Fuad, M. 2015. “Skeleton Based Gesture to Control Manipulator.” In 2015 International Conference on Advanced Mechatronics, Intelligent Manufacture, and Industrial Automation (ICAMIMIA), 15.10.2015 - 17.10.2015, 96–101. Surabaya, Indonesia: IEEE. doi:10.1109/ICAMIMIA.2015.7508010
  • Gardasevic, G., H. Fotouhi, I. Tomasic, M. Vahabi, M. Bjorkman, and M. Linden. 2017. “A Heterogeneous IoT-Based Architecture for Remote Monitoring of Physiological and Environmental Parameters.” In 4th EAI International Conference on IoT Technologies for HealthCare, Angers, France, October 24–25. doi:10.1007/978-3-319-76213-5_7.
  • Garg, A., D. B. Chaffin, and G. D. Herrin. 1978. “Prediction of Metabolic Rates for Manual Materials Handling Jobs.” American Industrial Hygiene Association Journal 39 (8): 661–674. doi:10.1080/0002889778507831.
  • Golabchi, A., S. Han, J. Seo, S. Han, S. Lee, and M. Al-Hussein. 2015. “An Automated Biomechanical Simulation Approach to Ergonomic Job Analysis for Workplace Design.” Journal of Construction Engineering and Management 141 (8): 4015020. doi:10.1061/(ASCE)CO.1943-7862.0000998.
  • Haggag, H., M. Hossny, S. Nahavandi, and D. Creighton. 2013. “Real Time Ergonomic As- Sessment for Assembly Operations Using Kinect.” In Proceedings of the 2013 UK- Sim 15th International Conference on Computer Modelling and Simulation, 495–500. IEEE. doi:10.1109/UKSim.2013.105.
  • Heilala, J., and P. Voho. 2001. “Modular Reconfigurable Flexible Final Assembly Systems.” Assembly Automation 21 (1): 20–30. doi:10.1108/01445150110381646.
  • Hignett, S., and L. McAtamney. 2000. “Rapid Entire Body Assessment (REBA).” Applied Ergonomics 31 (2): 201–205. doi:10.1016/S0003-6870(99)00039-3.
  • Hong, C., Z. Chen, J. Zhu, and X. Zhang. 2017. “Interactive Humanoid Robot Arm Imitation System Using Human Upper Limb Motion Tracking.” In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), 05.12.2017 - 08.12.2017, 2746–2751. Macau: IEEE. doi:10.1109/ROBIO.2017.8324706.
  • Islam, J., A. Ghosh, M. I. Iqbal, S. Meem, and N. Ahmad. 2020. “Integration of Home Assistance with a Gesture Controlled Robotic Arm.” In 2020 IEEE Region 10 Symposium (TENSYMP), 05.06.2020 - 07.06.2020, 266–270. Dhaka, Bangladesh: IEEE. doi:10.1109/TENSYMP50017.2020.9230893.
  • Kaczmarek, W., J. Panasiuk, S. Borys, and P. Banach. 2020. “Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode.” Sensors (Basel, Switzerland) 20 (21): 6358. doi:10.3390/s20216358.
  • Kandalaft, N., P. S. Kalidindi, S. Narra, and H. N. Saha. 2018. “Robotic Arm Using Voice and Gesture Recognition.” In 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), 01.11.2018 - 03.11.2018, 1060–1064. Vancouver, BC: IEEE. doi: 10.1109/IEMCON.2018.8615055.
  • Karhu, O., P. Kansi, and I. Kuorinka. 1977. “Correcting Working Postures in Industry: A Practical Method for Analysis.” Applied Ergonomics 8 (4): 199–201. doi:10.1016/0003-6870(77)90164-8.
  • Khan, M. B., Z. Zhang, L. Li, W. Zhao, M. A. M. A. Hababi, X. Yang, and Q. H. Abbasi. 2020. “A Systematic Review of Non-Contact Sensing for Developing a Platform to Contain COVID-19.” Micromachines 11 (10): 912. doi:10.3390/mi11100912.
  • Li, X. 2020. “Human–robot Interaction Based on Gesture and Movement Recognition.” Signal Processing: Image Communication 81: 115686. doi:10.1016/j.image.2019.115686.
  • Liles, D. H., S. Deivanayagam, M. M. Ayoub, and P. Mahajan. 1984. “A Job Severity In- Dex for the Evaluation and Control of Lifting Injury.” Human Factors: The Journal of the Human Factors and Ergonomics Society 26 (6): 683–693. doi:10.1177/001872088402600608.
  • Lin, C.-S., P.-C. Chen, Y.-C. Pan, C.-M. Chang, and K.-L. Huang. 2020. “The Manipulation of Real-Time Kinect-Based Robotic Arm Using Double-Hand Gestures.” Journal of Sensors 2020: 1–9. doi:10.1155/2020/9270829.
  • Lu, Z., X. Chen, Q. Li, X. Zhang, and P. Zhou. 2014. “A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices.” IEEE Transactions on Human-Machine Systems 44 (2): 293–299. doi:10.1109/THMS.2014.2302794.
  • Manghisi, V. M., M. Fiorentino, A. Boccaccio, M. Gattullo, G. L. Cascella, N. Toschi, A. E. Pietroiusti, and A. E. Uva. 2020. “A Body Tracking-Based Low-Cost Solution for Monitoring Workers’ Hygiene Best Practices During Pandemics.” Sensors 20 (21): 6149. doi:10.3390/s20216149.
  • Manghisi, V. M., A. E. Uva, M. Fiorentino, V. Bevilacqua, G. F. Trotta, and G. Monno. 2017. “Real Time RULA Assessment Using Kinect V2 Sensor.” Applied Ergonomics 65: 481–491. doi:10.1016/j.apergo.2017.02.015.
  • Manghisi, V. M., A. E. Uva, M. Fiorentino, M. Gattullo, A. Boccaccio, and G. Monno. 2018. “Enhancing User Engagement Through the User Centric Design of a Mid-Air Ges Ture-Based Interface for the Navigation of Virtual-Tours in Cultural Heritage Ex- Positions.” Journal of Cultural Heritage 32: 186–197. doi:10.1016/j.culher.2018.02.014.
  • Marinho, M. M., A. A. Geraldes, A. P. L. Bo, and G. A. Borges. 2012. “Manipulator Control Based on the Dual Quaternion Framework for Intuitive Teleoperation Using Kinect.” In 2012 Brazilian Robotics Symposium and Latin American Robotics Symposium, 16.10.2012 - 19.10.2012, 319–324. Fortaleza, Brazil: IEEE. doi:10.1109/SBR-LARS.2012.59.
  • Martin, C. C., D. C. Burkert, K. R. Choi, N. B. Wieczorek, P. M. McGregor, R. A. Herrmann, and P. A. Beling. 2012. “A Real-Time Ergonomic Monitoring System Using the Microsoft Kinect.” In Proceedings of the 2012 IEEE Systems and Information Engineering Design Symposium, 50–55. IEEE. doi:10.1109/SIEDS.2012.6215130.
  • McAtamney, L., and E. N. Corlett. 1993. “RULA: A Survey Method for the Investigation of Work-Related Upper Limb Disorders.” Applied Ergonomics 24 (2): 91–99. doi:10.1016/0003-6870(93)90080-S.
  • MediaPipe Handpose. 2022. ( 0.0.7), Accessed June 5 2022. https://www.npmjs.com/package/@tensorflow-models/handpose
  • Michalos, G., A. Karvouniari, N. Dimitropoulos, T. Togias, and S. Makris. 2018. “Workplace Analysis and Design Using Virtual Reality Techniques.” CIRP Annals 67 (1): 141–144. doi:10.1016/j.cirp.2018.04.120.
  • Michalos, G., S. Makris, L. Rentzos, and G. Chryssolouris. 2010. “Dynamic Job Rotation for Workload Balancing in Human Based Assembly Systems.” CIRP Journal of Manufacturing Science and Technology 2 (3): 153–160. doi:10.1016/j.cirpj.2010.03.009.
  • Moe, S., and I. Schjølberg. 2013. “Real-Time Hand Guiding of Industrial Manipulator in 5 DOF Using Microsoft Kinect and Accelerometer.” In 2013 IEEE RO-MAN, 26.08.2013 - 29.08.2013, 644–649. Gyeongju: IEEE. doi:10.1109/ROMAN.2013.6628421.
  • Moin, A., A. Zhou, A. Rahimi, A. Menon, S. Benatti, G. Alexandrov, S. Tamakloe, J. Ting, N. Yamamoto, Y. Khan, F. Burghardt, L. Benini, A. C. Arias, and J. M. Rabaey. 2021. “A Wearable Biosensing System with In-Sensor Adaptive Machine Learning for Hand Gesture Recognition.” Nature Electronics 4 (1): 54–63. doi:10.1038/s41928-020-00510-8.
  • Moore, J. S., and A. Garg. 1995. “The Strain Index: A Proposed Method to Analyze Jobs for Risk of Distal Upper Extremity Disorders.” American Industrial Hygiene Association Journal 56 (5): 443–458. doi:10.1080/15428119591016863.
  • Nguyen, T. D., M. Kleinsorge, A. Postawa, K. Wolf, R. Scheumann, J. Krüger, and G. Seliger. 2013. “Human Centric Automation: Using Marker-Less Motion Captur- Ing for Ergonomics Analysis and Work Assistance in Manufacturing Processes.” In Proceedings of the 11th Global Conference on Sustainable Manufacturing. Berlin, Germany. . 586–592. https://www.gcsm.eu/Papers/132/18.3_34.pdf
  • Occhipinti, E. 1998. “OCRA: A Concise Index for the Assessment of Exposure to Repetitive Movements of the Upper Limbs.” Ergonomics 41 (9): 1290–1311. doi:10.1080/001401398186315.
  • Paliyawan, P., C. Nukoolkit, and P. Mongkolnam. 2014. “Office Workers Syndrome Moni- Toring Using Kinect.” In Proceedings of the 20th Asia-Pacific Conference on Com- munication (APCC2014), 58–63. IEEE. doi:10.1109/APCC.2014.7091605.
  • Pedersen, M. R., L. Nalpantidis, R. S. Andersen, C. Schou, S. Bøgh, V. Krüger, and O. Madsen. 2016. “Robot Skills for Manufacturing: From Concept to Industrial Deployment.” Robotics and Computer-Integrated Manufacturing 37: 282–291. doi:10.1016/j.rcim.2015.04.002.
  • Peppoloni, L., A. Filippeschi, E. Ruffaldi, and C. A. Avizzano. 2016. “A Novel Wearable System for the Online Assessment of Risk for Biome- Chanical Load in Repetitive Efforts.” International Journal of Industrial Ergonomics 52: 1–11. doi:10.1016/j.ergon.2015.07.002.
  • Peruzzini, M., F. Grandi, S. Cavallaro, and M. Pellicciari. 2021. “Using Virtual Manufacturing to Design Human-Centric Factories: An Industrial Case.” International Journal of Advanced Manufacturing Technology 115 (3): 873–887. doi:10.1007/s00170-020-06229-2.
  • Peruzzini, M., and M. Pellicciari. 2017. “A Framework to Design a Human-Centred Adaptive Manufacturing System for Aging Workers.” Advanced Engineering Informatics 33: 330–349. doi:10.1016/j.aei.2017.02.003.
  • Plantard, P., H. P. Shum, A. S. Le Pierres, and F. Multon. 2017. “Validation of an Ergonomic Assessment Method Using Kinect Data in Real Workplace Conditions.” Applied Ergonomics 65: 562–569. doi:10.1016/j.apergo.2016.10.015.
  • Rehman, M., R. A. Shah, M. B. Khan, N. A. A. Ali, A. A. Alotaibi, T. Althobaiti, N. Ramzan, S. A. Shaha, X. Yang, A. Alomainy, M. A. Imran, and Q. H. Abbasi. 2021. “Contactless Small-Scale Movement Monitoring System Using Software Defined Radio for Early Diagnosis of COVID-19.” IEEE Sensors Journal 1. doi:10.1109/JSEN.2021.3077530.
  • Roman-Liu, D. 2014. “Comparison of Concepts in Easy-To-Use Methods for MSD Risk Assessment.” Applied Ergonomics 45 (3): 420–427. doi:10.1016/j.apergo.2013.05.010.
  • Romero, D., P. Bernus, O. Noran, J. Stahre, and Å. Fast-Berglund. 2016b. “The Operator 4.0: Human Cyber-Physical Systems & Adaptive Automation Towards Human-Au- Tomation Symbiosis Work Systems.” In Proceedings of IFIP International Confer- ence on Advances in Production Management Systems, 677–686. doi:10.1007/978-3-319-51133-7_80.
  • Romero, D., J. Stahre, T. Wuest, O. Noran, P. Bernus, A. Fast-Berglund, and D. Gorecky. 2016a. “Towards an Operator 4.0 Typology: A Human-Centric Per- Spective on the Fourth Industrial Revolution Technologies.” In Proceedings of In- ternational Conference on Computers & industrial Engineering (CIE46). Tianjin, China, 1–11.
  • Salamea, H. T., A. R. Auquilla, O. Alvarado-Cando, and C. U. Onate. 2020. “Control of a Telerobotic System Using Wi-Fi and Kinect Sensor for Disabled People with an Industrial Process.” In 2020 IEEE ANDESCON, 13.10.2020 - 16.10.2020, 1–6. Quito, Ecuador: IEEE. doi:10.1109/ANDESCON50619.2020.9272069.
  • Schaub, K., G. Caragnano, B. Britzke, and R. Bruder. 2013. “The European Assembly Work- Sheet.” Theoretical Issues in Ergonomics Science 14 (6): 616–639. doi:10.1080/1463922X.2012.678283.
  • Schmitt, J., A. Hillenbrand, P. Kranz, and T. Kaupp. 2021. “Assisted Human-Robot-Interaction for Industrial Assembly: Application of Spatial Augmented Reality (SAR) for Collaborative Assembly Tasks.” In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 52–56. doi:10.1145/3434074.3447127.
  • Shirwalkar, S., A. Singh, K. Sharma, and N. Singh. 2013. “Telemanipulation of an Industrial Robotic Arm Using Gesture Recognition with Kinect.” In 2013 International Conference on Control, Automation, Robotics and Embedded Systems (CARE), 16.12.2013 - 18.12.2013, 1–6. Jabalpur, India: IEEE. doi:10.1109/CARE.2013.6733747.
  • Sun, X., R. Houssin, J. Renaud, and M. Gardoni. 2018. “Towards a Human Factors and Ergonomics Integration Framework in the Early Product Design Phase: Function-Task-Behaviour.” International Journal of Production Research 56 (14): 4941–4953. doi:10.1080/00207543.2018.1437287.
  • Tang, G., and P. Webb. 2018. “The Design and Evaluation of an Ergonomic Contactless Gesture Control System for Industrial Robots.” Journal of Robotics 2018: 1–10. doi:10.1155/2018/9791286.
  • Tsarouchi, P., A. Athanasatos, S. Makris, X. Chatzigeorgiou, and G. Chryssolouris. 2016a. “High Level Robot Programming Using Body and Hand Gestures.” Procedia CIRP 55: 1–5. doi:10.1016/j.procir.2016.09.020.
  • Tsarouchi, P., S. Makris, and G. Chryssolouris. 2016b. “Human – Robot Interaction Review and Challenges on Task Planning and Programming.” International Journal of Computer Integrated Manufacturing 29 (8): 916–931. doi:10.1080/0951192X.2015.1130251.
  • Tsarouchi, P., G. Michalos, S. Makris, T. Athanasatos, K. Dimoulas, and G. Chryssolouris. 2017. “On a Human–robot Workplace Design and Task Allocation System.” International Journal of Computer Integrated Manufacturing 30 (12): 1272–1279. doi:10.1080/0951192X.2017.1307524.
  • Turk, M. 2014. “Multimodal Interaction: A Review.” Pattern Recognition Letters 36: 189–195. doi:10.1016/j.patrec.2013.07.003.
  • Vignais, N., M. Miezal, G. Bleser, K. Mura, D. Gorecky, and F. Marin. 2013. “Innovative System for Real-Time Ergonomic Feedback in Industrial Manufacturing.” Applied Ergonomics 44 (4): 566–574. doi:10.1016/j.apergo.2012.11.008.
  • Waters, T. R., V. Putz-Anderson, A. Garg, and L. J. Fine. 1993. “Revised NIOSH Equation for the Design and Evaluation of Manual Lifting Tasks.” Ergonomics 36 (7): 749–776. doi:10.1080/00140139308967940.
  • Webster, D., and O. Celik. 2014. “Systematic Review of Kinect Applications in Elderly Care and Stroke Rehabilitation.” Journal of Neuroengineering and Rehabilitation 11 (1): 108. doi:10.1186/1743-0003-11-108.
  • W. H. O. (World Health Organization). 2003. Protecting Workers-Health Series No. 5, Preventing Musculoskeletal Disorders in the Workplace, 2003.
  • Wilhelm, M., V. M. Manghisi, A. Uva, M. Fiorentino, V. Bräutigam, B. Engelmann, and J. Schmitt. 2021. “ErgoTakt: A Novel Approach of Human-Centered Balancing of Manual Assembly Lines.” Procedia CIRP 97: 354–360. doi:10.1016/j.procir.2020.05.250.
  • Yang, Y., H. Yan, M. Dehghan, and M. H. Ang. 2015. “Real-Time Human-Robot Interaction in Complex Environment Using Kinect V2 Image Recognition.” In 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), 15.07.2015 - 17.07.2015, 112–117. Siem Reap, Cambodia: IEEE. doi:10.1109/ICCIS.2015.7274606.
  • Zhang, F., V. Bazarevsky, A. Vakunov, A. Tkachenka, G. Sung, C. L. Chang, and M. Grundmann. 2020. “Mediapipe Hands: On-Device Real-Time Hand Tracking.” arXiv preprint arXiv:2006.10214.