61
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Gesture-driven interaction service system for complex operations in digital twin manufacturing cells

, , &
Received 15 Oct 2023, Accepted 24 May 2024, Published online: 13 Jun 2024
 

Abstract

Industry 5.0 emphasises collaborative work between humans, advanced technology, and artificial intelligence robots, focusing on human-centric principles and integrating flexibility and sustainability to enhance workflows. The advancement of human-robot interaction services can significantly improve the operations efficiency of digital twin manufacturing cells. Motivated by the above background, a gesture-driven interaction architecture for digital twin manufacturing cells is proposed, including data acquisition layer, data processing layer, and application service layer. Secondly, deep learning algorithms are employed for recognising predefined gestures. An improved YOLOv5 algorithm is used to solve the problem of low accuracy in static gesture recognition; while a 3D-CNN-based multimodal data fusion algorithm is used to solve the problem of continuity, diversity, and dimensionality in dynamic gesture recognition. Ultimately, the prototype system is developed utilising Kinect 2.0 and Unity 3D, which involves linking the gesture recognition to the digital twin model, and linking the digital twin model to the physical manufacturing cells. This study is expected to provide theoretical and practical insights to empower human-robot interaction technology in manufacturing cells.

Acknowledgements

This work was supported in part by the National Key R&D Program of China (No. 2021YFB3301702), and the Major Special Science and Technology Project of Shaanxi Province, China (No. 2018zdzx01-01-01).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Competing interests

The authors declare that they have no competing interests.

Additional information

Funding

This work was supported by National Key Research and Development Program of China: [Grant Number ]; Major Special Science and Technology Project of Shaanxi Province, China: [Grant Number 2018zdzx01-01-01].

Log in via your institution

Log in to Taylor & Francis Online

PDF download + Online access

  • 48 hours access to article PDF & online version
  • Article PDF can be downloaded
  • Article PDF can be printed
USD 61.00 Add to cart

Issue Purchase

  • 30 days online access to complete issue
  • Article PDFs can be downloaded
  • Article PDFs can be printed
USD 438.00 Add to cart

* Local tax will be added as applicable

Related Research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.