Effect of 2D/3D Visual and Haptic Cues for Fine Operation Tasks With Varying Difficulties

Abstract

Fine operations exist in the form of teleoperation, human-machine cooperative work, and fine manual assembly. However, there is a lack of research that explores the characteristics of different assistive guidance methods and their use for fine operation tasks with different levels of difficulty. For this issue, this study first designed a prototype of a magnetic-based kinesthetic interaction device, and then using this device, a user study (n = 27) was designed and conducted. The experiment explored the performance of 2D and 3D visual cues as well as haptic cues when facing operation tasks with different levels of difficulty and analyzed the characteristics of each guidance modality and its reasons. The experimental results suggest that (1) visual cues are better suited than haptic cues to convey directional information in fine operations, (2) haptic cues can introduce an additional physical load, and (3) 2D visual cues’ missing depth information can affect operations in the view direction.

Keywords:

• Haptic interaction
• fine manipulation
• user research
• augmented reality
• assembly

Acknowledgement

Thanks to Prof. He for the inspiration on ideas and innovations. Thanks to Prof. Wang for suggestions on writing and experimentation. Thanks to all volun-teers who participated in this experiment.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Key R&D Program of China [Grant No. 2021YFB1716200].

Notes on contributors

Zhenghang Hou

Zhenghang Hou received the bachelor’s degree in advanced manufacturing engineering from Northwestern Polytechnical University, in 2019, where he is currently pursuing the Ph.D. degree with the Cyber-Physical Interaction Laboratory, supervised by Prof. W. He. His research interests include digital twin assembly, augmented reality assembly, human computer interaction.

Weiping He

Weiping He is a professor in the School of Mechanical Engineering at Northwestern Polytechnical University. He is interested in geometrical modeling, 2D bar-code recognition, direct part marking and automatic identification, augmented reality, and mixed reality.

Shuxia Wang

Shuxia Wang is a professor at Northwestern Polytechnical University. She is interested in VR/AR/XR-based intelligent assembly, digital twin-based, Intelligent manufacturing, natural human-computer interaction based on augmented reality, virtual training and assessment based on cognitive fatigue.