Abstract
We developed an intelligent shadow play system (ShadowTouch) based on multi-dimensional interactive perception technologies to let user interact with the shadow play figure naturally. ShadowTouch included a shadow play hardware subsystem and a multi-dimensional interactive software subsystem. In shadow play hardware subsystem, each joint of the shadow play figure was actuated by the connecting rod which was rotated through one steering gear. In multi-dimensional interactive software subsystem, a somatosensory interaction module synchronized motions between shadow play figure and human body based on skeleton point detection, and an automatic choreography module generated the motions automatically for the shadow play figure based on musical emotion recognition. The proposed system improved the immersive interaction experience and let users feel involved during the performance of shadow play, and the motions of shadow play figure (steering gear motion angles) could be saved digitally to further be applied for the reactivation and inheritance of traditional culture.
Disclosure statement
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Funding
Notes on contributors
Zhichuan Tang
Zhichuan Tang received a PhD degree in Computer Science at Zhejiang University, China. He is an associate professor at Zhejiang University of Technology, China. His research interests include human-computer interface, deep learning and artificial intelligence.
Yidan Hu
Yidan Hu is currently a master student majoring in industrial design at Zhejiang University of Technology, China. Her research interests include human-computer interface and intelligent design.
Weining Weng
Weining Weng is currently a Ph.D. candidate in Computer Science at Chinese Academy of Sciences, China. His research interests include deep learning, affective computing and pattern recognition.
Lekai Zhang
Lekai Zhang received a PhD degree in Computer Science at Zhejiang University, China. He is a lecturer at Zhejiang University of Technology, China. His research interests include human-computer interface and affective computing.
Lingtao Zhang
Lingtao Zhang is currently a master student majoring in industrial design at Zhejiang University of Technology, China. His research interests include human-computer interface and deep learning.
Jichen Ying
Jichen Ying is currently a master student majoring in industrial design at Zhejiang University of Technology, China. His research interests include human-computer interface and intelligent system.