Human-Centered Interaction in Virtual Worlds: A New Era of Generative Artificial Intelligence and Metaverse

Abstract

The metaverse has emerged as an exciting new paradigm for human-computer interaction (HCI) and virtual collaboration. This paper presents a comprehensive review of the metaverse to address the gap in the existing literature where there is a lack of a survey that reviews the nature of the metaverse and its building blocks from a human-centric perspective. We first synthesize a definition of the metaverse from existing literature and delineate key affordances. We then introduce a detailed framework encompassing the metaverse’s nature, infrastructure technologies, and input/output technologies that facilitate multi-sensory HCI, alongside applications across diverse domains. The components within this framework are explained in depth, offering insights into the metaverse’s nature and the readiness level of current technologies. Based on this comprehensive analysis, we outline major open challenges and propose promising directions demanding further exploration and investigation. By clarifying the vision for the metaverse and characterizing the building blocks required to realize it, this review provides essential insights and serves as an invaluable resource for metaverse developers and researchers working to advance this transformative new medium.

Keywords:

• Metaverse
• human-computer interaction
• technical framework

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Yuying Wang

Yuying Wang is a PhD in the Department of Information Systems at the City University of Hong Kong. Her research revolves around human-computer interaction within the metaverse, with a focus on the enabling technologies and the profound behavioral impacts that the metaverse brings to individuals.

Le Wang

Le Wang is an assistant professor in the Department of Information Systems at the City University of Hong Kong. His general areas of research are health informatics, application of AI in business, and user behaviour in digital platforms, with a focus on understanding individual behaviour in this digital era.

Keng Leng Siau

Keng Leng Siau is the Head and Chair Professor of the Department of Information Systems and Chair Professor (Affiliate) of the School of Data Science at the City University of Hong Kong. He is an AIS Distinguished Member–Cum Laude and an AIS Fellow.