Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

ABSTRACT

Gestural interaction devices emerged and originated various studies on multimodal human–computer interaction to improve user experience (UX). However, there is a knowledge gap regarding the use of these devices to enhance learning. We present an exploratory study which analysed the UX with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality (VR) environment: one as the helmsman aboard Vasco da Gama’s fifteenth-century Portuguese ship and the other as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a VR headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterisation of UX, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human–computer interaction.

KEYWORDS:

• Multimodal interaction
• user experience
• digital game-based learning
• virtual reality
• gesture interaction
• augmented reality

Acknowledgements

The goal of InMERSE was to study the potential of combining gestural interaction, immersive visualisation, and augmented visualisation, using readily available low-cost devices, in entertainment or business use contexts.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. http://www.nintendo.com/wiiu/accessories.

2. https://leapmotion.com/.

3. https://parallax.com/product/28046.

4. https://thalmic.com/myo/.

5. http://us.playstation.com/ps2/accessories/eyetoy-usb-camera-ps2.html.

6. https://microsoft.com/en-us/kinectforwindows/.

7. https://google.com/cardboard/.

8. http://vuzix.com/UKSITE/consumer/products_vr920.html.

9. https://www.oculus.com/en-us/.

10. https://developers.google.com/glass/.

11. https://www.metavision.com/.

12. https://www.microsoft.com/microsoft-hololens/en-us.

Additional information

Funding

This work was developed in the scope of the InMERSE project, developed at INESC TEC in cooperation and with funding from Portugal Telecom Inovação (now Altice Labs). This work is partly financed by the ERDF – European Regional Development Fund – through the Operational Programme for Competitiveness and Internationalisation – COMPETE 2020 Programme, and by National Funds through the Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) within the project ‘POCI-01-0145-FEDER-006961’.