Analysing student engagement with 360-degree videos through multimodal data analytics and user annotations

ABSTRACT

This paper presents the findings from a case study which explores how students engage with learning materials mediated through 360-degree panoramic videos. The case study extends findings from a project which developed an online platform for viewing and annotating 360-degree videos, and for providing data analytics (heatmaps, viewpoint tracking and area of interest displays) to map student engagement with video content. In this case, information obtained from the data analytics forms the basis for the analysis which, taking a social semiotic perspective, explores (a) what multimodal resources in the video-recorded classroom activity function to attract and hold students’ attention, and (b) what multimodal resources students employ when interacting with the 360-degree videos and for which purpose, taking the context into account. The findings indicate that for students to engage meaningfully with educational content requires familiarity with the demands of 360 video technology, which for some users can prove distracting, or even disabling.

KEYWORDS:

• 360-degree video technology
• student engagement
• data analytics
• heatmaps
• multimodality

Acknowledgments

This work extends findings from the project ‘Encouraging and Mapping Student Engagement through 360-Degree Video Annotation and Data Analytics’ with Lead Chief Investigator Professor Kay O’Halloran, funded by the Australian Technology Network (for further information, see: http://online360video.education/).

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed here.

Additional information

Funding

This work was supported by the Australian Technology Network (ATN).

Notes on contributors

Sabine Tan

Dr Sabine Tan is a Senior Research Fellow in the School of Education, Faculty of Humanities at Curtin University. Her research interests include critical multimodal discourse analysis, social semiotics and visual communication. She isparticularly interested in the application of social semiotic theory to the analysis of institutional discourses involvingtraditional and new media, and interdisciplinary projects involving the development of interactive and collaborativesoftware for research and educational purposes.

Michael Wiebrands

Michael Wiebrands is an information technology (IT) professional with 20 years’ experience in the tertiary education sector. He has filled a broad range of IT roles, including software development, system administration, applicationsupport, database administration and project management. His current focus is the use of game engines to facilitate thecreation of education tools. He has a particular interest in 3D visualisation and virtual reality.

Kay O’Halloran

Professor Kay O’Halloran is Head of Department and Chair in Communication & Media in the School of the Arts at the University of Liverpool in the UK. Her areas of research include multimodal analysis, social semiotics, mathematics discourse and the development of interactive digital media technologies and visualisation techniques for multimodaland sociocultural analytics.

Peter Wignell

Dr Peter Wignell was a Senior Research Fellow in the School of Education, Faculty of Humanities at Curtin University. Hisresearch interests are in Systemic Functional Linguistics, especially in its application to the analysis of multimodal texts, with a recent focus on discourses of violent extremism. His research has also focused on the role of language in the construction of specialised knowledge.