Physiological Data for User Experience and Quality of Experience: A Systematic Review (2018–2022)

Abstract

The evaluation of human responses in multimedia experiences using physiological data has a well-established presence in the academic literature. However, this field is currently undergoing transformative changes, driven by the accessibility of diverse and cost-effective devices, innovative software analysis methods, and the emergence of novel application domains such as Virtual and Augmented Reality and mulsemedia. To address the imperative of contextualizing these evolving trends in a contemporary context, this paper presents a systematic review with the objective of delineating the array of physiological data utilized in assessing Quality of Experience (QoE) and User Experience (UX) in multimedia studies. It also examines the devices employed for data collection and the analytical techniques applied to interpret the acquired data. While our review exposes both constraints and promising discoveries in these domains, it also emphasizes the escalating significance and practicality of leveraging physiological data in user assessments, especially as the boundaries between the physical and digital domains continue to blur.

Keywords:

• Physiological data
• biosignals
• user experience
• quality of experience

Acknowledgments

This study was financed partly by the Coordination for the Improvement of Higher Education Personnel (CAPES, Brazil) – Finance Codes 88887.570688/2020-00 and 88881.689984/2022-01, National Council for Scientific and Technological Development (CNPQ, Brazil) – Finance Code 307718/2020-4, and Fundação de Amparo à Pesquisa e Inovação do Espírito Santo (FAPES, Brazil) – Finance Code 2021-GL60J.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 A positive form of stress having a beneficial effect on health, motivation, performance, and emotional well-being.

2 http://www.emotiv.com.

3 A rapid and simultaneous movement of both eyes between two or more phases of fixation in the same direction.

4 https://www-periodicos-capes-gov-br.ezl.periodicos.capes.gov.br/.

5 Vx-WCT is the ECG vector signal measured from the Wilson’s Central Terminal (WCT) voltage to the Vx position.

6 ExG sensors are used to measure electrical activity between a pair of sensors (differential).

7 There are even proposals of integrated systems to detect users’ trust level in AI technologies, and of biofeedback sensors with VR such as EmteqVR (Mavridou et al., 2019).

8 https://www.ericsson.com/en/6g/internet-of-senses.

Additional information

Notes on contributors

Aleph Campos da Silveira

Aleph Campos da Silveira, a PhD student in Computer Science at UFES, Brazil, holds a BSc in Computer Science and Information Systems and an MSc in Education from UFLA, Brazil. His research interests include Usability Evaluation, Multisensory Human-Computer Interaction, Biofeedback, and Games

Mariane Lima de Souza

Mariane Lima de Souza is an Associate Professor at UFES, Brazil, with a Ph.D. in Developmental Psychology from UFRGS, Brazil. Her research focuses on Development and Cognition, specifically cognitive neuroscience and experimental phenomenology

Gheorghita Ghinea

Gheorghita Ghinea is a Professor of Mulsemedia Computing at Brunel University. He holds degrees in Computer Science and Mathematics from the University of the Witwatersrand, South Africa, and a Ph.D. in Computer Science from the University of Reading, U.K. His research focuses on the intersection of computer science, media, and psychology.

Celso Alberto Saibel Santos

Celso Alberto Saibel Santos is a Professor in the Department of Informatics at UFES. He earned his Ph.D. in Informatics from Université Paul Sabatier de Toulouse III, France, and his M.Sc. and B.Sc. in Electrical Engineering from USP and UFES, Brazil. His research includes Multimedia Systems, Computing Systems Engineering, and Multisensory Applications.