Highlights from the 33rd European conference on cognitive ergonomics (ECCE 2022) Evaluating the Reality–Virtuality Continuum

October, 4–7, 2022

Kaiserslautern, Germany

This special issue of the Behaviour & Information Technology journal covers six selected papers from the 33rd European Conference of Cognitive Ergonomics or just for short ECCE 2022 which was held October 4–7, 2022, in Kaiserslautern, Germany. The conference is the annual event of the European Association of Cognitive Ergonomics (EACE), and ECCE is the leading conference in human-media interaction and cognitive ergonomics. It provides an opportunity for both researchers and practitioners to exchange new ideas and practical experiences in a variety of domains.

The special theme of ECCE 2022, Evaluating the Reality–Virtuality Continuum, is dedicated to a variety of research areas and applications that are encompassed by the so-called Reality-Virtuality continuum (Milgram and Kishino 1994; Milgram et al. 1995). These include systems that are used in real-world settings such as applications scaling from smart devices to large industrial setups, as well as Mixed Reality (MR) applications that integrate real and virtual objects and Virtual Reality (VR) applications that immerse users into purely virtual environments. ECCE 2022 features contributions from researchers and practitioners which address the broad spectrum of Cognitive Ergonomics challenges in the analysis, design, and evaluation of virtual and physical interactive systems as part of a rich conference programme.

In this special issue, the selected papers of ECCE2022 address several aspects of the methods, design, use and evaluation of interactive systems within the virtual-reality continuum as well as other interactive media that provide additional methods, results, insights, reflections, as well as societal and sustainable practices for various applications and settings.

Starting with the contributions related to the virtual-reality continuum, four papers highlight various aspects of it. In the paper titled ‘The impact of developmental dyslexia on workplace cognition: Evidence from a virtual reality environment’ James Smith-Spark, Rebecca Gordon, and Ashok Jansari studied how some cognitive difficulties associated with dyslexia would persist into adulthood, and subsequently their impact on employment in office settings. They used a virtual office environment for assessing both executive function and prospective memory, which are two areas of cognition frequently requested in office work. The participants allotted a pre-given ‘To Do’ items list during the assessment that resulted in further tasks or events, similar to the course of actions that often occurs in office settings by navigating in a virtual office environment. It was shown that the group with dyslexia overall performed worse. The group with dyslexia scored lower on the selective thinking and planning measures of executive function. They also performed worse on two of the three prospective memory measures, specifically time-based and event-based prospective memory. The findings indicate how dyslexia can affect workplace cognition, which in the long run could provide targeted support for employees with dyslexia beyond literacy-related skills. The obtained findings also emphasise the value of VR methodologies in testing office-based cognition. Finally, their study offers insights into appropriate support to be in place to help all individuals reach as well as achieve their full potential at work, which is asked for from societal and sustainable perspectives.

Nils O. Beese, René Reinhard, and Thomas Lachmann presented an article titled ‘The right tools for the job: Towards preference and performance considerations in the design of virtual reality interactions’. In their paper, they focused on the importance that usable input devices like controllers play for users to properly interact in VR scenarios. The design of separate input mechanisms may result in differences in the perceived user experiences of the Wands and Index controllers, and it was implicitly assumed that this difference in design might indirectly impact the usability negatively for the more abstract interaction mechanisms because of its reduced natural design in comparison to other controllers. They formulated two hypotheses about user preference and task performance. In this study, Vive Wands and Valve Index controllers were contrasted and compared in three tasks: direct interaction with objects (throwing), tool usage (bow), and indirect control of a character (remote control). The obtained results show differences in preference only for the remote-control task. It should be mentioned, however, that there were no performance differences between the input devices in any of the tasks. Another noticeable aspect was that the more direct interactions with virtual objects, where the Index controller seemed to offer more natural interactions, did not result in more positive evaluations. Hence, no significant differences could be shown between the two different input devices regarding task performance measures.

Nadine Flegel and Tilo Mentler presented an article titled ‘“What do you see now?”: Lessons learned while integrating a virtual reality application in ambulatory care of a pediatric surgery’ where they applied several human-centred design methods while developing and integrating a VR application in ambulatory care of children after bone fractures. Their work focused on the challenges of doing research in everyday clinical practice, in which the VR application must serve both the users like doctors, nurses, as well as patients and their caregivers while simultaneously being effortlessly integrated into the organisation’s work practices at the wand. The study focused on a certain kind of medical treatment which included at least a short, but painful moment. One or several metal wires are usually drilled into the fractured bone during surgery to stabilise it. When pulling out the wire from the bone with a tongue, complementary aspects for a smoother medical treatment were asked for, and especially children were often scared to participate in this kind of treatment in which experiences of discomfort, tension and anxiety were elicited. The authors intended that an appropriate VR application could be used to reduce anxiety and fear before and during the wire removal as well as reduce suffering and pain during the pulling out phase. A design concept for the wire removal in which a VR intervention was developed with the purpose to distract the child during the treatment, and then some wire removals with the developed VR intervention were studied at the wand. The findings revealesd that there were some problems when putting on the VR headset, and some problems while using the VR application. The lessons learned highlight that the VR application has to be properly introduced and known in advance to achieve its intended potential. Hence, the use of the VR application should be considered from a holistic perspective, involving all persons present and its usage should be well-aligned temporally (pre, during and post) with the wire removal actions.

Marja Liinasuo, Timo Kuula, Vladimir Goriachev and Kaj Helin presented the final contribution related to the virtual-reality continuum in their article titled ´’Remote testing of an augmented reality system’ where they presented and discussed two kinds of user test methodologies that focused on usability and user experience aspects of an AR application that was modified to be applied in two different domains. The main focus of their paper was on the appropriateness of the methodologies, particularly about being remote or physically present during the tests, which they problematised and contemplated upon. They primarily conducted remote testing in two separate domains but used a similar AR application, procedure, and methods. One of the domains was space, in which the participants as the ‘astronauts’ received AR-based instructions about how to carry on with several tasks. The other domain was a digital twin of an imagery-building information model, in which the participants should perform several tasks with the model in the AR-based environment. In this case, however, a test instructor needed to be present to grasp when the participants had completed a certain task, based on the participant’s oral utterances, and then offered instructions for the upcoming task to carry out in the test. They evaluated the appropriateness of the test methodologies used that centred on the obtained results, the ease of use as well as the understanding of the test methods, and the test atmosphere. The physical presence of a test instructor who provided technical support about how to handle the AR application as such to the participants was demonstrated to be of major importance for AR-related testing, including being able to make technology-clarifying comments to facilitate data collection and other matters. Finally, they described several pros and cons when facing the situation when one has to select between face-to-face and remote test settings, and they concluded that hybrid testing, as they characterised it, could be a viable approach.

Turning to other interactive systems, Christof van Nimwegen and Emiel van Rijn presented an article titled ‘Time swipes when you’re having fun: Reducing perceived waiting time while making it more enjoyable’ in which they explored and studied the concept of waiting in human–computer interaction. They focused on the experience of time and time estimation, and how to improve the user experience while the user is being in a waiting mode. They clarified that the experience of time is shaped by several factors. They pointed out that multiple design techniques currently are presented to support users to properly manage to wait, but periods longer than 10 seconds are still an under-researched area. They aimed to investigate and analyse if there was a difference in how waiting is experienced between different waiting strategies such as ‘no activity’ (progress bar), ‘passive waiting’ (reading) or ‘active waiting’ (actively doing something) on mobile devices. They presented a novel approach for the latter strategy in which the users should scratch an image which would disappear and the hidden content underneath the surface would be revealed similar to scratching a lottery ticket. They experimented by using a mobile website where the users could book a train journey using their smartphones with the three above conditions to measure the effects on estimated waiting time and enjoyment. The obtained results show that the ‘no activity’ and ‘active waiting’ conditions were estimated as being faster than the ‘passive waiting’ condition. Moreover, the ‘passive’ waiting and ‘active’ waiting conditions were significantly more enjoyable than the ‘no activity’ condition. When combining ‘waiting time estimation’, in which short was better and ‘enjoyment’ in which higher was better, they revealed that the ‘active waiting’ condition generated better results. Hence, it seemed that the situation when users ‘were doing something’, either reading or scratching an image, resulted in higher enjoyment during the waiting time. Indeed, when the authors combined this evidence with the scores on the ‘perceived waiting time’ and the ‘estimated waiting time’, it was revealed that a higher enjoyment does not necessarily result in the experience that the waiting was shorter. They concluded that tentative alternatives for scratching a lottery ticket offer an interesting challenge, especially in additional situations and other interactive devices than mobile phones, concerning equivalent waiting strategies.

Finally, Oronzo Parlangeli, Margherita Bracci, Enrica Marchigiani, Paola Palmitesta, and Stefano Guidi presented an article titled ‘She's better at this, he's better at that. Gender role stereotypes in humanoid robots’ in which they addressed several aspects of gender stereotypes in human-robot interaction from an online survey of eight different robots with various gender aspects of their appearance. Of the eight humanoid robots used, four were perceived as more feminine and four were perceived as more masculine. The robots used in the survey ranged on a continuum from generally being perceived as masculine while the ones on the other side of the continuum were generally perceived as feminine. Robots at the extremities of the continuum had higher human likeness and were more distinctly perceived as either masculine or feminine. Moving to the middle of the continuum, robots in the centre tended to be less human-like and more gender-neutral. The users should assess the humanoid robots’ adequacy to perform eight roles of which four roles were stereotypically masculine and four were stereotypically feminine. Data were collected on possible subjective gender role stereotypes. The participants were also asked to answer how well that role could be performed by a particular robot. Their obtained results show that gender role stereotypes were triggered for both males and females, but they showed that men more strongly responded to those roles that traditionally are related to male ones. They also showed that these stereotypes also were generated in these situations, although prior research has indicated that robots in general were regarded as less appropriate for carrying out female roles. They presented additional findings which show that an increased degree of similarity of robots to humans, i.e., human likeness, demonstrates a positive effect in assessing the appropriateness to perform female roles just for female robots, whereas the other way around does not occur for male robots. The authors pointed out that their results indicate that the categorisation of male and female robots differs. They concluded that robots generally are perceived as being fundamentally male entities, whereas female robots are perceived as modified ‘male exemplars’. They concluded that their work offers a foundation for future work on the gender-sensitive design of humanoid robots, where one also needs to study gender stereotypes about the actual performance of the robots, beyond male or female appearance, although this research provides a significant step in that direction.

To conclude, we hope that the above articles in this special issue have attempted to offer several answers as well as opportunities for new insights and reflections to further enhance our understanding of the emerging Cognitive Ergonomics challenges that we are facing in the Reality-Virtuality continuum and in other media-technology interactions. The selected papers highlight the ongoing development of various kinds of technologies which demands additional theoretical and methodological frameworks, and appropriate research and design approaches to investigate, analyse and critically reflect upon the impact of these technologies on the individual, organisation, and societal levels.

Achim Ebert and Jessica Lindblom

ECCE 2022 Special Issue Editors