From gestures to gaming: Visible embodiment of remote actions

Abstract

Teleoperation is the act of controlling an object that exists in a space, real or virtual, physically disconnected from the user. During such situations, it is not uncommon to observe those controlling the remote object exhibiting movement consistent with the behaviour of the remote object. Though this behaviour has no obvious impact on one's control of the remote object, it appears tied to one's intentions, thus, possibly representing an embodied representation of ongoing cognitive processes. In the present investigation, we applied a natural behaviour approach to test this notion, (a) first by identifying the representational basis for the behaviour and (b) by identifying factors that influence the occurrence of the behaviour. Each study involved observing participant behaviour while they played a racing video game. Results revealed that the spontaneous behaviour demonstrated in a teleoperation setting is tied to one's remote actions, rather than local actions or some combination of remote and local actions (Experiment 1). In addition, increasing task demand led to an increase in the occurrence of the spontaneous behaviour (Experiment 2). A third experiment was conducted to rule out the possible confound of greater immersion that tends to accompany greater demand (Experiment 3). The implications of these results not only suggest that spontaneous behaviour observed during teleoperation reflects a form of visible embodiment, sensitive to task demand, but also further emphasizes the utility of natural behaviour approaches for furthering our understanding of the relationship between the body and cognitive processes.

Keywords:

• Teleoperation
• Action
• Natural behaviour
• Embodied cognition
• Cognitive demand

This work was supported by a Natural Sciences and Engineering Research Council of Canada (NSERC) Graduate Fellowship to J.D.C., an NSERC Postdoctoral Fellowship and a Killam Postdoctoral Fellowship to E.F.R., and NSERC operating grants to A.K. We would like to thank Tom Foulsham for the creation and use of VideoCoder, a video annotation application (https://sites.google.com/site/tomfoulshamresearch/research/programming).

Notes

¹ A first-person perspective was chosen to provide participants with a more compelling experience. However, given recent evidence from the field of perspective taking (e.g., Vogeley & Fink, 2003), whether perspective is an important factor in the emergence of spontaneous behaviours is an interesting question for future investigation.

² Analysing these data with nonparametric statistical methods also yielded the same outcomes. This was also the case for the analyses in Experiments 2 and 3.

³ The gender distribution in normal and reversed conditions was uneven—more females were randomly assigned to the reversed mapping condition (7 vs. 4). Although gender played a role in task performance (i.e., trend for males to outperform females, partially accounting for the lap time result), the frequency of the spontaneous behaviour was not influenced by gender (p > .05).

⁴ It is worth noting the similarity in averages for mental demand in Experiments 1 and 2 despite the effect being significant in Experiment 2 and not Experiment 1. A Bayesian analysis (Rouder, Speckman, Sun, Morey, & Iverson, 2009) revealed Bayesian (JZS) factors of 1.00 (Experiment 1) and 0.16 (Experiment 2), which provides no evidence and substantial evidence for a difference in mental demand across conditions, respectively (Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011). Importantly, the comparison of the overall demand measure in both experiments confirms this pattern.

⁵Although a future investigation aimed specifically at examining whether rGDMs are influenced by practice is needed, we performed an analysis to assess whether practice-related effects were present in our data. Comparing the frequency of rGDMs across game sessions for each of the three experiments revealed no reliable practice effects (all ps > .05).