Gaze transfer in remote cooperation: Is it always helpful to see what your partner is attending to?

Abstract

Establishing common ground in remote cooperation is challenging because nonverbal means of ambiguity resolution are limited. In such settings, information about a partner's gaze can support cooperative performance, but it is not yet clear whether and to what extent the abundance of information reflected in gaze comes at a cost. Specifically, in tasks that mainly rely on spatial referencing, gaze transfer might be distracting and leave the partner uncertain about the meaning of the gaze cursor. To examine this question, we let pairs of participants perform a joint puzzle task. One partner knew the solution and instructed the other partner's actions by (1) gaze, (2) speech, (3) gaze and speech, or (4) mouse and speech. Based on these instructions, the acting partner moved the pieces under conditions of high or low autonomy. Performance was better when using either gaze or mouse transfer compared to speech alone. However, in contrast to the mouse, gaze transfer induced uncertainty, evidenced in delayed responses to the cursor. Also, participants tried to resolve ambiguities by engaging in more verbal effort, formulating more explicit object descriptions and fewer deictic references. Thus, gaze transfer seems to increase uncertainty and ambiguity, thereby complicating grounding in this spatial referencing task. The results highlight the importance of closely examining task characteristics when considering gaze transfer as a means of support.

Keywords:

• Gaze transfer
• Cooperation
• Pointing
• Ambiguity
• Resolution
• Grounding

Acknowledgments

Parts of this research were presented at the 10th Annual Conference of the Society for Cognitive Science (KogWis 2010, Potsdam, Germany, October 2010) and at the 4th International Conference on Cognitive Science (Tomsk, Russia, June 2010). We thank Daw-An Wu, Evan F. Risko, Cathy Reed, and Cathy Nangini for very helpful comments on an earlier version of the manuscript. Furthermore, we thank Anett Reiche, Josephine Hartwig, Caroline Gottschalk, and Antje Grabowski for their support in data acquisition and Gernot Pascher for his assistance in the technical implementation of the experimental setup. This research was supported by a grant of the TU Dresden Centre for Continuing Education to RM, by the European Commission (FP7-PEOPLE-2009-IEF, EyeLevel 254638) to SP, and by the Russian Foundation for Basic Research (Interdisciplinary oriented research 09-06-12003) to BMV.

Notes

¹ To test for practice effects, we analysed solution times separately for each blocks with regard to the order in the experiment. We found the same pattern of speech leading to slower performance than all other communication conditions, although the difference between speech and gaze & speech missed the significance level in block 4, p = .077. Critically, the cursor transfer conditions did not differ from each other in any block, all p values > .3.

² The first cursor landing does not necessarily indicate pointing, especially during gaze transfer where visual scanning is transmitted as well. However, the expert immediately knew the correct location of a piece when looking at it due to the coordinate marks. Therefore, the need for scanning was minimal after the initial fixation. Moreover, even if scanning differed between gaze and mouse, it should be similar for selecting and moving, so that the relative effect for each communication condition should not be affected.

³ An analysis of the experts' eye movements revealed that they tried to prevent such ambiguities by adapting temporal and spatial parameters of their gaze when using it for communication. They fixated longer and on fewer pieces, and especially tried to keep their gaze still during periods when the novice needed to interpret it. However, controlling one's eye movements can severely impair performance (Ballard, Hayhoe, & Pelz, 1995), which implies that although such strategies may ameliorate some problems of gaze transfer, they can give rise to others.