Interactive Cognition: An introduction

ABSTRACT

Humans do not passively respond to agents and stimuli; we engage with our environments in reciprocal and interactive fashion. This special issue of Visual Cognition presents a collection of twelve original articles which demonstrate that interactive reciprocity changes the expression of basic cognitions and motor actions as well as resulting overt cognitive and social behaviours. The results from these studies highlight the need for a greater understanding of the mechanisms by which interactive reciprocity influences cognitions. They also showcase the feasibility of addressing research questions involving reciprocity using both the existing classic methods as well as novel approaches. The future of the emerging science of Interactive Cognition is bright!

KEYWORDS:

• Interactive Cognition
• reciprocity
• basic cognitions
• complex behaviour
• social cognition

This article is related to:

Can attitude similarity shape social inhibition of return?

The eye contact smile: The effects of sending and receiving a direct gaze

Look away to listen: the interplay of emotional context and eye contact in video conversations

Interactions are at the core of human behaviour, as we engage with agents and stimuli within our environments daily. Human development, social well-being, and life satisfaction are predicated on seeking and maintaining such interactions. Curiously, the field of cognition has only recently started to examine how cognitive and neural systems support behaviour in interactive situations, like social exchanges (Schilbach et al., 2013), collaborative tasks (Wahn et al., 2020), joint actions (Sebanz & Knoblich, 2021), and music performance (Palmer & Zamm, 2017). The results from this work show that basic cognitions (i.e., attention, memory, communication) operate differently when situations require reciprocal interactions (i.e., with agents or stimuli responding back to observers; Edwards et al., 2015) relative to when they are assessed when observers respond to stimuli passively in spectator-based fashion (Frischen et al., 2007). The main goal of this special issue was to showcase the work in the emerging field of Interactive Cognition while also demonstrating the feasibility of studying cognitions in interactive contexts using both classic and novel experimental and theoretical approaches.

The articles that make up this special issue centre around two main themes.¹ The articles within the first theme Effects of reciprocity on attention, motor actions, and memory show that experimental tasks that present test situations that require reciprocal interactions (either between participants or between participants and experimental stimuli) change the expressions of well-known basic cognitive processes. The articles clustering around the second theme The effects of reciprocity on social and cognitive behavior further demonstrate that experiencing reciprocal interactions within a task affects participants’ functional behavioural outcomes evident in their social and cognitive function and overt behaviour.

The articles in Theme I: Effects on reciprocity on attention, motor actions, and memory (Dalmaso et al., 2021; Dudarev et al., 2022; Edwards et al., 2022; Lanthier et al., 2022; Tufft & Gobel, 2022; Willemse et al., 2022) show that attention, memory, and motor functions are expressed differently when test situations include interactive reciprocity, that is when participants are asked to interact with agents and experimental stimuli. Using a task in which two participants take turns in responding to targets (e.g., Welsh et al., 2005), Tufft and Gobel (2022) and Dalmaso et al. (2021) investigated how reciprocity in responses influenced the social IOR effect. The results from these studies show that the magnitude of social IOR is affected by task context but not by perceptions of similarity between participants. By manipulating the cooperativeness and gender of participants, Tufft and Gobel (2022) demonstrated larger magnitudes of joint IOR when the task was construed as cooperative vs. competitive with males generating larger IOR in cooperative team relative to competitive individual contexts. Interestingly, and pointing to a potential robustness in the mechanisms underlying social IOR, Dalmaso et al. (2021) reported that joint and individual IOR were equally enhanced regardless of how much participants perceived their partner as being similar to them.

Examining the links between behavioural reciprocity and motor actions, Dudarev et al. (2022) investigated how joint responding in a task affected the Simon effect (Dolk et al., 2014), which is marked by faster responses for targets that occur at spatial locations congruent with a response. Demonstrating that participants mentally represent motor actions of their partners, in the interactive version of this task, spatial compatibility is also observed for responses that are congruent with the partner’s response side. Dudarev et al. (2022) examined the purported social nature of the joint Simon effect by comparing the performance of typically developing children and adults with the performance of children with an autism-spectrum disorder (ASD). Their results revealed that despite showing normative joint Simon effect magnitudes, individuals with an ASD represented motor actions of interactive partners later in the processing stream, which is consistent with the notion that representations of other’s actions depend on normative social functionality.

Converging findings relating joint actions and social function were also reported by Willemse et al. (2022), who examined the links between behavioural reciprocity, motor movements, and representation of social context. The authors measured whether the gaze leading effect (e.g., Edwards et al., 2015), or a tendency to make eye contact with someone who has followed our gaze previously, is similarly expressed in mouse tracking responses and whether this behaviour varied with ascribed agency of the interaction partner. The results indicated typical gaze leading effects with mouse motor actions and suggested that those effects were modulated by the perception of interactive partner’s agency. That is, gaze leading was observed only when the robot followed participants responses most of the time relative to when it followed participants responses only sporadically. In line with this result, Edwards et al. (2022), who explored the role of social variables in gaze leading using the typical gaze following paradigm, found that gaze leading was relatively unaffected by socio-contextual factors such as facial dominance of the interactive partner and emotional valence of the interactive object.

Finally, Lanthier et al. (2022) demonstrated the effects of behavioural reciprocity on memory. The authors examined how interactive behaviour via gaze influenced memory for words when interactions occurred in person relative to when they occurred online. Their data indicated that participants had better memory for words when they made eye contact with the experimenter. However, this effect was modulated by social presence, such that memory suffered in the conditions when the eye contact was experienced online relative to when it was experienced live.

Thus, taken together, the studies clustering within Theme I show significant changes in basic cognitive and motor functions when participants are utilizing those abilities in tasks that include interactive and reciprocal responses from human partners or task stimuli.

Extending these results, the studies centring around Theme II: The effects of reciprocity on social and cognitive behavior (Breil & Böckler, 2021; Capozzi & Ristic, 2022; Dawson & Foulsham, 2022; Haensel et al., 2022; Hietanen & Peltola, 2021; Horn et al., 2022) show that in addition to changing basic cognitive and motor functions, reciprocity in responses is also associated with changes in overt behaviour, such as smiling and looking responses, as well as relating to others. Thus, interactive reciprocity also changes the outcomes of cognitive operations by modulating observable behaviour and attitude formation. Utilizing a novel type of shutter display to control stimulus presentation time, Hietanen and Peltola (2021) elegantly showed that the awareness of gaze reciprocity changed muscle activity associated with smiling, such that overt smiles were more prominent when participants perceived direct gaze relative to no gaze (i.e., occluded eyes). Breil and Böckler (2021) extended this notion by showing that experiencing direct gaze with interactive partners affected how participants connected with them on a social and emotional level, such that those individuals who made eye contact with observers were rated as having more empathy and higher trustworthiness. This result varied with emotional context such that deviated gaze was found less detrimental for connecting with others in negative emotional contexts.

Thus, awareness of a reciprocal gaze signal as well as the experience of that signal modulates how we behave towards others and how we connect with them. Further supporting this point and replicating the contextual nature of gaze reciprocity (e.g., Laidlaw et al., 2011), Horn et al. (2022) reported that looking frequency at another person was reduced when a participant waited in a room with a confederate stranger. Interestingly, Horn et al. (2022) also showed that in this context, the potential for social interaction (i.e., high potential while confederate was working on a computer relative to low potential while they were talking on a phone or listening to headphones) did not modulate participants’ looking behaviour, as few fixations to the confederate were observed even when the potential for social interaction was high.

Similar socially avoidant behaviours were classically theorized to reflect cultural preferences. Specifically, individuals from East Asian cultures were thought to generally avoid making direct eye contact with others. This is in contrast to individuals from Western societies, who were thought to prefer direct eye contact as a culturally accepted communicative norm (Argyle & Cook, 1976). Haensel et al. (2022) present one of the first studies that assessed this prevailing notion experimentally. Using a dual head-mounted eye tracking setup, Haensel et al. (2022) measured fixations to partner’s eye regions during live interactions between East Asian (EA) dyads and Western Caucasian (WC) dyads. Contrary to the prevailing notion, EA dyads engaged in mutual gaze more often than WC dyads, highlighting contextual effects in gaze communication both within and across cultures. Haensel et al. (2022) also reported that both dyad groups looked at faces more while listening to their partners versus while speaking.

This important variable was also examined by Dawson and Foulsham (2022), who compared gaze patterns during real-life interactions with gaze patterns observed while the same participants viewed video clips of those interactions. Their results indicated similar eye movement patterns during live interactions and during viewing of recordings of those interactions, with participants frequently fixating the speaker first and then shifting their gaze in response to changes in speakers. Finally, Capozzi and Ristic (2022) presented a taxonomy of gaze behaviours during group interactions. This taxonomy is based on the captured nature of gaze reciprocity and the amount of looks between group members. Three types of gaze dynamics were identified, i.e., Social referencing, Participation, and Mutual gaze (see also Capozzi et al., 2019), with each behaviour found to reliably occur during group live interactions. Importantly, indices of group interactive patterns linked reliably with both overall group and individual function. That is, group members who exhibited higher levels of Social referencing during an interaction (i.e., those who were looked at more by the other group members) were also perceived higher in leadership capacities and their social cues (i.e., gaze direction) elicited greater gaze following responses in fellow group. As such, the studies from Theme II show that reciprocal interactions affect overt social and cognitive behaviours as well as the perceived ability to relate to others.

In sum, the collection of articles in this Special Issue undoubtedly shows that the future of the emerging field of Interactive Cognition is bright. As the research questions and theoretical approaches investigating reciprocity in responses and the resulting cognitive function and behaviour become mainstream, investigators will be able to broaden the scope of research questions and to inquire about how interactive reciprocity influences other cognitive faculties, such as working memory, decision making, or complex actions. They will also be able to study how cognitions operate and are modulated by the complexity of interactive context, in two-person dyads, smaller three or four-person groups, and larger crowds. There is a growing understanding that the complexity of interactive context may place different demands on cognitive functions and change how operations are deployed and utilized (e.g., Will et al., 2021). Finally, integrating novel findings and approaches collected in this Special Issue with neuroimaging investigations will be able to shed more light onto how synchrony in brain function and behaviour is supported and modulated by interactive reciprocity (e.g., Redcay & Schilbach, 2019).

In closing, we would like to thank Visual Cognition editors, James Tanaka and Chris Olivers, Visual Cognition board of editors, and Taylor & Francis for their support. Putting together a special issue featuring interactive responses in a worldwide pandemic in which all research, and especially in-person research, was placed on hold was challenging for all authors, reviewers, and editors. We thus also thank each of the authors for their effort to bring this volume to fruition. Finally, thank you to our numerous colleagues who have provided valuable input and points for discussion for articles in this volume.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Due to the publisher's error, three articles belonging to this special issue have been printed ahead in separate regular Visual Cognition volumes. The links to those articles are provided in the assembled special issue. These articles are: Breil and Böckler (2021), Dalmaso et al. (2021) and Hietanen and Peltola (2021).