A systematic survey of multiple social robots as a passive- and interactive-social medium

Abstract

This paper presents a systematic survey of the existing research in the literature on multiple social robots, which are a social medium whose behaviours are designed to interact with people in both direct and indirect ways. This medium uses multiple social robots that interact with each other. This paper explored academic databases (IEEE Xplore, ACM, PubMed, Science Direct, Springer, and Google Scholar) to locate publications from the last five years: January 2018 to June 2023. I found 59 papers on social robots with passive- and interactive-social-medium approaches and categorised and summarised them. These works discuss typical research topics and possible future avenues related to the interaction design of multiple social robots.

GRAPHICAL ABSTRACT

KEYWORDS:

• Passive-social medium
• interactive-social medium
• multiple social robots
• robot-robot-interaction
• systematic survey

1. Introduction

The use of multiple robots in various applications is rapidly expanding. Sophisticated designs for robot-robot interactions, such as those in autonomous driving [1,2] and warehouse transport robots [3,4], have fuelled the development of numerous technologies that facilitate safer and more accurate cooperation between robots. These technologies, which largely rely on advanced sensing and network-based communication capabilities, emphasise the importance of precise object recognition [5,6], localisation [7,8], and human motion trajectory prediction technologies [9,10].

Interaction design is becoming even more critical in social interaction where multiple robots collaborate. Past survey papers that focused on multiple agents [11,12], including social robots, reported the effectiveness of using multiple robots as a group of human teams and how the robot’s behaviours influenced the observers’ actions during the interactions. These survey papers summarised interaction styles between people and robots by considering interaction graphs and their relationships, but focused less on behaviours that design concepts between the robotselves. A key aspect of the social interaction of multiple robots is the concept of viewing robots as a medium. This concept, known as the passive-social/interaction medium [13], posits that using multiple robots leads to more effective information by providing a medium (detailed definitions are described in the next section). This concept is based on the understanding that humans, as social beings, are influenced by the number of others in their environment [14–16]. Such social influences are also caused by multiple artificial agents (including robots) on human behaviours and impressions [12,17].

Showing conversation between multiple robots would support the understanding of people who are observing robots, because dialogues are easier to understand than monologue [18,19]. Moreover, expressing social relationships among multiple social robots will increase not only the amount of information but also influence the cognitive aspects. For example, past studies reported that friendly relationships between humans, animals, objects, and robots increased positive impressions toward observers [20–22]. In other words, using multiple robots enables them to manifest their sociality on their own without human interactions.

However, no systematic investigation has concentrated on such effects, particularly in the context of a passive/interactive-social medium. Past surveys on multiple robots have largely focused on non-social aspects, e.g. swarm robot navigation that does not involve humans. While survey papers have focused on multiple agents, most merely highlight representative examples [11] or downplay the concept of the passive/interactive-social medium [12].

This study fills this gap by conducting a systematic review from the perspective of the passive-social/interaction medium and examines a wide range of sources, including academic articles, focusing on studies from 2018 to the present to ensure relevance and topicality.

2. Passive/interactive-social medium

The concept of passive/interactive-social medium using robot(s), based on a previous work [13], is depicted in Figure 1. Sakamoto et al. categorised social media into four types: passive, interactive, passive-social, and interactive-social. A passive medium indicates that one robot offers information as a medium without any interaction between the robot and users, i.e. a one-way information-providing system where people only observe a robot’s behaviour. An interactive medium, on the other hand, indicates that one robot passively offers information like a medium. A key difference in the interaction style between passive and interactive media is that the robots change their verbal/non-verbal behaviours depending on such human behaviours as their locomotion and interruption toward the robot. Therefore, in an interactive medium, a robot’s behaviours during the task change based on human behaviours. In other words, the robot interacts with people in various ways by changing their behaviours by exchanging information, i.e. two-way information-providing. A passive-social medium indicates that two (or more) robots offer information by interacting with each other through verbal/non-verbal behaviours; similar to a passive medium, the robots do not interact with users. A interactive-social medium indicates that two (or more) robots offer information as a medium like a passive-social medium, although the robots interact both with users and other robots [13].

Figure 1. Concept of passive/interactive-social medium based on a previous work [13].

Several past studies [23,24] have compared the effectiveness of a passive-social medium to a passive medium, i.e. concluding that observing the interaction between robots is more effective than observing a single robot’s monologue in an information-providing context. Using multiple robots enables designers to assign different roles to them, allowing for more diverse expressions during interactions. Moreover, this interaction style allows a robot to abandon advanced sensing functions, simplifying robot deployment in the environment.

Similar to a passive-social medium, numerous studies have reported the advantages of an interactive-social medium over an interactive medium in such contexts as information-providing, education, and behaviour changes. These studies highlight the importance of the number of robots. Even though the amount of information is identical between one robot and two robots in a conversation, people preferred and changed their behaviours while interacting with two robots.

Although past surveys reported the effectiveness of both the passive- and interactive-social media [11,12], no systematic surveys related to a passive/interactive-social medium have yet been conducted. Considering the rapid growth of this research topic, a comprehensive survey will provide useful knowledge for researchers working on interaction designs for multiple social robots.

3. Methodology

This paper systematically reviewed six databases, which were chosen due to their extensive coverage of the field of Human-Robot Interaction (HRI) and related disciplines: IEEE Xplore, ACM, Springer, Science Direct, PubMed, and Google Scholar. The search strategy chose the following keywords: ‘robot-robot interaction,’ ‘multiple social robots,’ ‘Passive social medium AND social robots,’ ‘Passive social media AND social robots,’ ‘Interactive-social medium AND social robots,’ ‘Interactive social media AND social robots,’ and ‘two (three, four … ten) social robots.’ These keywords captured a broad range of studies that focus on the interaction designs of multiple social robots. The target period ranged from January 2018 to June 2023. This study focused on the last five years due to several critical technological advancements, such as learning models, sensor systems, and the construction of cost-effective robots, enabling researchers to simultaneously use multiple robots.

Studies for inclusion in the review were selected based on the following criteria: (1) those aimed at using multiple social robots and (2) those aimed at designing interactions between robots. I excluded studies when the authors used each robot separately (e.g. employing multiple robots but not simultaneously in evaluations) and interaction designs under a non-social context (e.g. a navigation algorithm in swarm robots without any people in the environments) because this review is focused on interaction design for multiple social robots working together.

The survey identified 770 documents (Figure 2). I removed articles that were repeated in more than one database, a step that reduced the number of documents to 582. Then I only selected documents based on the selection criteria described above by reading abstracts and full texts, winnowing the number to 60, including 5 additional articles from other sources. The details of each paper will be explained in the following section.

Figure 2. Flow diagram used in literature review in this study.

4. Results

4.1. Summary

4.1.1. Number of studies with passive/interactive-social-medium approaches

The amount of published articles found from 2018 is shown in Figure 3, based on the types of interaction styles between robots.

Figure 3. Number of studies with passive/interactive-social-medium approaches.

In 2018 four papers addressed passive-social media, a number that dropped to two in 2019 and increased to six in 2020. The number dropped to three in 2021 and rose to five in 2022. As of June 2023, the number was three, which will likely grow by the end of the year. The general trend shows an increase around 2020, followed by slight decreases in 2021 and 2022. The COVID-19 pandemic seems a likely culprit for the recent research decreases because passive-social medium evaluations with multiple social robots can be conducted by web surveys whose popularity increased due to the difficulties of face-to-face experiment settings during the pandemic. The requirements of experiment time in such an approach are relatively quick, perhaps related to condensing the duration from experiment to publication. Although fears related to COVID-19 have subsided, the number of studies with passive-social approaches has remained rather steady.

Two papers addressing interactive-social media were published in 2018. That increased to six in 2019 and to ten in 2020. The passive-social-medium approach displayed a similar trajectory: slight decreases in 2021 and 2022. In the first six months of 2023, the number rose to six, suggesting a similar trend by the interactive-social medium's activity from 2020. The COVID-19 pandemic may again have caused this decrease. The number of studies in 2021 and 2022 decreased due to the difficulties of conducting face-to-face experiments and field trials, although its momentum has returned in 2023 as COVID-19’s impact wanes.

4.1.2. Robot-control settings, environments and number of robots

In the appendix, Tables 1 and 2 summarise the studies with passive- and interactive-social medium, including the above information.

The characteristics of the surveyed studies from the viewpoints of robot-control approaches (autonomous and semi-autonomous) and their environments (video, lab, and field), are shown in Figure 4. Note that since some studies conducted video-based surveys in laboratory environments, the ‘video’ label includes both web- and video-based surveys in laboratories. The studies with real robots, i.e. if robots were physically presented to participants, are labelled ‘lab.’ All the studies that employed a passive-social-medium approach used fully autonomous robot systems. Therefore, their behaviours do not need any support from human operators. On the other hand, many researchers employed human operators to control/support robot systems with a Wizard of OZ approach for natural interaction between robots and people.

Figure 4. Characteristics of studies: robot control and environments.

Nine studies conducted field trials in the interactive-social-medium approach, nearly double the passive-social-medium approach. In field trials, attracting people in environments for service robots to provide services is important. Interactive-social-medium approaches enable robots to directly interact with people, an appropriate method compared to passive-social-medium approaches for providing services.

The number of robots used in each study is shown in Figure 5. Note that this value is the maximum number of robots that interact together. Most studies with a passive-social-medium approach used two robots, and a few used three robots to express multi-party conversation settings. In the interactive-social-medium approach, researchers also primarily used two robots, although the variety in the number is relatively diverse. In these studies, researchers focused on the effects of different numbers of robot compared to using two robots or simply employed multiple robots to provide richer services.

Figure 5. Number of robots in surveyed studies.

4.2. Passive-social medium

4.2.1. Exhibitions in real environments

This category includes studies that focused on exhibitions in real environments using a passive-social medium. Using a passive-social medium as an exhibition is a common approach because it enables robots to avoid difficulties caused by direct interaction with people in real environments. For example, Petrović et al. [25] developed a system with which multiple social robots interacted to create robotics co-actors with human actors. Nikolic et al. [26] created two robots that can discuss philosophical topics at exhibitions. They used a recurrent neural network to trigger a human-less creative process by interaction and a serial neck mechanism to express different emotions using non-humanoid robots. For this purpose, they used a recurrent neural network to trigger a human-less creative process by interaction and a serial neck mechanism to express different emotions using non-humanoid robots [27]. Krzyżaniak [28] developed a musical robot platform that enables social robots to collaborate in the context of musical interaction. The work developed a simple responsive rhythm synthesiser and analyzed the equilibria that arise when multiple robots play music together. Nijholt [29] investigated how a passive-social medium is useful to express various interaction styles in the context of stand-up comedy. The work surveyed three kinds of trials that tried to make people laugh by interaction among multiple social robots and discussed humour in the context of human-robot interaction. Swaminathan et al. [30] also focused on robot comedy performance using multiple social robots and conducted a street-style performance to gather audience responses.

4.2.2. Supporting children

This category includes studies where the participants are mainly children or those whose systems aim to support children instead of adults. Similar to exhibitions in real environments, the researchers employed a passive-social-medium approach to support children by showing interaction among multiple social robots. For example, Brink et al. [31] investigated whether 3-year-olds can learn the names of novel objects from a pair of social robots or inanimate machines, i.e. investigating the capabilities of a passive-social medium using different kinds of robots by video stimuli. They reported that although children can learn more from social robots (and trust them) compared to inanimate machines, they trust their information more when the robots appear to have mindful agency. So et al. [32] investigated the effects of a passive-social medium with multiple social robots to promote responses to the joint attention abilities of low-functioning autistic children. They described how a robot drama provided support for children over time and argued that the children’s abilities improved after observing such robot dramas. Peng et al. [33] designed a simulated multi-robot theatre with three similarly-shaped robots and conducted an experiment with children who watched a video and retold its story. Through observing interactions between children and robots, they concluded that contextual behaviours with emotional expressions by robot performances can be understood by children over six years old.

4.2.3. Relationships between robots and people

This category includes studies that focused on investigating how relationships between robots influence observers. Researchers used a passive-social-medium approach to express different relationships between robots (and people) to investigate how showing robot-robot interaction influenced the perceived human impressions of the robots. Xu et al. [34] investigated the psychological effects of the behaviours of multiple robots, as such actions are related to perceptions of social rejection. They conducted a video-based study where multiple robots showed rejection behaviours toward a human experimenter and reported that a sense of rejection rises when robotic groups are less cohesive. Ueno et al. [35] investigated how interaction between robots influences the perceived impressions of observers. Their experiment compared two situations where a robot named NAO interacts/doesn’t interact with another robot called Roomba with verbal/non-verbal behaviour. Observers positively evaluated the interaction of the robots. Thus, they reported that a passive-social-medium approach increased positive impressions of the robots.

4.2.4. Verbal expression design

This category includes studies where researchers employed a passive-social-medium approach to investigate the effects of verbal expression designs between robots, i.e. focusing on what factors in verbal interaction influenced the perceived impressions. Singh et al. [36] increased the comprehension of the observers of verbal expressions in interactions among multiple social robots by implementing conversational functions for a group of robots based on Grice’s maxim of quantity. They conducted a video-based survey and reported that their implemented system led to the highest amount of understanding of the robots’ actions by participants (Figure 6(a)). Velentza et al. [37] investigated the effects of conversational styles among multiple social robots by preparing videos with different storytelling styles (serious, cheerful, friendly, and expressive movements) and conversational interaction between robots. Their participants preferred the one who originally told them the story and disliked the collaboration with a robot that had an extremely friendly attitude and storytelling style, indicating the importance of appropriate verbal expression design in interaction between robots (Figure 6(b)).

Figure 6. Robots with passive-social-medium approaches: (a) Pepper robots talk together [36] (b) Two Nao robots in a discussion [37].

Some researchers focused on the effects of verbal expressions between robots under specific situations. Itahara et al. [38] focused on discussions between robots and different opinions and investigated how such discussions changed the opinions of observers. They built a web-based survey around two patterns of opinion changes to a different side and two patterns of opinion reaffirmation. They argued that observing opinion changes from the positive side (i.e. negative–positive) or positive opinion reaffirmation (positive–positive) effectively provided positive and fair impressions. Observing an opinion change that became negative effectively provided negative and fair impressions, although negative opinion reaffirmation (negative–negative) led to significantly less trust in media. Karatas et al. [39,40] focused on situations where participants overheard conversations among three driving agents under driving situations and investigated how the drivers (i.e. participants) were affected by observing conversations among the three driving agents. They developed a social interface named NAMIDA, which provides location-based information from conversations with each other, i.e. drivers gain such information by basically eavesdropping on their conversations. They reported that the passive-social-medium approach more effectively decreased the workload than the direct interaction approach between the drivers and the robots.

4.2.5. Non-verbal expression design

Non-verbal expression design in interaction between robots is another active research topic in studies that focus on the passive-social-medium approach, particularly gestures in conversational interaction. This category includes studies that focused on the effects of non-verbal expressions represented by multiple robots, e.g. the robot-number effects in similar motions and non-verbal interaction between robots.

For example, Mizumaru et al. [41] investigated the perception of emotional relationships through body expressions between multiple social robots using video stimuli. They applied different combinations of four characteristic body emotion expressions (sadness, fear, pride, and happiness) and described how the emotional movement of each robot’s body influenced the relationships between the two robots and interpreted them using the valence-arousal model. Wicke et al. [42] developed a storytelling system using multiple social robots by focusing on pantomime and naturalistic gestures to enhance the storytelling contents. They conducted a web-based survey with videos, i.e. a passive-social-medium approach, and evaluated their developed system using spatial movements in embodied performances.

Some researchers focused on the physical interaction between robots, e.g. touching behaviours. Hirayama et al. [43] investigated the effects of motion parameters when a robot touches another robot. They prepared multiple videos with different touch behaviours using multiple social robots in a web-based survey, identified a speed boundary between patting and slapping behaviours, and described the appropriate speed parameters to express positive/negative relationships between robots. Okada et al. [44] experimentally investigated the perceived impressions of the conversational content of the effects of touching and whispering behaviours between robots and described the effectiveness of both between robots in information-providing tasks.

A few studies focused on the effects of non-verbal behaviours under specific situations, e.g. gaming, guiding, and apologising. LC et al. [45] focused on non-verbal emotional expressions when multiple arm robots played a game. They developed a multiple robot-control system and evaluated how a robot’s behaviour changed the perceptions of emotional impressions in game settings by video stimuli, concluding that such knowledge of arm-type robots is useful for the behaviour design for non-humanoid robots. Kondaxakis et al. [46] proposed appropriate pointing gestures to align symbols in robot-robot interaction. Their main aim is not evaluations from human observers but achieving non-verbal interaction between robots for designing and automating a passive-social medium with multiple social robots. They conducted simulation experiments to evaluate the effectiveness of their system and demonstrated with a heterogeneous two-robot system the practical viability of this approach. Okada et al. [47] investigated the effects of the number of robots in apology settings. They prepared video stimuli where a robot apologised for its mistake in a cafeteria setting and investigated such actions with an additional robot. They conducted a web-based survey using videos and concluded that apologies from multiple robots are more acceptable than from a single robot.

4.3. Interactive-social medium

4.3.1. Field trials for customer services and analysis

Several researchers employed an interactive-social-medium approach for field trials where multiple robots provided services to customers in real environments. This category includes studies that focused on such trials in real environments. Barbareschi et al. [48] reported a parallel tele-operation system that enabled disabled workers to control multiple robots in a cafeteria, where an operator controlled multiple robots that provided customer services. Moreover, since these controlled robots interacted with another robot managed by a different operator, they functioned as an interactive-social medium for customers. Using multiple social robots controlled by an operator at a bakery, Song et al. [49] also conducted recommendation services based on collaboration with multiple robots and showed the effectiveness of collaborating between robots. Iwamoto et al. [50] investigated the effectiveness of a playful recommendation system where robots stimulated pleasant feelings using multiple social robots with an interactive-social-medium approach by showing conversations between the robots as well as the effectiveness of self-recommendation robots (Figure 7(a)). From another perspective, Kamino et al. [51] conducted an ethnographic observational study in a cafeteria and reported such interaction between people and multiple robots as pet-type robots.

Figure 7. Robots with interactive-social approach at real environments: (a) Playful recommendation from five robots [50] (b) Eight Sota robots [54].

Other researchers used multiple social robots in real environments as a reception service. Nakanishi et al. [52] developed an interactive-social medium using multiple robots that engage in friendly interactions with hotel customers. They deployed their system in a hotel's public area and collected customers’ impressions. The perceived impressions of the robots were influenced by customer gender and interaction durations. Aizawa et al. [53] focused on interaction design to encourage visitors to use guiding robots in public spaces and investigated the effects of different factors, such as the numbers of robots and their non-verbal behaviours. Amada et al. [54] investigated pseudo-crowd effects using different numbers of multiple social robots to attract the interest of passersby in advertising contexts (Figure 7(b)). They compared the effects of the number of robots and their verbal/non-verbal behaviours for attracting visitors who approached the robots and described the importance of speaking the guidance information and looking in various directions.

4.3.2. Supporting children

Similar to a passive-social medium, an interactive-social medium can support children. This category includes studies that focused on supporting children with an interactive-social medium. Compared to a passive-social medium, an advantage of the interactive-social-medium approach is its direct interaction with children, a situation that may stimulate their curiosity. Some researchers focused on the effectiveness of using multiple social robots to support ASD/ADHD children. Efthymiou et al. [55] developed an integrated robotic system (ChildBot) that can participate in and perform a wide range of educational and entertainment tasks. Their system enables multiple robots to actively interact with children, and their user experience study showed that children enjoyed playing with different robots. Soleiman et al. [56] also developed a robot system consisting of multiple social robots for stimulating social environments for children. Their case study showed how their system helps children with autism and reported that it improved emotion recognition skills through interaction with robots. Esfandbod et al. [57] investigated the benefits of using a multiple robot system toward educational interventions for enhancing children’s engagement, attention, and retaining novel words. Lytridis et al. [58] also investigated the effectiveness of an interactive-social medium using multiple social robots to facilitate and enhance interventions in special education. They used multiple heterogeneous robots in ASD interventions and conducted an experiment with children, concluding that they demonstrated a high engagement level and an eagerness to participate in activities through interaction with an interactive-social medium. Amanatiadis et al. [59] investigated the effectiveness of interaction between an interactive-social medium and multiple children with autism, i.e. group interaction between robot and children. They investigated whether children played with other children to assess the benefits of a more naturalistic and interactive type of therapy. They concluded that interaction with multiple social robots indicates positive effects in participants’ communication and interaction skills, joint attention, and cognitive flexibility.

Other researchers focused on supporting typically developing children. Tamura et al. [60] created an interactive storytelling system for children using multiple social robots. They assigned different roles to a robot, a reader, and a listener to attract children’s interest by conversations between social robots and reported that storytelling with multiple social robots attracted more children than just one robot. Alemi et al. [61] also described a case study’s results using multiple social robots controlled by an operator. Although their study did not report any advantages of using multiple social robots, they did describe their effectiveness in the context of education settings.

4.3.3. Collaborative behaviour design

Because an interactive-social medium involves people linked through robot-robot interaction, the behaviour design of multiple social robots and such system designs is becoming more complex. This category includes studies that focused on how to design collaborative behaviour between robots through both system developments and design workshops. Tan et al. [62] created a system that collaborates with two social robots (reception and mobile) to investigate the effects of different robot-robot interaction. They compared different communication strategies between robots and suggested the possibility of instilling socialness to improve the likability of a functional robot by having a social robot interact with it. They also conducted three design workshops to gather ideas on how multiple robots can work together to provide services and offered several guidelines and open questions as an outcome of the workshops [63]. Moreover, they investigated different strategies concerning how a mobile robot might join an existing multi-modal interaction between a person and a stationary robot and reported that an improper strategy, i.e. where a mobile robot is standing too far away from a person and a stationary robot, repositioned the robot self to decrease the distances between interactants [64]. Correia et al. [65] developed a platform for playing a digital game that involves a social dilemma between a mixed team of humans and robots. They did not conduct any evaluations of their system, although their proposed architecture for playing games with multiple social robots might provide useful knowledge for researchers. Other studies focused on the effectiveness of the number of robots in specific situations, such as providing social rewards [66] and mealtime conversations [67], and both reported that two robots are better than just one.

4.3.4. Relationships between robots and people

Researchers have also used an interactive-social-medium approach to express more complex relationships between robots and people and delve into how the relationship of robot-robot-human interactions influenced the perceived impressions. This category includes studies that focused on the relationships between robots and people in interactions. Erel et al. [68] investigated how a robot-robot-human interaction, i.e. an interactive-social medium, can lead to ostracism using non-humanoid social robots with different attitudes toward the participants. They also investigated how the social experiences of exclusion or inclusion with two non-humanoid robots shaped participants’ interactions with others [69]. They concluded that such experiences may produce carryover effects that extend beyond the interaction with robots, impacting interaction with others. Söderlund [70] investigated how robot-robot interaction provides impressions of warmth toward interlocutors. They conducted an experiment with Wizard of OZ settings to control two different robots, where a service robot displayed warmth toward another service robot during interactions with participants. Their experiment results showed that the service robot's high level of warmth boosted the participants’ overall evaluations.

4.3.5. Verbal expression design

Similar to studies with a passive-social-medium approach, researchers have investigated the effects of verbal expression design between robots with the interactive-social-medium approach. Iio et al. [71] proposed an approach that conceals incoherence using a double-meaning agreement among multiple robots and reported that the two robots using it produced better feelings of being understood than those who talked with one robot. They also focused on the effectiveness of such conversational robots against speech recognition failure in a conversation with seniors by comparing the number of robots [72] (Figure 8(a)). Arimoto et al. [73] also investigated effective conversational patterns for concealing incoherent responses using multiple social robots to investigate the effects of the number of robots.

Figure 8. Conversational interaction with multiple robots: (a) Senior and two robots [72] (b) Conversation with two androids [77].

Samson et al. [74] investigated the effectiveness of different two-party conversational voice guidance in driving situations and described how such conversations allowed drivers to better reflect on their choices after finishing driving tasks. Goto et al. [75] assigned different roles to multiple social robots and investigated how humans changed their decisions and behaviours in a moral dilemma environment through interaction with multiple robots that played different roles. Velentza et al. [76] also investigated the combination effects of different roles between two museum guide robots and reported that people remember more information when they are guided by two cheerful robots than by two serious ones.

Nishio et al. [77] developed an effective information medium using two android robots and compared the effectiveness of a passive-social-medium condition with an interactive-social medium (they called it ‘semi-passive,’ although in this condition the robots interacted with the participants) and argued that participants recalled more content from the conversations and felt more empathy for the robots (Figure 8(b)).

4.3.6. Non-verbal expression design

Non-verbal behaviour design is also essential for natural interaction between a passive-social medium and interacting people. Golcic et al. [78] focused on social movements for different types of robots. They gathered social movement data using depth sensors and applied these movements to robot systems to investigate how such movements increase the interactivity of robots. They also used their system with humanoid robots and reported that participants had positive reactions to their interactions with them. Fraune et al. [79] investigated how robot behaviour toward humans and other robots affected interactions using minimally social robots. They conducted a video-based survey and compared the cultural differences between Japan and the U.S. as well as a laboratory experiment where American participants did simple tasks with two robots that behaved differently toward them. The robot behaviour toward social robots instead of functional robots increased the anthropomorphism of the robots.

Some researchers focused on behaviour design during conversations between robots and people, such as gazing behaviour. Eshed et al. [80] used an interactive-social medium to deepen understanding of the psychological determinants of behaviour in a novel social interaction. In this study, the participants non-verbally interacted with three social robots, and the work analyzed how the gaze behaviours of the former changed during interactions. Based on their gaze behaviour analysis, they concluded that information-gathering behaviour is initially predicted by psychological inflexibility and subsequently by curiosity toward the interaction’s conclusion. Oertel et al. [81] investigated effective listening behaviours for multiple social robots that participate in multi-party interactions as an interactive-social medium. They analyzed listener behaviour in human-human, multi-party interaction and implemented an attentive listening system that generates multi-modal listening behaviour for social robots. They experimentally showed the advantages of their system in multi-party interactions among multiple humans and robots.

4.3.7. Persuasive robotics

One unique approach for using an interactive-social medium is for persuasive interaction, such as peer pressure and conformity; this category includes studies related to this research topic. Shiomi et al. [82] investigated the effects of peer pressure from multiple social robots. They conducted experiments with two, four, and six robots to investigate how people’s behaviour changed due to social pressure from the robots and reported that six robots had stronger persuasive powers than the other amounts of them. Salomons et al. [83] also investigated the effects of peer pressure from multiple social robots and how trust affects conformity. They experimented with three social robots and concluded that groups of robots led people to conform when they trusted the robots; losing trust caused such conforming to stop. Hashemian et al. [84] also investigated the persuasive effects of an interactive-social medium with two social robots. They compared two types of persuasive strategies based on social power, reward, and expertise in a situation where two robots attempted to persuade a user to make a concrete choice. Both were similarly persuasive, although the perceived competence and warmth were different between the strategies.

5. Discussion

5.1. Advantages and disadvantages of passive- and interactive-social medium

In this subsection, I discuss the advantages and disadvantage of the passive- and interactive-social media in the development of multiple robot systems. The typical advantage of a passive-social medium is the simplicity of the behaviour designs between robots, which do not consider direct conversational interaction with observers. For example, robots may need to change such non-verbal behaviours as gaze direction toward visitors during conversational interaction between robots, although they do not need to consider changing conversational flows due to interruptions from observers. This advantage simplifies developing robot content and lowers the burden on the sensor system to capture people’s information. Ironically, this advantage can be a disadvantage in a passive-social medium, because it cannot conduct deeper interaction with observers due to such limited interaction capabilities as sensing information and fixed conversational interaction. Overall, the passive-social medium is suitable for developing robot-based content that needs no consideration of interaction with others, such as exhibitions, commercial, and one-way information-providing services.

The typical advantage of an interactive-social medium is rich interaction styles with people, allowing interruptions in robot-robot interaction. For example, when two robots discuss specific content, visitors can interrupt with questions. Moreover, by using multiple robots in conversations, robots can conceal incoherence and provide a better atmosphere than a single robot [71–73]. However, this advantage is directly related to a disadvantage of the interactive-social medium: the need for a complex system to allow interruptions of multiple robot conversations. Unlike a conversation with a single robot, managing the content of multiple robot conversations for interruption is more complex and complicates avoiding the corruption of interaction with people. For this purpose, a rich sensing system is also needed. Overall, the interactive-social medium is suitable for developing robot-based content that provides rich interaction experiences for people through two-way information-providing tasks.

Note that the effect of the number of robots is a common discussion topic in both passive- and interactive-social media. Past studies have already shown the effectiveness of using multiple robots in various contexts, such as effective information-providing, attracting people’s attention, and better attitudes of robots. [13,22,54]. Therefore, although using multiple robots increases the costs and burdens for preparing such content as verbal/non-verbal behaviour designs of robots, this approach offers sufficient merit for developing social robot services.

5.2. Number as a non-verbal modality

The passive/interactive-social-medium approach can be regarded as a way for robots to use their numbers as a new non-verbal modality in social interactions with people. Robotics researchers have already used several non-verbal modalities for social robots, including facial expressions, whole-body gestures, voice characteristics, approaching trajectories, etc. However, the amount of information expressed by such modalities is limited by physical and time constraints. Using more robots will probably increase the amount of information that a robot system can represent as a whole. In other words, using multiple robots avoids a limitation on the amount of non-verbal information per unit time expressed by a single robot.

Related to this perspective, investigating robot numbers complicates behaviour designs for them. For example, robotics researchers need to design timing and consistent behaviour among facial expression, voice characteristics, and gestures for creating multi-modal behaviours like emotional expressions. Inappropriate behaviours and interaction designs among multiple robots may foment negative impressions from the interacting people. Furthermore, since the appropriate modality for conveying information and its design will differ by particular tasks, the appropriate number of robots must also be changed depending on the situations. Although most of the papers examined in this study compared one and two robots, comparisons with a larger number of robots will lead to appropriate interaction designs for multiple social robots.

5.3. Challenges and direction for future work

Due to the decreasing impact of COVID-19, studies with interactive-social media are once again increasing. Because robotics companies have been employing multiple robots to provide services in real environments, these robots need to interact and collaborate in social contexts, e.g. greetings between robots with (non-)verbal behaviours. Although such behaviours are unnecessary for communication between robots, manifesting social relationships between robots conveys positive feelings to the people around them. Therefore, an interaction design based on a passive-social-medium approach among multiple social robots is essential for achieving a socially acceptable existence while multiple social robots work together in real environments.

The interactive-social-medium approach is also crucial for raising the quality of the services provided by multiple social robots. When they work together in such actual environments as cafeterias, not every robot will always be as fully operational as humans. In such a situation, robots without specific tasks can help another robot, not only with physical tasks but also with such social tasks as joining conversations, providing additional information, showing hospitality, etc. Based on these contexts, incorporating a passive/interactive-social-medium approach for the behaviour design among multiple robots is an essential future work for robotics companies that use multiple robots in their services.

One future work from a research perspective is achieving multiple social touch interactions in an interactive-social-medium approach. Although many kinds of conversational interactions have been investigated with non-verbal behaviour designs, such as touch interaction, which is an essential modality in human-human interaction, such approaches remain understudied in the context of an interactive-social medium. Past studies have shown that the number of robots influences changes in people's behaviours, not the amount of information in verbal modalities. Similar to this perspective, investigating whether the amount of information in a touch modality (e.g. comparing two touches from a single robot and a single touch from two robots) is effective for behaviour change is an interesting future work from an interactive-social-medium approach.

6. Conclusion

This study surveyed the recent research activities related to interaction designs for multiple social robots under social contexts in the field of human-robot interaction. Robotics researchers simultaneously use multiple social robots based on passive/interactive-social-medium approaches for rich interaction with people. Researchers have employed both approaches to investigate how multiple social robots influence people’s behaviours and impressions through both direct and indirect interactions between groups of social robots and people. Using multiple social robots is a promising approach to extend the capabilities of social interaction for such robots by expressing the social relationships between them and enabling them to use a new non-verbal modality of numbers.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by JST Moonshot R&D Grant Number JPMJMS2011.

Notes on contributors

Masahiro Shiomi

Masahiro Shiomi received M. Eng. and Ph.D. degrees in engineering from Osaka University in 2004 and 2007, respectively. During that same time, he was an intern researcher at the Intelligent Robotics and Communication Laboratories (IRC). He is currently a group leader in the Agent Interaction Design department at the Interaction Science Laboratories (ISL) and the Advanced Telecommunications Research Institute International (ATR). His research interests include human-robot interaction, social touch, robotics for childcare, networked robots, and field trials.

Appendix

Table A1. Summary of studies with passive-social-medium approach.

Author(s)	#	Year	Environment	Robot control	Max. number
Petrović et al.	[25]	2019	Field	Autonomous	2
Nikolić et al.	[26]	2020	Field	Autonomous	2
Nikolić et al.	[27]	2022	Lab	Autonomous	2
Krzyżaniak et al.	[28]	2021	Lab	Autonomous	2
Nijholt	[29]	2018	Field	Autonomous	3
Swaminathan et al.	[30]	2020	Field	Autonomous	2
Brink et al.	[31]	2020	Video	Autonomous	2
So et al.	[32]	2023	Lab	Autonomous	2
Peng et al.	[33]	2020	Lab	Autonomous	3
Xu et al.	[34]	2022	Lab	Autonomous	3
Ueno et al.	[35]	2019	Lab	Autonomous	2
Singh et al.	[36]	2021	Video	Autonomous	3
Velentza et al.	[37]	2021	Video	Autonomous	3
Itahara et al.	[38]	2023	Video	Autonomous	2
Karatas et al.	[39]	2018	Lab	Autonomous	3
Karatas et al.	[40]	2018	Lab	Autonomous	3
Mizumaru et al.	[41]	2022	Video	Autonomous	2
Wicke et al.	[42]	2020	Video	Autonomous	2
Hirayama et al.	[43]	2022	Video	Autonomous	2
Okada et al.	[44]	2020	Lab	Autonomous	2
LC et al.	[45]	2022	Video	Autonomous	2
Kondaxakis et al.	[46]	2018	Lab	Autonomous	2
Okada et al.	[47]	2023	Video	Autonomous	2

Table A2. Summary of studies with interactive-social-medium approaches.

Author(s)	#	Year	Environment	Robot control	Max. number
Barbareschi et al.	[48]	2023	Field	Semi-autonomous	2
Song et al.	[49]	2023	Field	Semi-autonomous	2
Iwamoto et al.	[50]	2022	Field	Autonomous	5
Kamino et al.	[51]	2023	Field	Semi-autonomous	2
Nakanishi et al.	[52]	2020	Field	Autonomous	2
Aizawa et al.	[53]	2021	Field	Semi-autonomous	2
Amada et al.	[54]	2023	Field	Autonomous	8
Efthymiou et al.	[55]	2022	Lab	Autonomous	3
Soleiman et al.	[56]	2023	Lab	Semi-autonomous	2
Esfandbod et al.	[57]	2023	Lab	Autonomous	2
Lytridis et al.	[58]	2020	Lab	Autonomous	2
Amanatiadis et al.	[59]	2020	Lab	Autonomous	2
Tamura et al.	[60]	2021	Lab	Semi-autonomous	2
Alemi et al.	[61]	2020	Field	Semi-autonomous	2
Tan et al.	[62]	2020	Lab	Autonomous	2
Tan et al.	[63]	2020	Lab	Autonomous	2
Tan et al.	[64]	2022	Lab	Autonomous	2
Correia et al.	[65]	2019	Lab	Autonomous	2
Shiomi et al.	[66]	2020	Lab	Autonomous	2
Fujii et al.	[67]	2021	Lab	Semi-autonomous	2
Erel et al.	[68]	2021	Lab	Autonomous	2
Erel et al.	[69]	2022	Lab	Autonomous	2
Söderlund	[70]	2021	Lab	Autonomous	2
Iio et al.	[71]	2021	Lab	Autonomous	2
Iio et al.	[72]	2020	Field	Autonomous	2
Arimoto et al.	[73]	2018	Lab	Semi-autonomous	2
Samson et al.	[74]	2020	Lab	Semi-autonomous	2
Goto et al.	[75]	2022	Lab	Autonomous	2
Velentza et al.	[76]	2019	Lab	Semi-Autonomous	2
Nishio et al.	[77]	2019	Lab	Semi-autonomous	2
Golcic et al.	[78]	2019	Lab	Autonomous	2
Fraune et al.	[79]	2020	Lab	Semi-autonomous	3
Eshed et al.	[80]	2021	Lab	Autonomous	4
Oertel et al.	[81]	2021	Lab	Semi-autonomous	2
Shiomi et al.	[82]	2019	Lab	Semi-autonomous	6
Salomons et al.	[83]	2018	Lab	Autonomous	3
Hashemian et al.	[84]	2019	Lab	Semi-autonomous	2