Sharing biosignals: An analysis of the experiential and communication properties of interpersonal psychophysiology

KEYWORDS:

• Psychophysiology
• computer-mediated communication
• biosignals
• media psychology

1. Introduction

What is it like to hear another person’s heartbeat during communication? How would direct access to a person’s skin conductance influence our sense of interpersonal intimacy or trust? Can biosignals add value in disambiguating or enriching online communication? Would shared, and potentially synchronized, interpersonal biosignals lead to a higher sense of mutual understanding or empathy? These are examples of the kinds of questions that become relevant and urgent as intimate technologies enter our daily lives and physio-social communications are becoming a reality (Baym, 2015; Janssen et al., 2010, 2014; Van Est et al., 2014).

Biosensing – measuring one’s physiological activity – has seen a great rise in popularity over the past decades. This is mostly due to the fast technological development of biosensing devices, which has made them accessible to a wider audience and has expanded the range of possible applications. We now have the ability to measure our physiology in an unobtrusive way, giving access to information on physiological measures such as heart rate, skin conductance, and respiration, collectively referred to as biosignals. These biosignals provide a new source of information that create novel application opportunities in our day-to-day interactions and interpersonal communication. In this paper, we will consider biosignals as interpersonal communication cues. We will review the current state-of-the art in social biosensing research, drawing on established work in communication and media psychology to develop and deploy a framework to assess the communication properties of biosignal sharing. This framework allows us to eventually structure the design space of the large variety of biosignal communication devices. We will analyze the published literature from two complementary perspectives to elucidate our understanding of the communication characteristics and individuals’ experiences of social biofeedback devices, and the kinds of ethical considerations and research questions that need to be addressed in future research.

1.1. The intimate and social-emotional nature of biosignals

As human beings, we continuously experience that our physiology is tightly linked to our mental states (Cacioppo et al., 2007). We find ourselves blushing when we are embarrassed, we hold our breath when we are anxious, and feel our heart rate rising when we are aroused. Biosignals are also inherently very intimate; you have to be physically close to one another to hear a heartbeat or breathing – a situation that typically only occurs between sexual partners, close friends, or parents and their children. Because of their emotional nature, biosignals also possess intrinsic social qualities, as shown by studies on psychophysiological synchronization that suggest that when people feel empathy toward each other, their physiological signals tend to synchronize (Marci & Riess, 2005; Palumbo et al., 2017). We have integrated these social and emotional aspects of biosignals naturally in our daily lives; it is part of our communication with common language expressions such as “my heart skipped a beat” and “to break a sweat”, and in media, for example with the addition of sounds of a thumping heart to a thrilling movie scene.

In the past decade, the field of psychophysiology has benefited notably from the rapid development of technologies to measure physiological activity. This advancement in biosensing devices has made physiological measurements less costly and much easier to conduct, especially with the increased availability and quality of available wearable devices. Perhaps the most widespread use of biosensing is known as biofeedback, which generally refers to providing a person with real-time information about his or her physiological activity. With biofeedback, this information is usually only presented to the person whose data are measured, mostly with the purpose to give insight in physiological changes or to facilitate increased control over these same physiological processes. However, because our physiology also has an intrinsic emotional and social nature, our biosignals might not only be relevant to ourselves, but could also provide valuable information in communication when they are shared with others. This latter application possibility – biosignal sharing – constitutes the topic of this review.

1.2. Biosignal sharing as a new communication medium

Biosignal sharing has its academic roots in psychophysiological literature as well as in affective computing, which is a branch of computer science focused on sensing, understanding, and simulation of emotion, and building functional affective systems to enhance human-computer interaction. Neither computer science nor psychobiological perspectives are generally informed by communication theories, which emerge from media psychology and human communication studies. Instead of viewing biosignal sharing with a biological or technical lens, communication theories bring its communication properties to the fore, focusing on the functionality of the system in terms of their characteristics in interpersonal communication, rather than purely technological properties, or psychobiological effects. Analyzing biosignal sharing through this lens also allows us to benefit from knowledge gathered with previous communication media, instead of approach it as an entirely new phenomenon.

Biosignal sharing can be considered a new communication medium. In communication and media literature, this is defined as the means by which a message is communicated or transmitted, and how a message is delivered or presented via that medium is referred to as the modality (Bracken & Dalessandro, 2017). For example, newspapers deliver their messages by the modality text, and videoconference uses the modalities audio and video. In the conceptualization of biosignal sharing as a new communication medium, one can consider the biosignals themselves as a new modality.

The types of modalities that are used in a message differ across different media, and the selected modalities determine which verbal and non-verbal cues are being exchanged, or in other words, which (social) information is available in social interaction. A substantial amount of work has been devoted to investigating the effects of various verbal and non-verbal communication cues on the interpersonal communication process (e.g., Fichten et al., 1992; Knapp & Daly, 2011; Knapp et al., 2013). These studies show that our communication is highly influenced by the available social cues – the particular communication cues that are being exchanged in the interaction.

1.3. A framework of communication dimensions

To explain the differences in communication properties across various media, Clark and Brennan (1991) introduced an influential framework outlining several dimensions on which communication media can vary. They argue that each medium constitutes a unique combination of these communication dimensions that determine its particular characteristics. These characteristics, in turn, can be used to help understand the different experiences of the communication process through various media and the social consequences of their use. Clark and Brennan’s (1991) dimensions are supported and extended in later work following the development of new technologies and corresponding emergence of new communication media (e.g., Baym, 2015; Herring, 1999). Similarly, this approach can be applied when conceptualizing biosignal sharing as a new communication medium. Because this communication framework originates from the field of communication and media psychology, it captures the design space in terms of relational aspects – functions that only emerge in interaction with and through the system – rather than isolated technical properties or psychobiological effects.

Various influential communication theories could be applied to analyze the communication process of biosignal sharing interactions. For instance, Grice’s cooperative principle (Grice, 1989) might help to clarify how people cooperate in communication to achieve mutual understanding, and Tannen’s Difference Theory (Tannen, 1990) and Norton’s Theory of Communicator Styles (Norton, 1983) could facilitate our understanding of differences between people’s styles of communicating. While providing valuable knowledge on the human communication process, these theories do not allow to shed light on communication experiences in light of the particular characteristics of communication media. A specific strength, then, of using a framework of communication dimensions such as Clark and Brennan’s (1991) is that it provides us with a means to assess communication properties in order to understand how the mediated interactions are experienced. As such, it is a mid-level theoretical framework that helps us understand how various design choices in mediated communication uniquely afford or constrain mediated interpersonal interactions, thus shaping communication experiences. We have adopted and extended this framework of existing communication dimensions (Clark & Brennan, 1991; Heeter, 1999; Baym, 2015) and selected and adapted them based upon their support in literature and their relevance for differentiating biosignal communication systems (e.g., Evans et al., 2017). Therein, we have used Clark and Brennan’s terms as a basis for our framework, while being aware that similar dimensions have also been referred to with different terms. For instance, reviewability is also known as persistence (e.g., Treem & Leonardi, 2013) or permanence (Whittaker, 2003), and revisability is also known as editability (e.g., Walther, 1993). In addition, we have merged Clark and Brennan’s (1991) dimensions of cotemporality and simultaneity into synchronicity, as this term is more widely applied in research on communication media (e.g., Burgoon et al., 2002; Heeter, 1999). The resulting communication dimensions are presented in Figure 1.

Figure 1. Communication dimensions and their descriptions.

In earlier work, these dimensions have been applied in the analysis of other communication media and how their characteristics influence the communication process (e.g., Baym, 2015; Burgoon et al., 2002; Clark & Brennan, 1991; Dennis et al., 2008; Friedman & Currall, 2003; Sutcliffe et al., 2011). For instance, synchronicity is associated with faster message transmissions (Baym, 2015) and allows for immediate feedback on how the message is received, so potential miscommunications can be repaired quickly, and understanding is improved (Dennis et al., 2008). Face-to-face and telephone conversations are synchronous and therefore have these benefits. However, they do not afford signals to be revised before sending, whereas revisability has been shown to facilitate conveyance of the intended meaning (Dennis et al., 2008). E-mail and text messaging do afford this, but this asset then again comes at a cost of lack of synchronicity, which can delay the message exchange and lets mistakes persist (Baym, 2015; Friedman & Currall, 2003). It also increases the chance of confusing the order of turns in a conversation – i.e., no sequentiality – which can result in misunderstandings (Clark & Brennan, 1991). More generally speaking, there are trade-offs between the communication characteristics that different media afford, and which choices are desired is dependent on the purpose of the interaction (Clark & Brennan, 1991) and other factors, such as how familiar participants are with their tasks and each other (Dennis et al., 2008).

In a similar fashion, our framework of communication dimensions can be deployed as a tool to examine the communication characteristics of biosignal sharing. Assessing developed prototypes for the biosignal sharing medium along such a framework provides us with an overview of the characteristics of biosignal sharing and could potentially serve as a design space to facilitate the design of new biosignal sharing systems.

1.4. The experience of sharing biosignals

Besides viewing the biosignal sharing medium along our framework of communication properties, one also needs to consider subjective aspects such as experiences and perceptions, to fully understand how biosignal sharing affects our social interactions. How are the signals perceived and interpreted; which feelings and thoughts does communication through this medium evoke; and what are social consequences of its use? These questions seem to be even more relevant due to several unique characteristics of biosignals. For instance, we have become familiar with media that use the modality text as the means to convey a message – i.e., text messaging, e-mail, chat – so with the introduction of a new medium that uses text we can draw on previous experiences to guide the mediated interaction. In contrast, the modality of biosignals is entirely new, and we are still unfamiliar with communicating using physiological information. Moreover, our physiology is, in origin, mostly covert, private, and intimate. In addition, biosignals are automatic in the sense that they are generated without conscious effort. Therefore, when biosignals are shared, information that used to be largely obscured and subconscious, suddenly becomes explicitly visible to oneself and other people. A signal for which you previously had to be in close physical proximity in order to perceive can now be received at a distance, and distributed over time, space, or numerous interactions. In short, biosignal sharing can transform the experience of social interaction in new ways, which can have both positive and negative effects on the communication experience and interpersonal relationships.

On the positive side, because of the personal and private nature of our physiological information, exchanging these intimate signals could help establish rapport between interacting parties. In a time in which technology is often seen as ‘cold’ and ‘impersonal’, and concerns are growing that its ubiquity will lead to disconnectedness between people, exploring the potential of sharing biosignals to enhance closeness seems worthwhile. Indeed, first studies have shown that using physiological feedback in communication can enhance interpersonal intimacy (Janssen et al., 2010), closeness (Werner et al., 2008) and positive affect (Tan, Luyten et al., 2014). At the same time, studies also show that sharing intimate signals through technology can evoke feelings of vulnerability and raise concerns with respect to privacy (Baym, 2015). People might envision daunting future scenarios in which we will not be able to keep secrets anymore – our minds and feelings constantly exposed. In other words, there are conceivable benefits as well as costs in the affective outcomes of biosignal-mediated communication that should be considered in the design and evaluation of such communication systems (IJsselsteijn et al., 2009).

1.5. Current study

In short, this study aims to address the question: what are the communication characteristics of biosignal sharing and how do they influence the experience of biosignal-mediated interactions? We will approach our question by conducting a systematic literature review of the existing work on sharing biosignals in social interaction in order to assess the current state of the field. We specifically focus on studies that use biosignal feedback as a communication cue, that is, where information about physiological activity is intentionally exchanged with the purpose of providing (additional) information to a social interaction. In addition, we focus only on interactions between humans.

Our review will combine two methods of analysis to cover both parts of our research question. First, we will assess the developed biosignal sharing systems along a framework of known communication dimensions to get an overview of the communication characteristics of the biosignal sharing medium. Second, to capture the experiences of communicating through biosignal sharing systems, we will conduct a thematic analysis of reported users’ perceptions. These findings will be cross-examined to get a deeper understanding of how the communication characteristics of biosignal sharing relate to the experience of biosignal-mediated interactions.

By systematically reviewing the existing work on this subject and drawing on theoretical knowledge about communication and media psychology, we hope to make progress in gaining an understanding of biosignal sharing, and create a descriptive framework upon which to build in future biosensing research. This basis also serves as a tool for well-informed choices in the design, development, and evaluation of new systems for biosignal communication, and we present several considerations for design and directions for future research at the end of our review.

2. Methods

2.1. Search process

A systematic literature review was conducted to gain an overview of the existing literature on biosignal sharing in social interaction. The procedure of the review followed guidelines set out by Okoli and Schabram (2010). To cover all the aspects of our scope, we defined two sets of keywords: one to capture the physiology aspect, involving various physiological measures, and one to focus on the social interaction aspect. The two sets were combined using the search operator AND, which implies that each retrieved record matched with at least one term in each set. See Figure 2 for the individual keywords. The search was performed in three databases: ACM Digital Library, Web of Science, and Scopus on the fields ‘title’ and ‘topic’. Only original research articles, conference papers, and conference workshops written in the English language were included. In the case of Web of Science and Scopus, the query produced such a large set of results that entries were further selected on research area, excluding fields not relevant to our research questions (e.g., physics, cardiology, chemistry). Appendix presents the entire search query for each database.

Figure 2. Individual keywords used in the search queries.

The search in the digital libraries was conducted on articles published until August 2019. Figure 3 presents the flow chart for the selection process of articles to be in- or excluded.

Figure 3. Selection process.

Across the three databases, the initial search resulted in 4,695 entries. These were screened with manual inspection of titles and abstracts, according to the following selection criteria:

• Using physiological measures: i.e., heart rate, skin conductance, respiration, brain signals

• Using biosignals as a communication cue: i.e., intentionally conveying the signal to one or more interactional partners (not being the researcher)

• Human-human interaction

When there was any doubt, records were included for later assessment of the full-texts. During the selection process, we found several articles related to the same research projects. We chose to include articles from the same projects only when the paper described new and relevant information. This database search yielded a set of 42 papers. Then, a second round was performed with an iterative process of screening the references in the selected records and in case a new article was selected, their references were checked as well. This resulted in the addition of 31 new records. As a final step, all articles that cited the included papers were inspected by entering the included papers in Google Scholar and retrieving the records under ‘Cited by’, which yielded 29 new records. The total search process resulted in a selection of 102 records.

Subsequently, the full-text articles of this selection were submitted to an eligibility assessment by two independent reviewers, using the same criteria for selection as listed above. Discrepancies regarding inclusion or exclusion were discussed until agreement was reached. During this assessment, 36 articles were excluded. The most prominent reason for exclusion (n = 18) was that, after inspection of the full-text article, it appeared that the physiological information was not primarily used as a communication cue, but for example as information on the sender’s performance. Another reason for exclusion was that the physiological information was available only to the researchers (n = 7) or to the sender (i.e., individual biofeedback; n = 3) instead of exchanged between interactional parties. On five occasions, sharing of biosignals was not the main focus of the study, for example, in a literature review on physiological synchronization. Three records appeared to be proposals for research or design and were therefore excluded. In total, 66 entries met all the inclusion criteria.

2.2. Data analysis

To get an overview of the included studies, we started our review with examining them for their general characteristics. Several aspects were taken into account, including the authors, year of publication, scientific field, type of study, research methodology (i.e., kind of data collection, lab or field study, number of participants), whether the study included the development of a prototype, and whether the authors made use of established theories to base their work on. Categorizing articles proved challenging, because most of the studies consist of a mix of design and research, which made it hard to assign them to one particular category. Therefore, we decided to employ non-mutually exclusive categorizations. More specifically, we examined whether the articles contained the following elements: description of (technical details of) a self-developed prototype, description of the design process of a prototype, user evaluation of a prototype, study on experiences of biosignal sharing, or a literature review/reflection.

To address the first part of our research question and assess the characteristics of biosignal sharing in the communication process, we focused on the developed prototypes for biosignal communication. We started with making an inventory of the general features of the prototypes, which included the physiological measure(s), how the physiological signals were transformed, represented, and distributed, and – if described – the rationale for its design. Then, we assessed the prototypes along our framework of communication dimensions as listed in Figure 1 to structure the design space of these systems and enable a structured examination of their communication characteristics.

To answer the second part of our research question, findings on the experiences of sharing biosignals were gathered and analyzed. Because most of the reviewed articles employed a qualitative approach, a formal meta-analysis of the results was not possible. To capture the rich insights provided in the articles, we conducted a thematic analysis. For this analysis, we followed the steps outlined by Braun and Clarke (2006). First, we selected the papers that reported on users’ interpretations, perceptions, and experiences of biosignal sharing – totaling a number of 50 papers. Within these papers, we coded the sections that covered their findings. That is, our primary source was the authors’ interpretation of their data, not the user reports themselves. The extracted codes were categorized and further abstracted into themes. Using this method, the analysis yielded 18 categories, such as ‘lack of control’, ‘desire for context’, and ‘expression of emotional states’. Subsequently, these categories were inductively clustered into three themes, which are described in Section 3.3.

As a final step of our review, we integrated our findings by cross-examining the results from both analyses together with established knowledge on the properties of physiological signals. This provided us with a first understanding of how inherent properties of biosignals and specific communication characteristics of biosignal sharing systems are related to experiences of biosignal-mediated interactions, which we discuss in Section 4.1. Based on these insights, we present considerations for design and directions for future research in Section 4.2.

3. Results

3.1. General description of existing literature

The 66 articles included in our selection (results listed in Figure 4) were published in 1975 (1 study) and between 2002 and 2019. As can be seen from Figure 5, there is a sharp increase in the number of published papers in the last five years. The papers were published in peer-reviewed conference proceedings (49 studies, 74%) or scientific journals (17, 26%) in the fields of computer science/engineering, psychology, or design, with the vast majority in the interdisciplinary field of human-computer interaction that combines these three domains. Regarding the intended aim of the biosignal communication, biosignal sharing was most frequently applied to facilitate the interpersonal relationship or communication (37 studies), for example by facilitating feelings of empathy or the expression of emotions. Other use cases in the reviewed papers were gaming and entertainment (7 studies), digital collaboration (6 studies), arts (6 studies), sports (4 studies), psychotherapy (3 studies), or research in general (3 studies).

Figure 4. Overview of the reviewed articles. NA = Not Available.

Figure 5. Year of publication of included papers.

Most of the reviewed studies included a description of one or more self-developed prototype(s) for biosignal communication (54 studies and 57 unique prototypes). Therein, we considered a developed biosignal sharing system a prototype when the design of the artifact was part of the purpose of the paper, instead of solely a means to answer a research question. All of them provided the technical details, and nearly half (21 studies) also described the design process, such as multiple design iterations or a pilot study. Fifty of the total body of reviewed papers reported qualitative findings of how biosignal-mediated interactions are experienced, which were included in our thematic analysis. Last, two articles were a conceptual review and a literature review of a related topic (i.e., live biofeedback).

The majority of the 66 included papers described an empirical study (55 studies) conducted as part of the design process and/or to examine user experiences. These studies used qualitative measures (36 studies), quantitative measures (7 studies), or a combination of both (12 studies). The remaining consisted of literature reviews (2 studies) or descriptions of a prototype concept (9 studies). Of the empirical investigations, the main methods used for data collection were interviews or focus groups (33 studies) and questionnaires (32 studies), often in combination with each other or with another method such as observations, video recordings, or chat logs (23 studies). The questionnaires were mostly custom questionnaires with open-ended questions. In a third of the studies these were supplemented by validated questionnaires (e.g., Affective Benefits and Costs of Communication Technologies by IJsselsteijn et al., 2009; Self-Assessment Manikin by Bradley & Lang, 1994). A minority (15 studies) mentioned to have collected the measured physiological activity as a source of data, of which only seven also reported results. Regarding the research environment, most studies were conducted in a laboratory setting (34 studies), and the remaining either in the field (18 studies), combination of lab and field (2 studies), online (2 studies) or at an exhibition (3 studies). The study samples were typically recruited from university participant pools, with average ages of the participants between 20–30 years old. Most sample sizes varied between 10–30 participants – generally concerning a qualitative user evaluation of a prototype -, and higher numbers when the study typically utilized more time-efficient methods such as an online survey. Only 16 studies included some account of considering research ethics in their procedure, which typically consisted of letting participants sign an informed consent form before participating.

3.2. Prototypes for biosignal communication systems

3.2.1. Characteristics of reviewed prototypes

In total, 57 custom prototypes for biosignal communication are described in the literature, presented in Figure 6. All of the papers included a description of the technical details of the prototype, although there was a great variation in the level of detail provided by the authors. Some described the design process (21 systems), consisting of, for example, focus groups, surveys, or pilot studies, sometimes in multiple iterations. In general, the papers referred to related work to support their approach, and only eight studies explicitly mentioned established theories, for example literature on emotion (Cornelius, 1996; Ekman & Davidson, 1994), self-presentation (Goffman, 1959), and motivation (Baumeister & Leary, 1995). Interestingly, only two referred to a media or communication theory (i.e., Daft & Lengel, 1986; Walther, 1996); references to information visualization theories were never made, even though this was expected considering that biosignal sharing implies the visualization of physiological information.

Figure 6. Overview of the reviewed prototypes. X = Yes, – = No, NA = Not available, O = Optional, M = Multi-directional, R = Receiver.

Regarding the type of physiological activity measured, heart rate and heart rate variability were the most frequent (39 systems), followed by skin conductance (17 systems), respiration (9 systems), EEG (7 systems), and body temperature (2 studies), often in combination. The devices used for measurement varied widely, ranging from consumer devices (e.g., Polar band and NeuroSky MindWave) to specialized instruments (e.g., TMSi MOBI), and custom-built sensors, generally consisting of electrodes connected to a platform, such as Arduino.

The majority of the prototypes (49 systems) used a visual representation of the biosignals, for example numbers, graphs, lights, or shapes. Other forms were auditory (8 systems) and haptic (14 systems). Nine systems applied multiple representations in combination. Regarding transformation of the signals, the presentation was ‘raw’ in 23 prototypes, such as showing the heart rate as a number or graph.¹ Other developers (23 systems) chose to convert the signals to a particular (self-determined) representation (e.g., assigning colors to heart rate values). In another group of prototypes (17 systems) the signals were first interpreted by relating them to other (psychological) constructs such as arousal or attention, before converting them to a particular representation (e.g., assigning colors to levels of arousal). Three prototypes used multiple transformations.

3.2.2. Assessment in terms of communication dimensions

The prototypes were assessed using our framework of communication dimensions (listed in Figure 1). Results of the entire assessment can be found in Figure 6, and an overview of scores is presented in Figure 7. As this figure shows, there were distinct differences in scores between dimensions: most prototypes afford synchronicity, but not revisability, reviewability, sequentiality, or autonomy. For copresence and reciprocity the scores were more equally divided. We will now discuss the observations of this assessment in more detail.

Figure 7. Overview of the scores of the reviewed prototypes on the communication dimensions. O = Optional, M = Multi-directional, R = Receiver.

Most of the prototypes (31 systems) were designed for remote collaboration, whereas others (23 systems) required copresence, meaning that both parties have to be in the same physical environment to be able to perceive the transmitted signals. In three systems this was optional, depending on the specific use case (indicated with ‘O’ in Figures 6 and 7). Furthermore, most of the prototypes (49 systems) were synchronous – showing real-time data. In two systems this was optional, again depending on the specific use case (indicated with ‘O’ in Figures 6and 7). As explained in the introduction, this also implied that the large majority of the prototypes (52 systems) did not allow revisability, that is, there was no option to view and modify the signals before they are sent to the recipient. Also closely related was the observation that in 46 prototypes the signals were transient, which means senders and receivers could not review the signals after they are sent. In the other prototypes (11 systems), the data was saved and could be reviewed at a later point in time.

Scores on the dimension of reciprocity showed more variation: in 34 prototypes communication was reciprocal, with 25 systems exchanging signals in two directions, and nine systems in multiple directions – exchanging biosignals within a group (indicated with ‘M’ in Figures 6 and 7). The other 23 prototypes were non-reciprocal, i.e., there was only one sender. Further inspection of these cases showed that in eight of them there was only one recipient, in five systems the signals were presented to a select group, and in 10 prototypes the sender’s biosignals were visible to the public. Remarkably, there were five cases in which senders could not view their own signal, whereas recipients could. Sequentiality only applies when communication is two- or multidirectional. Of the reciprocal systems, only five afforded sequentiality, whereas in the remaining 29 systems, management of the turns (i.e., clear order of sending and receiving) was not facilitated.

A notable difference between the new biosignal sharing medium and traditional media concerned the dimension of autonomy. Most prototypes (41 systems) did not afford autonomy by the sender, that is, measuring and sending of the biosignals was triggered by the system. Even stronger, in four systems the receiving party could probe the senders’ biosignals at will (indicated with ‘R’ in Figures 6 and 7). Only 11 biosignal sharing systems allowed the sender to initiate this process, as is most common in other communication media.

3.3. Experiences of biosignal sharing in social interaction

Of the selected articles, 50 studies reported findings on the experience of biosignal sharing in social interaction. These were examined with a thematic analysis following the procedure described in Section 2.2. During this analysis, the following themes emerged: benefits of sharing biosignals, discomfort with sharing biosignals, and mixed views on interpreting biosignals.

3.3.1. Benefits of sharing biosignals

In our review, studies reported on multiple ways in which biosignal sharing can be beneficial in social interaction. Most research efforts focused on the potential of social biofeedback to enhance the interpersonal relationship between interacting parties. Although the applied measures varied widely, the general finding was that biosignal sharing can increase feelings of connectedness (Buschek et al., 2018; Curmi et al., 2013, 2017; Hassib et al., 2016; Kim et al., 2015; Liu et al., 2017a, 2019; Marci & Riess, 2005; Robinson et al., 2016; Slovák et al., 2012), empathy (Frey & Cauchard, 2018; Hassib et al., 2017; Tan, Luyten et al., 2014), intimacy (Howell et al., 2019; Janssen et al., 2010), affective interdepence (Salminen et al., 2018), and sharing of an experience (Kurvinen et al., 2007; Sun & Tomimatsu, 2017; Walmink, Wilde et al., 2014).

Several studies found that participants felt it took less effort and time to send a biosignal message compared to for example typing a text message, lowering the barrier to connect (Hassib et al., 2017; Liu et al., 2019). Findings also suggest that the shared signals could support social interactions by triggering conversations about what participants were feeling and experiencing (D’Souza et al., 2018; Hassib et al., 2017; Kurvinen et al., 2007; Liu et al., 2017a, 2019; McHugh et al., 2010; Walmink, Chatham et al., 2014).

Besides the context of interpersonal relationships, several studies used the physiological feedback in contexts of remote collaboration and education, with the aim to facilitate communication in dyads (Dey et al., 2018; George & Hassib, 2019; Tan, Luyten et al., 2014; Tan, Schöning et al., 2014) as well as in teams (Knierim et al., 2017; Lyons et al., 2004). Furthermore, it was found that the addition of biosignal information resulted in higher engagement, enjoyment, and playfulness of the (mediated) interaction (Hassib et al., 2017; Lee et al., 2014; Liu et al., 2017a), game (Al Mahmud et al., 2007; D’Souza et al., 2018; Robinson et al., 2017) or cycling activity (Walmink, Wilde, & Muller, 2014). There were some accounts of the social biofeedback to be calming, with participants reporting they found it soothing to hear or feel the other’s heartbeat or breathing (Frey et al., 2018; Hassib et al., 2017; Howell et al., 2019; Salminen et al., 2018).

These results present several benefits that sharing biosignals can have. But besides (mostly qualitative) support for these positive effects, the reviewed studies also frequently showed mixed experiences. Importantly also, of the 19 studies that used quantitative measures, 11 included a control condition without biofeedback, and only six of them found a significant positive effect on (at least) one of their measures.

3.3.2. Discomfort with sharing biosignals

In the studies with qualitative data, feelings of discomfort with sharing and mixed perceptions on the interpretation of biosignals were frequently reported. For instance, participants stated that the signals would be too revealing (Liu et al., 2017b), or that they felt exposed or vulnerable (D’Souza et al., 2018; Hassib et al., 2016). In line with these findings, people preferred to receive signals rather than share their own (Frey, 2016; Hassib et al., 2016). Several studies described that users explained these feelings by the uncontrollable and personal nature of this information (Liu et al., 2017a; Slovák et al., 2012). Some users stated that they did not want to share their biosignals in fear of judgment (Hassib et al., 2016), or even that they felt the other party would ‘take advantage’ of the information (Williams et al., 2015). Accordingly, participants reported that they would like their biosignals to support the image they intended to project to others (Howell et al., 2016).

Furthermore, the willingness to share seemed to be highly dependent on the relationship between sender and recipient: multiple studies reported that participants expressed they would only feel comfortable to share their signals in close relationships (Buschek et al., 2018; Frey & Cauchard, 2018; Hassib et al., 2017, 2016; Liu et al., 2017a, 2019). Another factor influencing willingness to share appeared to be the valence of the shared signals: participants sometimes stated they would only prefer to share positive information (Buschek et al., 2018; Hassib et al., 2017). Additionally, equal access to the signals seems to play a role: in two studies, senders were found to experience higher levels of stress when sharing their biosignals without seeing them compared to when they could view the signals themselves too (D’Souza et al., 2018; Frey, 2016). It has to be noted that there were also two studies reporting that users that did not mind other people viewing their signals and felt rather indifferent about who had access to them (Kurvinen et al., 2007; Walmink, Wilde, & Muller, 2014).

A number of studies report on other adverse effects or issues regarding the security and ethical side of sharing biosignals. In two studies, some users reported concerns about being mentally overloaded and distracted by the extra stream of information (Liu et al., 2017b; Tan, Schöning et al., 2014). One study described that participants were concerned whether their biosignals could be individually identifiable (Howell et al., 2019), and two studies elaborated on challenges of data integrity (Buschek et al., 2018; Curmi et al., 2013). Some authors briefly touched upon privacy considerations in designing their study (Chanel & Mühl, 2015; Howell et al., 2019), others mentioned to expect privacy and security concerns from users, but did not provide concrete examples or reports (Curmi et al., 2014; Frey et al., 2018).

3.3.3. Mixed views on interpreting biosignals: ambiguities and importance of context

The extent to which people were able to interpret the (intended) meaning of the shared biosignals varied greatly, and reports of significant ambiguity were frequent across the reviewed studies. In general, studies found that participants assumed biosignals to be related to emotions and mental states (e.g., Dey et al., 2018; Merrill & Cheshire, 2016; Slovák et al., 2012; Snyder et al., 2015). This is in line with participants on the receiving end reporting that the conveyed biosignals helped them to get a better understanding of the sender’s emotional state (Chanel et al., 2010; Dey et al., 2018, 2017; DiMicco et al., 2002; Hassib et al., 2017; Kuber & Wright, 2013; Liu et al., 2017b; Marci & Riess, 2005). This seemed to work both ways: senders also reported that they felt biosignal sharing could help them to convey their emotions and mental states to the receiving party (Liu et al., 2017a; Roseway et al., 2015). At the same time, however, other studies found that the shared biosignals were often met with confusion and questions; the ambiguity of and unfamiliarity with this kind of information leaves a lot of room for speculation and personal interpretations (Curmi et al., 2017; Curran et al., 2019; D’Souza et al., 2018; Kurvinen et al., 2007; Tan, Luyten et al., 2014). Because of that, some of the participants had doubts about the added value of sharing biosignals in communication (Hassib et al., 2016; Liu et al., 2017a, 2019; Merrill & Chesire, 2016).

A few studies tried to investigate more systematically how we interpret biosignals, finding that, compared to normal values, elevated values of biosignals are generally interpreted as signaling negative mood states in an uncertain situation (Merrill & Chesire, 2016; Merrill & Cheshire, 2017). Multiple studies describe factors that influence the interpretation, such as the way the signals are visualized (Curmi et al., 2013; Liu et al., 2017b), a specific challenge being that the dynamic nature of physiology might make changes too subtle to notice (D’Souza et al., 2018; George & Hassib, 2019; Hassib et al., 2017). Additional contextual factors that influence the interpretation of biosignals are the specific situation (Merrill & Chesire, 2016), the relationship with the recipient (Slovák et al., 2012), and previous knowledge and beliefs about physiological signals (Curmi et al., 2017; Merrill & Chesire, 2017; Liu et al., 2019). This corresponds to reports of recipients who expressed a need to know more about the context of a signal in order for it to be informative (Liu et al., 2017a; Slovák et al., 2012), and senders supplementing their biosignals with additional information in text to achieve mutual understanding (Buschek et al., 2018; D’Souza et al., 2018; Liu et al., 2019). The reliance on textual information is in accordance with Curran et al. (2019), who found that narrative information improved empathic understanding of another’s emotional response to a video, whereas biosensory information did not.

4. Discussion

Advances in biosensing technologies expand the range of possible applications for physiological signals, including applications in interpersonal interaction. We presented the results of a systematic review of the relevant literature, in which we have analyzed biosignal sharing from the perspective of a communication framework. Our analysis demonstrated that there are multiple potential benefits of sharing biosignals, but also complexities regarding feelings of discomfort with sharing and ambiguities in how social meaning is constructed from the received physiological information. The majority of existing biosignal sharing systems is characterized by sharing signals in real-time, but by a low level of user control, i.e., not allowing to review or revise messages and showing a high level of automation in measuring and communicating biosignals. Cross-examining the data from the assessment of existing prototypes, the thematic analysis of experiences, and previous knowledge on biosignals elucidates how characteristics of biosignal sharing influence the experience of biosignal-mediated interactions. The reviewed data does not allow us to draw conclusive statements on these relationships – this would require structured comparisons of the effects of systems with and without certain properties and adequate control conditions – however, based on our findings, we can make informed observations about the reported experiences in context of the communication framework. In the next sections, we will further elaborate on these observations and provide considerations for research and design that follow from our main insights.

4.1. Disentangling the complexities of biosignal sharing interactions

4.1.1. Different use contexts relate to specific characteristics

Our review presents several scenarios in which biosignal sharing can have a potential benefit, such as private communication, gaming and entertainment, and remote collaboration. Looking at this variety of purposes, one can link each intended use context to specific inherent biosignal characteristics, such as their intimate nature, emotional connotations, and inherent ambiguity. For instance, many reviewed prototypes were developed with the aim to support feelings of closeness and empathy between interactional partners. Thereby, these systems emphasize the intimate nature of biosignals. Other researchers employed biosignal sharing as a way to facilitate the expression of emotions and experiences between interactional partners, thereby utilizing the emotional information that physiological measures contain. Yet another group applies biosignal sharing in a gaming and entertainment context, tapping into the properties of novelty and ambiguity by adding biosignal communication as a playful element to the game. In these latter areas of application the ambiguity and uncertainty of biosignals is used as a feature rather than a bug (comparable to the functional use of uncertainties inherent in GPS and WiFi as localization signals – Benford et al., 2006; also see Gaver et al., 2003).

Besides linking the various potential benefits to inherent properties of biosignals, the application of the communication framework in the current review allows to relate communication dimensions to these benefits. Our analysis shows that depending on the specific context of use, different dimensions become more or less important, which helps to understand why biosignal sharing systems do or do not achieve their intended purpose. For example, an important characteristic for the benefit of enhancing closeness was synchronicity. Apparently, receiving biosignals of your interaction partner at the exact same moment that they are generated, evokes a feeling of closeness with that person. This finding is in line with earlier work on the effect of synchronicity in computer-mediated communication, which states that the immediacy of the communication fosters the perception of closeness between interactional partners (O’Sullivan et al., 2004). This suggests that synchronicity in biosignal-mediated communication has similar effects on affiliative feelings between interactional partners as in other communication media. Feelings of closeness can also be fostered by supporting reciprocity: bidirectional sharing of biosignals seems to facilitate a feeling of intimacy with the other person, in line with the association between reciprocal self-disclosure and intimacy found for text-based mediated communication (Jiang et al., 2011). Furthermore, closeness is influenced by the dimension of autonomy: as Janssen et al. (2014) have demonstrated, signals that are intentionally shared by the sender are experienced as more meaningful and intimate compared to automatically generated messages, even though message content can be exactly the same. When the use context concerns gaming or sports, synchronicity is also a facilitating characteristic. In these contexts, synchronicity seems to encourage playfulness, an effect that was also found for other communication media (Danet, 2001). On the other hand, this dimension is less important when the intended purpose is to facilitate reflection on emotions and experiences. In these cases, reviewability is more helpful, because this allows to jointly reflect on experiences. In short, the various inherent properties of biosignals make them applicable to a variety of interaction contexts and depending on the use context, specific communication dimensions come to the fore.

4.1.2. Difficulties in constructing social meaning

Another prominent theme emerging from the current review is that people often do not know what these signals actually mean. An important source of our difficulties with interpreting biosignals seems to lie in our inexperience in making sense of physiological information. Traditionally, our communication relies heavily on language and we have become very familiar with deriving meaning from words. This not only holds for language, but also for non-verbal cues such as facial expressions and gestures (e.g., Andersen & Guerrero, 1998; Knapp et al., 2013), and indirectly even for physiological information, as we intuitively interpret people’s blushing and sweating. However, with biosignal sharing, this information now becomes explicit and presented in a way that we are unfamiliar with. When such explicit access to our physiological information becomes more common, it could be that we may develop the required skillset in understanding and using them, similar to how we learned to make sense of cues that were introduced with the development of earlier digital media, such as emoticons (Dresner & Herring, 2010).

Despite our inexperience in interpreting biosignals, people do have assumptions about their meaning and automatically associate them with mental states. The reviewed studies also point out that the specific emotion that the biosignal is attributed to is highly context-dependent, in line with past work on the effects of physiological feedback on the labeling of emotional stimuli (e.g., Valins, 1966). Accordingly, the current review shows that additional contextual information is often needed for mutual understanding of the shared biosignals. These results resonate with existing work on earlier communication media that showed that the available contextual cues have a great influence on how we interpret messages in mediated communication (Baym, 2015), and that mutual understanding is compromised when contextual cues are missing (Clark & Brennan, 1991; Burgoon et al., 2002). This suggests that biosignal-mediated communication faces similar issues regarding the need for context as other communication media. In terms of our communication framework, copresence and synchronicity have a large impact on this need, as they determine to a great extent how much contextual information is available. Interactional parties who are copresent have access to more contextual information than those who communicate remotely, as they are aware of each other’s physical location and circumstances. This helps to establish common ground and so improves mutual understanding (Clark & Brennan, 1991) Likewise, synchronicity provides information about the context of a biosignal: knowing exactly when it was sent, supports its interpretation and affords repair of potential misunderstanding. The number of studies on the effect of contextual information is still relatively small, and the question which contextual factors are particularly important for the interpretation of biosignals has to be more thoroughly studied.

4.1.3. Amplified concerns about self-presentation

Our review also shows that people experienced a substantial level of discomfort with biosignal sharing. An important part of this feeling appears to be due to concerns for how biosignal sharing affects self-presentation. When people engage in a social interaction of any kind, the way they present themselves is a major concern (Leary, 1996). People constantly monitor and manage their communication signals in an attempt to selectively provide information that constructs a positive image toward the other (Goffman, 1959). This behavior to control our self-presentation holds for both unmediated and mediated communication (O’Sullivan, 2000; Walther, 1993). From the current review, it can be concluded that in biosignal sharing, these concerns over self-presentation are amplified. This seems to be largely due to the fact that our physiology is experienced as a form of emotional self-disclosure that is beyond our volitional control, similar to non-verbal signals such as blushing or frowning. And, perhaps even more so than with blushing or frowning, we are typically not consciously aware of our physiological signals, which makes it difficult to know what we are sharing. Following Goffman’s (1959) distinction between intentional (e.g., verbal utterances) and unintentional communication cues (e.g., non-verbal signals such as body posture), biosignals should clearly be categorized as the latter, and so these signals might unintentionally reveal more (or different) information than one would desire.

As O’Sullivan (2000) argues, we attempt to know what the other party knows to get a sense of what information about ourselves is presented to the other, and when this information is obscured this results in a high degree of uncertainty about the other’s impression of us. This line of reasoning strongly resonates with Erickson and Kellogg’s (Erickson & Kellogg, 2000) concept of social translucence, which can be defined as the extent to which a communication system makes social cues visible and thereby the amount of social information that is available in the communication process. According to Erickson and Kellogg, a socially translucent system supports mutual awareness between interacting parties: to be aware of the other party but also of what the other party is aware of. They argue that (a certain degree of) social translucence is crucial for mutual understanding and that the predominant lack of social translucence in computer-mediated communication explains why smooth interaction flows in computer-mediated communication often prove to be challenging.

We can relate the degree of mutual awareness to our communication dimensions, in particular revisability, copresence, reviewability, and sequentiality. If a communication system does not afford revisability, the sender cannot check the content of the message that is being communicated; if parties are not copresent, the sender cannot confirm that the recipient has perceived the message in the intended way and cannot repair potential misunderstanding of the signal. In our review, we find that many of the reviewed prototypes do not afford these communication characteristics, which implies they have a low degree of social translucence. This helps to understand why communication through biosignal sharing can feel uncomfortable to many people: not knowing which signals you have sent (revisability and reviewability), whether they are received and perceived in the intended fashion (copresence), and which of the signals is being responded to (sequentiality). All of these properties introduce a considerable amount of uncertainty in the communication process, especially with such ambiguous and unfamiliar signals. Moreover, detection and repair of miscommunication is hard: lack of sequentiality and revisability are associated with increased occurrence of misunderstandings (Dennis et al., 2008), whereas lack of reviewability hinders the detection (and correction) of mistakes (Clark & Brennan, 1991).

In addition, whereas biosignals in themselves are autonomously generated, and hence can be considered unintentional cues, the choice to explicitly share these signals can be designed as an intentional act in any particular communication system (Janssen et al., 2014) – in our communication framework covered by the dimension of autonomy. Nevertheless, as shown by the scores on this dimension, in most of the developed prototypes, measurement and communication of biosignals is initiated by the system. This reinforces the limited influence we are able to exert on our physiological signals, resulting in a low level of information control (i.e., the extent to which a user can regulate the flow of social information during interactions, Feaster, 2010). In addition to this, the ambiguity and unfamiliarity of these signals introduces a high amount of uncertainty about their meaning and how the receiver interprets them. As a consequence of the aforementioned characteristics, the biosignal communication might turn into unintended self-disclosure of private and sensitive information, which could explain why concerns with self-presentation are amplified when using biosignal sharing systems. It is the lack of control over – and unfamiliarity with – what you reveal about yourself, and more importantly, what it means to the other party, that may be the cause of feelings of discomfort.

Another characteristic that influences how comfortable users feel with biosignal communication is the reciprocity of sharing. Most common communication media, such as telephone and chat, are reciprocal – both parties send and receive – whereas almost half of the developed prototypes in our review were unidirectional. This means that one party sends messages – thereby revealing his or her intimate physiological information – and the other party only receives. Earlier work on awareness systems found that unidirectional systems run the risk of being perceived as monitoring systems, causing users to feel intruded in their privacy (Markopoulos, 2009). In this line, one can argue that biosignal sharing systems in which self-disclosure is non-reciprocal feel even more uncomfortable to senders than reciprocal systems.

Besides, a final factor that influences how comfortable we feel with sharing is the interaction context characteristic. In our review, people were less willing to share their biosignals when the other party was not that well-known, when it concerned communication of intimate emotions and experiences, and when the valence of the shared signals was considered negative. In other words, interaction contexts in which our self-presentation is threatened. According to O’Sullivan (2000), we prefer media channels that allow a high level of information control when threats to our self-presentation arise, in order to preserve a positive self-presentation. As a consequence of the communication characteristics of the reviewed biosignal sharing prototypes, information control is problematic in most of these systems. Therefore, in circumstances in which we feel our self-presentation is particularly at stake, the dimensions of revisability, reviewability, autonomy, reciprocity, copresence, and sequentiality become important and the use of biosignal sharing systems with low scores on these communication dimensions is likely to evoke feelings of discomfort with sharing.

In summary, the communication characteristics of biosignal sharing lead to socially complex interactional flows. Taking heed of these complexities, one could conclude that the most promising opportunity of biosignal sharing is within intimate relationships, where the other party is well-known and trusted, and both parties share a significant amount of common ground and contextual knowledge. Then, biosignal sharing can enhance feelings of connectedness and expression of emotions, especially if communication is synchronous, reciprocal, and intentional (i.e., high autonomy of the sender). Biosignal sharing seems to be perceived more positively when senders can view and control the sharing of their biosignals (revisability, reviewability, and autonomy) and when both parties share their signals (reciprocity). In addition, interpretation can be improved when coordination of turn-taking is supported (sequentiality and/or copresence), when signals can be reviewed to reflect on later (reviewability), or in case of real-time (synchronous) sharing when there is another modality available to provide additional information or to enable immediate repair of potential misunderstandings, for example through face-to-face communication (copresence).

4.2. Considerations for research and design

4.2.1. Research considerations

A general observation in our review is the absence of relevant theoretical perspectives from communication and media psychology literature in current published works on biosignal sharing. A recommendation to come to a more theory-driven, interpersonal-focused approach would be to include details on communication properties in the descriptions of the developed biosignal communication systems, for example by using the framework outlined in this review. Similarly, the use of standardized measures to assess how users experience biosignal communication systems and the employment of standardized quality measures of the technical characteristics of these systems would promote a more uniform evaluation and allow comparisons across different studies and systems.

Moreover, whereas the existing work in this field has yielded a rich dataset of qualitative reports, the underlying mechanisms of biosignal sharing experiences are in general not well understood. To improve our understanding of how, why, and when biosignal sharing affects our social interactions, we need explicit investigations of these underlying processes and how certain system properties lead to particular experiences – studies that will include adequate control conditions so as to be able to identify causal mechanisms. Therein, the communication dimensions in our framework could serve as a set of variables that researchers can specifically target for manipulation, e.g., compare systems that do or do not afford revisability or autonomy on reported willingness to share and feelings of intimacy and connectedness. More work also needs to be conducted on the interpretation of biosignals, and the influence of moderating factors such as the interaction context, the relationship between interactional partners, and the way the data is visualized.

For our review, we used a communication framework primarily based on the work of Clark and Brennan (1991), because this provided us with a practical tool that was particularly useful in our analysis to address our research question regarding the communication characteristics and experiences of biosignal sharing. More generally, this review shows how viewing a new communication medium from the perspective of communication and media psychology can contribute to understanding the communication process in mediated interactions.

4.2.2. Design considerations

From a design perspective, the communication properties can be seen as design dimensions that together make up a design space – a tool for designers that can help them to be more aware of the trade-offs and consequences of the choices they make. In this way, our framework is not only useful to describe and understand systems, but could also serve as a basis for their design; it facilitates designers to make conscious and well-justified decisions regarding the various communication dimensions relevant to the design at hand, so as to optimize the intended characteristics and related experiences of the mediated interaction.

Following our framework of communication dimensions, several specific design considerations can be made, illustrated by some examples of how this could be implemented:

Autonomy: in response to the observed low level of autonomy of the current systems and reported concerns on (lack of) control and self-presentation by users, it is desirable to increase the extent to which users have control over when and how their biosignals are shared. A useful framework that provides several options with varying levels of user control has been presented in the context of awareness systems (Markopoulos, 2009). This ranges from a straightforward switch to turn the display of values on or off to more indirect options such as allowing users to change the granularity and precision of the presented signals: a more abstract representation could help to make biosignals less revealing (Liu et al., 2017b). However, this also compromises interpretability, so this is a trade-off that designers have to be aware of. The highest level of user control presented by Markopoulos (2009) concerns a system with flexible privacy settings, where users can determine what information is captured and with which level of precision and accuracy, possibly even with different settings per interactional partner or groups of partners. Besides increasing information control, such features are also expected to increase the level of intentionality that is conveyed by the shared biosignals (Markopoulos, 2009), enhancing the perceived intimacy and meaningfulness of the message (Janssen et al., 2014; Romero et al., 2007). On the other hand, these options come at the cost of requiring increased effort and complexity of use (Markopoulos, 2009).

Revisability: another way to increase user control is by enabling revisability, i.e., incorporating features to view and possibly adapt the biosignal that is about to be sent, thereby providing the sender with the option to make an informed decision whether or not to communicate the message. A more traditional implementation of this dimension would be to ask for a confirmation to share a specific signal after it is measured, for example through a dialog box. Another implementation would be an override functionality, as was presented by Kuber and Wright (2013), that allows users to modify the signal before sharing. As these features support information control, they are likely to increase feelings of comfort with biosignal sharing, but they also require extra effort of the sender.

Reciprocity: another approach to influence comfort with sharing is by targeting reciprocity. Designers could consider to enforce reciprocal sharing, meaning that users can only receive the other’s biosignals if they share their own. This is in line with the general recommendation to minimize information asymmetry and implement a bidirectional information flow to support personal privacy (Jiang et al., 2002). In some settings, however, unidirectional sharing is more appropriate, such as in therapeutic settings where self-disclosure between interactional partners is inequal on purpose (e.g., Marci & Riess, 2005).

Copresence: even though copresence is not determined by a technical feature, but by how a system is designed to be used, designers should be aware of the cost of missing contextual cues and the ability to provide immediate (non-)verbal feedback in remote communication. Both improve social translucence and enable a faster detection and correction of misunderstandings, and thereby play an important role in facilitating interpretation and mutual understanding (Clark & Brennan, 1991). In remote communication, designers could compensate for this by including a separate channel for users to add contextual information or give feedback (e.g., the option to make video recordings or a comment box). Copresence also has added value in the exploration phase when users learn how to utilize a new system: jointly discovering its functionalities and responses helps to develop new skills and to establish common ground when communicating through this medium.

Synchronicity: our review shows that synchronicity is a great facilitator of feelings of closeness. Designers could reinforce this effect by emphasizing that the data is generated and sent in real-time, for example by a dynamic visualization of a beating heart, or through tangible feedback such as the expanding breathing frame in Kim et al. (2015). In addition, synchronicity provides information about the context of a biosignal: knowing exactly when it was sent, supports its interpretation and mutual understanding, a frequently reported need we found in our review. At the same time, synchronicity is also associated with an increased risk of misunderstandings, because turns are more likely to get out of sequence, especially if no elements are implemented to prevent this (see next point).

Sequentiality: mutual understanding can be supported by designing systems that afford sequentiality, as this facilitates interactional partners in clarifying the structure of messages and how they refer to each other. In the few prototypes that afforded sequentiality in the current review, this was generally implemented by presenting the physiological measurements paired to time stamps or particular text messages, (e.g., DiMicco et al., 2002; Buscheck et al., 2018). Another example of how turn-taking in mediated interactions can be supported is by adopting a quoting strategy (Herring, 1999). This is implemented in popular text messaging applications with a so-called ‘quote-reply’ function, where you can copy the signal that you respond to by tapping on it and then write your response.

Reviewability: accurate mutual understanding is also facilitated by affording reviewability, as one can track which biosignals have been sent and where misunderstandings have entered the conversation. Additionally, it enables post-hoc reflection and discussion between interactional partners. In general, one can afford reviewability by providing retrospective access to data and the ability to retrieve signals from a previous period. The few prototypes in our analysis that afforded reviewability mainly implemented this by showing a graph or list of measurements of the physiological signal over a specified period of time. This way of affording reviewability is generally meant to support reflection and understanding. A different approach was presented by Howell et al. (2019) with the Heart Sound Bench, where users had the option to record their heart rate while sitting on the bench, after which the sound could be replayed when another user sat on the bench later. In this way, receivers felt connected to the sender, even though there was a delay between recording and listening.

The current review finds little attention for ethical issues in designing the biosignal sharing systems, such as considerations on the privacy of users, ownership of the physiological data, or possible adverse effects of sharing biosignals. In general, authors of the works reviewed in this paper have focused primarily on the potential benefits of biosignal sharing, without addressing potential costs in psychosocial and ethical terms. When biosignal sharing scales to become a more common way of enriching or enhancing interpersonal communication, the intimate and private nature of our physiology demands careful consideration of alternate uses and unintended consequences that could be harmful to users. The potential severity of the consequences underlines the urgency of developing a framework for a systematic ethical assessment. This call has been made for specific fields such as health technology (Heintz et al., 2015) and persuasive technologies (Verbeek, 2006), resulting in frameworks with a general approach (e.g., ethical-constructive technology assessment, Kiran et al., 2015, and ethical impact assessment; Wright, 2011). In addition, the Value-Sensitive Design (VSD) framework (Friedman et al., 2013), and its extensions (e.g., Borning & Muller, 2012; Jacobs & Huldtgren, 2018) offer methods of impact assessment that are valuable in the current context. These frameworks provide fruitful points of departure for the development of an ethics framework for biosignal sharing systems. Developing such a framework is beyond the scope of the current paper, but the authors are of the opinion that this work requires urgent attention, in a multidisciplinary collaboration between technology developers, interaction designers, behavioral scientists, and ethics experts.

5. Conclusion

There is a growing interest in biosignal communication with a range of possible benefits in social interaction. This review has shown that these benefits include enhanced intimacy, connectedness, empathy, enjoyment, and engagement. But there are also multiple fundamental challenges that need to be addressed. As the current review shows, people have mixed feelings about sharing biosignals: it is part of an intricate interactional flow, with issues of ambiguity, uncertainty, concerns over self-presentation and privacy, and strong contextual effects modulating the potential beneficial effects. This urges us to remain critical in appraising for which purposes and under which conditions biosignal sharing truly adds value, instead of merely being something that is technically feasible. Additionally, in order to address the challenges identified in this review, there is a need to increase our understanding of their underlying factors and processes, and the various ways in which we make sense of biosignals.

To date, a deeper understanding of the communication characteristics and experiences of biosignal-mediated social interactions was lacking. By conceptualizing biosignal sharing as a communication medium and viewing this new medium along a communication framework, the current review demonstrates the applicability and added value of a more structured, theory-driven approach in designing biosignal communication media, and in understanding the relation between their media properties and the interpersonal communication experiences they engender.

Declaration of interest statement

No conflicts of interest to declare.

A2. Scopus

(TITLE(*feedback OR shar* OR interpersonal OR interaction OR communication OR “social cue” OR synchroni?ation) OR KEY(*feedback OR shar* OR interpersonal OR interaction OR communication OR “social cue” OR synchroni?ation)) AND (TITLE(physiol* OR *sympath* OR skin$conductance OR gsr OR eda OR hr OR hrv OR eeg OR ecg OR respirat* OR breath* OR bio$signals OR heart* OR electrocardio* OR electromyo* OR electroenceph* OR wearables OR arousal) OR KEY(physiol* OR *sympath* OR skin$conductance OR gsr OR eda OR hr OR hrv OR eeg OR ecg OR respirat* OR breath* OR bio$signals OR heart* OR electrocardio* OR electromyo* OR electroenceph* OR wearables OR arousal)) AND (LIMIT-TO (DOCTYPE,”ar “) OR LIMIT-TO (DOCTYPE,”re “) OR LIMIT-TO (DOCTYPE,”cp “)) AND (LIMIT-TO (SUBJAREA,”SOCI “) OR LIMIT-TO (SUBJAREA,”MEDI “) OR LIMIT-TO (SUBJAREA,”PSYC “) OR LIMIT-TO (SUBJAREA,”ENGI “) OR LIMIT-TO (SUBJAREA,”ARTS “) OR LIMIT-TO (SUBJAREA,”COMP “) OR LIMIT-TO (SUBJAREA,”BUSI “) OR LIMIT-TO (SUBJAREA,”NEUR “) OR LIMIT-TO (SUBJAREA,”NURS “) OR LIMIT-TO (SUBJAREA,”HEAL “) OR LIMIT-TO (SUBJAREA,”MULT “)) AND (LIMIT-TO (LANGUAGE,”English “)) AND (EXCLUDE (SUBJAREA,”BIOC”) OR EXCLUDE (SUBJAREA,”PHAR”) OR EXCLUDE (SUBJAREA,”IMMU”) OR EXCLUDE (SUBJAREA,”AGRI”) OR EXCLUDE (SUBJAREA,”CENG”) OR EXCLUDE (SUBJAREA,”MATE”) OR EXCLUDE (SUBJAREA,”MATH”) OR EXCLUDE (SUBJAREA,”PHYS”) OR EXCLUDE (SUBJAREA,”ENVI”) OR EXCLUDE (SUBJAREA,”CHEM”))

A3. Web of Science

(TI=(physiol* OR *sympath* OR skin$conductance OR gsr OR eda OR hr OR hrv OR eeg OR ecg OR respirat* OR breath* OR bio$signals OR heart* OR electrocardio* OR electromyo* OR electroenceph* OR wearables OR arousal)) AND LANGUAGE: (English) AND DOCUMENT TYPES: (Article)

Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, ESCI Timespan=All years

AND

(TI=(*feedback OR shar* OR interpersonal OR interaction OR communication OR “social cue” OR synchroni?ation)) AND LANGUAGE: (English) ANDDOCUMENT TYPES: (Article)

Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, ESCI Timespan=All years

Refined by: RESEARCH AREAS: (PSYCHOLOGY OR ENGINEERING OR SCIENCE TECHNOLOGY OTHER TOPICS OR PSYCHIATRY OR COMPUTER SCIENCE OR BEHAVIORAL SCIENCES) AND [excluding] RESEARCH AREAS: (BIOPHYSICS OR MATHEMATICAL COMPUTATIONAL BIOLOGY OR MATERIALS SCIENCE OR MATHEMATICS OR PHYSICS OR PHARMACOLOGY PHARMACY OR BIOCHEMISTRY MOLECULAR BIOLOGY OR CHEMISTRY OR SUBSTANCE ABUSE OR TRANSPLANTATION OR ZOOLOGY) AND [excluding] RESEARCH AREAS: (CARDIOVASCULAR SYSTEM CARDIOLOGY OR MEDICAL LABORATORY TECHNOLOGY OR ENDOCRINOLOGY METABOLISM OR RADIOLOGY NUCLEAR MEDICINE MEDICAL IMAGING OR AGRICULTURE OR AUTOMATION CONTROL SYSTEMS OR SURGERY OR VETERINARY SCIENCES OR CONSTRUCTION BUILDING TECHNOLOGY) AND [excluding] RESEARCH AREAS: (ENVIRONMENTAL SCIENCES ECOLOGY OR DEVELOPMENTAL BIOLOGY OR FOOD SCIENCE TECHNOLOGY OR MECHANICS OR OPHTHALMOLOGY) AND [excluding] RESEARCH AREAS: (RESEARCH EXPERIMENTAL MEDICINE OR BIOTECHNOLOGY APPLIED MICROBIOLOGY OR CRIMINOLOGY PENOLOGY OR OCEANOGRAPHY OR PUBLIC ENVIRONMENTAL OCCUPATIONAL HEALTH) AND [excluding] RESEARCH AREAS: (TOXICOLOGY OR ACOUSTICS OR ANTHROPOLOGY) AND [excluding]RESEARCH AREAS: (CELL BIOLOGY OR ENERGY FUELS OR FISHERIES OR GEOLOGY OR HISTORY PHILOSOPHY OF SCIENCE OR IMMUNOLOGY OR MARINE FRESHWATER BIOLOGY OR MUSIC OR NUTRITION DIETETICS OR OPERATIONS RESEARCH MANAGEMENT SCIENCE OR ROBOTICS OR WATER RESOURCES

Additional information

Funding

This work was supported by the Dutch Research Council (NWO) [grant number 055.16.141], project title Psychosocial Games Supporting Mental Health Professionals in Stress Reduction and Empathic Interactions in Remote/online Psychotherapy, under the Serious Games for Professional Skills Program.

Notes on contributors

Milou A. Feijt

Milou A. Feijt ([email protected]) is a Ph.D. candidate at Eindhoven University of Technology, the Netherlands. Her research project focuses on professionals’ adoption of technology in mental healthcare and investigates opportunities for novel technologies such as biosignal sharing to facilitate empathic interactions.

Joyce H.D.M. Westerink

Joyce H.D.M. Westerink ([email protected]) is principal scientist at Philips Research and a full professor at Eindhoven University of Technology, both in the Netherlands. Her key areas of expertise include psychophysiology, user interfaces, human-computer interaction, and stress.

Yvonne A.W. De Kort

Yvonne A.W. de Kort ([email protected]) is a full professor at Eindhoven University of Technology, the Netherlands. She is an environmental psychologist, focusing on the interplay between individuals and their surroundings. In particular, she investigates the effects of light and natural views on human functioning.

Wijnand A. IJsselsteijn

Wijnand A. IJsselsteijn ([email protected]) is a full professor at Eindhoven University of Technology, the Netherlands. He has an active research program on the impact of media technology on human psychology, and the use of psychology to improve technology design, with a specific interest in affective computing, artificial intelligence, mediated social touch, personal informatics, and virtual reality.

Notes

¹ In the context of this review ‘raw’ refers to a transformation without further conversion or interpretation by the developers, staying close to the original nature of the signal, such as a vibration with every heartbeat

Appendix

Search queries

A1. ACM

(acmdlTitle:(biofeedback OR shar% OR interpersonal OR interaction OR communication OR “social cue” OR synchroni_ation) OR keywords.author.keyword:(biofeedback OR shar% OR interpersonal OR interaction OR communication OR “social cue” OR synchroni_ation)) AND (acmdlTitle:((physiol% OR sympath% OR skin_conductance OR gsr OR eda OR hr OR hrv OR eeg OR ecg OR respirat% OR breath% OR bio_signals OR heart% OR electrocardio% OR electromyo% OR electroenceph% OR wearable% OR arousal)) OR keywords.author.keyword:(physiol% OR sympath% OR skin_conductance OR gsr OR eda OR hr OR hrv OR eeg OR ecg OR respirat% OR breath% OR bio_signals OR heart% OR electrocardio% OR electromyo% OR electroenceph% OR wearable% OR arousal))