Unwind: a musical biofeedback for relaxation assistance

ABSTRACT

Unwind is a musical biofeedback interface which combines nature sounds and sedative music into a form of New-Age music for relaxation exercises. The nature sounds respond to the user’s physiological data, functioning as an informative layer for biofeedback display. The sedative music aims to induce calmness and evoke positive emotions. UnWind incorporates the benefits of biofeedback and sedative music to facilitate deep breathing, moderate arousal, and promote mental relaxation. We evaluated Unwind in a 2 × 2 factorial experiment with music and biofeedback as independent factors. Forty young adults performed the relaxation exercise under one of the following conditions after experiencing a stressful task: Nature sounds only (NS), Nature sounds with music (NM), and Auditory biofeedback with nature sounds (NSBFB), and UnWind musical biofeedback (NMBFB). The results revealed a significant interaction effect between music and biofeedback on the improvement of heart rate variability. The combination of music and nature sounds also showed benefits in lowering arousal and reducing self-report anxiety. We conclude with a discussion of UnWind for biofeedback and the wider potential of blending nature sounds with music as a musical interface.

KEYWORDS:

• Biofeedback
• relaxation
• sonification
• musical display
• heart rate variability

1. Introduction

Biofeedback is a mind–body technique through which individuals learn how to control specific bodily processes to improve physical, mental, and emotional health (Frank et al. 2010). Biofeedback instruments measure one’s physiological processes or states (e.g. heart rate, respiration, skin conductance) and transform the biofeedback data into a visual or auditory representation. In mind–body practices (Schwartz and Andrasik 2016), the biofeedback information assists users in self-regulation. Through trial and error, the users improve their self-regulation skills to cope with a range of health problems, such as hypertension (Greenhalgh, Dickson, and Dundar 2010), headache disorders (Nestoriuc et al. 2008), anxiety (Moore 2000), and stress (Reiner 2008). In recent years, biofeedback has been widely used for relaxation assistance. For instance, the studies by Bouchard et al. (2012), Blumenstein et al. (1995), Wells et al. (2012), Zucker et al. (2009) have shown that biofeedback-assisted relaxation exercise can help users reduce stress and increase the effectiveness of physical and mental relaxation.

In everyday life, music is a source of pleasure for many of us. Numerous studies show that music listening may influence the heart activity (Iwanaga, Kobayashi, and Kawasaki 2005), blood pressure (Chafin et al. 2004), respiration (Bernardi et al. 2009) and physiological arousal (Rickard 2004). A fast and dynamic musical piece tends to have an excitative effect while a melodious and slow one has a sedative effect. Music can be a good medium to help people regulate emotion and boost mood. Pelletier (2004) suggests that listening to relaxing music helps the listener calm down and moderate arousal. Labbé et al. (2007) and Iwanaga et al. (1996) indicate that sedative music can reduce psychological anxiety and physiological stress. Calming, relaxing, and sedative music are also frequently used in relaxation exercise as a background accompaniment (see, e.g. Robb [2000]). The anxiolytic and therapeutic effects of music in stress management and music therapy have been widely studied and documented (see, e.g. Mandel [1996]).

Music is often used in biofeedback interfaces due to its ability to present information and induce calmness. Musical biofeedback interfaces can be broadly divided into two categories. The first is by modulating musical parameters with biofeedback data. For instance, Yokoyama et al. (2002) and Bergstrom, Seinfeld, and Arroyo-Palacios (2014) presented two similar musical interfaces, which represent heart rate data by modulating the tempo, pitch or volume of a melody. In the other category, the interfaces modify the sound effects of a pre-selected musical piece for biofeedback display. For instance, Harris et al. (2014) developed a musical biofeedback system that encourages slow breathing by adjusting the quality of a music recording in proportion to the user’s respiration rate. Similarly, Bhandari et al. (2015) developed a respiratory biofeedback system which presents the user’s breathing signal by modifying the intensity of the white noise that blends into a piece of music.

In our early explorations on auditory biofeedback (Yu et al. 2015), we have tried to present biofeedback data in the form of music. We developed an auditory display of heart rate variability (HRV) by mapping the timing variations of heartbeats to musical rhythms. The results of the user study demonstrated the effectiveness of the developed auditory interface for information display, but also revealed some challenges regarding user experience. For instance, some participants reported an increased psychological burden and anxiety due to the fast changes in the musical rhythm. The repeated musical expression quickly led to a feeling of boredom and tiredness.

The audio outputs of most musical biofeedback systems still sound quite different from those musical pieces that are well-arranged by a composer. According to Hermann (2008), the sonification approaches in those musical interfaces mostly belong to ‘parameter-mapping’, where the biofeedback data are directly mapped to musical or acoustic parameters, e.g. volume, pitch, tempo, or noise ratio. The musical structure has rarely been addressed as a means of auditory information display. Real-time music notation techniques (also referred to as dynamic musical scores) emerge recently (Freeman and Colella 2010) and offer a way to get into the essence of the music and create a melodious musical representation of data. As the real-time notation is a relatively new field, few tools are available for non-composers. Moreover, how to address the aesthetic of musical expressions might be another challenge for the designers who have little music knowledge and skills.

Figure 1. The concept of UnWind musical biofeedback.

New-age music is a genre of music that is intended to create artistic inspiration, relaxation, and optimism (Smith and Joyce 2004). It may create a peaceful atmosphere for reading, yoga, meditation, and relaxation. Nature sounds are a common integrant in new-age music. For instance, Dean Evenson, 1979, combined his peaceful flute music with nature sounds. A recent example could be Matthew Lien’s album, 2014, ‘Headwaters—Music of the Peel River Watershed’ in which the sounds of musical instruments and water were composed and assembled into a piece of music. Goel and Etwaroo (2006) suggested that listening to birdsong accompanied by music could reduce self-reported negative affect. This combination of nature sounds and smooth music has also been demonstrated to be effective in decreasing anxiety (Cutshall et al. 2011).

This paper presents the design and evaluation of UnWind, a musical interface for a HRV biofeedback system (Figure 1). UnWind aims to facilitate users’ breathing regulation in a relaxation exercise and foster a relaxing experience through music listening. The novelty of UnWind lies in presenting biofeedback data through a new form of music which combines a nature soundscape with sedative music. Different from other musical biofeedback displays (e.g. Yokoyama et al. 2002; Bergstrom, Seinfeld, and Arroyo-Palacios 2014; Harris et al. 2014; Bhandari et al. 2015), this combination enables UnWind to be informative for biofeedback, but also sound like a piece of well-composed music. In the experiment, we examined the possible effects arising from this combination, and investigated the effectiveness of UnWind for relaxation assistance, regarding optimising the breathing pattern, enhancing HRV, moderating arousal level, and reducing subjective anxiety.

2. Related work

2.1. HRV biofeedback

HRV refers to the change in the time intervals between adjacent heartbeats. Heart rate is regulated by the autonomic nervous system. The balance between its two divisions, sympathetic (accelerating) and parasympathetic (decelerating) systems, produces a complex and ongoing pattern of HRV (Sztajzel 2004). In long-term ambulatory recordings, HRV reflects the sympathetic activities (Malliani et al. 1991), which are related to the bodily stress responses, mental workload, and emotional responses (Appelhans and Luecken 2006). As demonstrated by Nolan et al. (2005), immediate feedback of HRV data would help users improve awareness of their stress responses and actively regulate their recovery after stress exposure.

Heart rate also varies in synchrony with respiration, by which inter-beat intervals (IBI) are shortened during inspiration and prolonged during expiration (Lehrer, Vaschillo, and Vaschillo 2000). The changing IBI can be modulated into a stable and sine-wave-like pattern by deep breathing. Therefore, the feedback of IBI data in the relaxation exercise helps users regulate the breathing pattern to their unique ‘resonant frequency’, at which the amplitude of HRV is maximised (Nolan et al. 2005). When an individual breathes at his/her ‘resonant frequency’, real-time heart rate and respiration co-vary in a perfect phase relationship so that users inhale until their heart rate peaks and exhale until it begins to rise again (Vaschillo et al. 2004). The relaxation exercise with ‘resonant breathing’ could stimulate the baroceptors, increase vagal activity, and strengthen the overall capacity of the body’s homeostatic functioning (Lehrer, Vaschillo, and Vaschillo 2000).

2.2. Relaxation with sedative music

Sedative music mainly refers to those pieces of melodic and soothing music that have a flowing melody with few major changes in pitch, dynamics, or rhythm. According to Iwanaga and Moroki (1999), sedative music is characterised by a slow tempo of 60–80 beats per minute (bpm), which is similar to the resting heart rate. Sedative music is widely used in music therapy. For instance, Voss et al. (2004) and Lorch et al. (1994) showed that heart rate and blood pressure were decreased by sedative music. A wide variation of music has been selected as the examples of sedative music for the experiment in the previous studies. A collection of new-age music, classical piano, slow modern jazz, and American Indian flute music have been selected by Voss et al. (2004). Besides, Lingham and Theorell (2009) suggested a sedative collection with a broader range: baroque string music, Greek vocal music, relaxing Tibetan music, relaxing jungle music, lullaby music, Pink Floyd (‘Wish you were here’), and Dolly Parton (‘I will always love you’). As some classical music is also melodious, delicate, soft, and beautiful, this is also regarded as sedative music and used in music therapy, such as Erik Satie’s Gymnopedie No.1 used by Iwanaga, Ikeda, and Iwaki (1996) and Beethoven’s Moonlight Sonata used by Lorch et al. (1994).

2.3 Relaxing with nature sounds

In addition to the well-composed sedative music, nature sounds have also been studied and applied to the stress relief and relaxation exercise. Many types of nature sounds are typically perceived as pleasant components of an acoustic environment and used for coping with stress. For instance, Benfield et al. (2014) demonstrated that the sound of birds and rain have a positive emotional effect. A mixture of sounds from a fountain and tweeting birds have also shown stress-relieving effects via the autonomic nervous system (Alvarsson, Wiens, and Nilsson 2010). The research by DeLoach, Carter, and Braasch (2015) demonstrated that the exposure to nature sounds improved the listener’s moods and the ability to focus. Also, nature sounds, such as the sounds of rain, fire, and wind, are also used to aid in focus or sleep (Stanchina et al. 2005).

In our previous work (Yu et al. 2016), a user study was conducted to investigate the user acceptance of different auditory contents for relaxation. The participants were exposed to 15 sound samples in different sound categories, namely classical music, ambient music, white noise, nature sounds, and the combination of nature sounds and music. The results suggested the combination of music and nature sounds could be a well-accepted auditory form to induce subjective relaxation experience. The participants indicated that the natural sounds could quickly visualise a nature scene in their mind. The imaginary nature scenes made them feel relaxed as if they were ‘on vast grasslands’ or ‘in a verdant forest’. Moreover, the music which goes with nature sounds further evoked positive memories, visual imageries, and emotions. The findings from this study mainly inspire the design of UnWind.

2.4. Biofeedback display with nature sounds

Nature sounds are among ‘everyday sounds’ around us. When we are outdoors in a forest or a garden, we hear the birds singing or the murmur of a distant brook. Nature sounds can not only be used to foster the experience of calmness and relaxation but also to present information in auditory interfaces. As the nature sounds are intuitive, familiar, and may be understood quickly and learned easily, they are often used in ambient displays and peripheral interactions by creating a ‘calm’ sonic environment. For instance, Eggen and Van Mensvoort (2009) used bird sounds in a peripheral display to communicate information about the human activities in the office. AmbientROOM by Ishii et al. (1998) modulated the volume and density of bird and rainfall sound to present the number of unread email messages and the value of a stock portfolio.

In our previous work (Yu et al. 2017), we have developed a model of nature soundscape (NS) through an empirical study. The NS model helps select and organise the sounds within a ‘nature theme’, and provides a means to manipulate the acoustic parameters of certain nature sounds, generating an informative ‘soundscape’ with calmness and pleasantness. The NS model allows presenting information by individual nature sounds and also the attributes of the overall soundscape. Based on the proposed NS model, we explored auditory biofeedback with a synthesised nature soundscape responding to the user’ physiological data including respiration, HRV and arousal activities. In the evaluation, several participants suggested blending the NS-based auditory biofeedback with a piece of music to further enhance the relaxing experience. We viewed this suggestion as a new direction for designing musical biofeedback with no need to deal with the musical structure. From that point on, we stepped up to create UnWind.

3. Design of unwind

3.1. System structure

The Unwind program can be divided into three parts: data processing procedure, sound synthesis procedure, and sounds library (indicated with different colours with the dotted box in Figure 2). Firstly, the blood volume pulse (BVP) signal is measured by a photoplethysmogram (PPG) sensor on the finger. The BVP signal is transmitted from the biosensing device to the Unwind program. In the data processing procedure, the IBI and index of HRV are calculated as the biofeedback data. In the sound synthesis procedure, the biofeedback data are mapped to the parameters of the synthesiser to control the audio output in real-time. The Unwind program is implemented on the Processing platform with the Minim Java audio library.

Figure 2. The structure of UnWind biofeedback system.

3.2. Unwind musical interface

The structure of Unwind musical interface is shown in Table 1. It is composed of two layers: nature sounds (NS) layer and sedative music layer. The nature sounds serve as the ‘information layer’ for biofeedback display, while the sedative music serves as the ‘background layer’ for promoting relaxation. Nature sounds shape an acoustic nature environment, where the users can retrieve the biofeedback information based on their perception towards the soundscape. The background sedative music is expected to induce calmness and evoke positive emotions. According to Iwanaga, Ikeda, and Iwaki (1996), in this study, we selected the original piano version of Erik Satie’s Gymnopedie No.1–No.3 and Gymnopedie No.1–No.3 as the sedative collection.

Table 1. The structure of the UnWind musical interface.

Layers	Sub-layers	Controllable parameters	Sound selections for the study
Nature sounds (NS)	Biological sound (Bs)	Density	silvereye, wren, greenfinch, collared dove, cuckoo, owl
		Variation
		Volume
	Geophysical sound (Gs)	Volume	Water stream
	Climatic sound (Cs)	Volume	Wind sound
Sedative music		Volume	Erik Satie’s Gymnopedie No.1-No.3 Gnossienne No.1-No.3.

The nature soundscape that arises from a real landscape tends to be very complex. It is challenging to modulate a real recording of nature soundscape for information display. As suggested by our previous study (Yu et al. 2017), the NS model could help in selecting and mixing various nature sounds so that the resulting soundscape can be perceived as a harmonious sonic environment. For UnWind interface, we selected the nature sounds from the recordings of a real forest as the auditory contents. As shown in Table 1, the NS layer is composed of three sub-layers: wind sound for the climatic sound (Cs), a water stream for geophysical sound (Gs), and several types of birds (i.e. silvereye, wren, greenfinch, collared dove, and cuckoo) for biological sounds (Bs). Each sublayer has at least one parameters to be modulated. With the NS model, the biofeedback data can be mapped to multiple parameters jointly and modulate the perceptual attributes of the overall soundscape, such as quietness, and richness.

3.3. Biofeedback mapping

Figure 3 shows the mapping from the data space to the sound space. The measured BVP signal is processed with a peak detection algorithm into the time series of inter-beat Intervals IBI data, which are directly mapped to the increase and decrease of wind volume. When an individual inhales, the IBI data decrease, the wind becomes quiet. Conversely, on exhale, the air is expelled from lungs, and accordingly, the wind becomes loud. We assume that the audio display of the IBI wave with the wind sound is intuitive to perceive and understand. With the immediate feedback of IBI data, the user can learn to regulate his/her breathing pattern by modifying the IBI wave into an approximate sinusoidal form.

Figure 3. Mapping from bio-data to the nature sounds and listener’s perception.

The short-term HRV is highly related to the respiratory cycle of an individual (Vaschillo et al. 2004). In this study, we use the short-term HRV to indicate the results of the breathing regulation in relaxation exercises. The feedback of HRV informs the user about the results of relaxation exercise and encourages breathing regulation. A specially modified form of standard deviation of IBI data (SDNN) is calculated with a moving window of 16 heartbeats as the index of HRV. There are two reasons for using a window of 16 beats. First, the window size needs to be large enough to include at least one complete respiratory cycle for filtering out the cycle phases (inhale/exhale). Second, the window size should be small to be sensitive to changes in breathing pattern. A normal resting heart rate for adults ranges from 60 to 100 beats per minute (Agelink et al. 2001). The optimal respiratory rate (resonance frequency [Vaschillo, Vaschillo, and Lehrer 2006]) in relaxation exercise may range from 4.5 to 7 breaths per minute. Here we selected a modest size of 16, the quotient in which 90 beats is divided by 5.5 breaths. The SDNN₁₆ is calculated with the following formula:

The SDNN₁₆ value is updated with each heartbeat. UnWind presents the SDNN₁₆ through the quietness and richness of the nature soundscape. When the SDNN₁₆ is increased, the volumes, density, and type variations are reduced accordingly. Thus, when the users perform deep breathing in a relaxation exercise, especially close to their resonant frequency, the increased HRV (SDNN₁₆) will shape a nature soundscape that can be perceived quiet, pure, and simple.

4. Evaluation

4.1. Subjects

Forty young adults (22 females, 18 males, age range: 20–30 years) participated in the study through informed consent procedures. Participants were compensated with 5 euros for their participation. The participants did not practice yoga, meditation, or deep breathing exercise regularly. Furthermore, the participants did not have any experience with biofeedback.

4.2. Experimental design

The experiment was aimed to 1) investigate the viability of UnWind as a musical interface for biofeedback display and 2) to examine the possible effects arising from the combination of biofeedback and sedative music. We evaluated Unwind in a 2 × 2 factorial experiment with sedative music and biofeedback as independent factors; each factor has two levels: presence or absence (see Table 2). The participants were randomly assigned to one of the four conditions: listening to pre-recorded nature sounds (NS), listening to pre-recorded nature sounds with music (NM), with auditory biofeedback with nature sounds only (NSBFB), and with musical biofeedback through the combination of nature sounds and music (NMBFB). In all conditions, the participants performed a 10-minute relaxation exercise after a 10-minute mentally challenging task. In non-biofeedback conditions (NS, NM), the participants were exposed to a piece of pre-recorded nature sounds with or without music layer; the nature sounds were controlled by the programme to shape a quiet and simple nature soundscape with a repetitive wind movement of a random cycle from 6 to 10 seconds. In biofeedback conditions (NSBFB, NMBFB), the nature sounds were controlled by the participants’ real-time IBI and HRV data.

Table 2. The 2 × 2 factorial experiment design.

Factor level	Non-Biofeedback	Biofeedback
No music	NS (Pre-recorded nature sounds)	NSBFB (Biofeedback through nature sounds)
With music	NM (Pre-recorded nature sounds with sedative music)	NMBFB (Biofeedback through nature sounds, combining sedative music as background layer)

4.3. Biofeedback protocol

In all conditions, the participants were suggested to relax by deep breathing in the relaxation exercise. In non-biofeedback conditions (NS, NM), the participants performed deep breathing on their own without the feedback of IBI and HRV data. In biofeedback conditions (NSBFB, NMBFB), the participants were exposed to the audio display which responds to their IBI and HRV data. They were informed that the wind sound would increase and decrease with their breathing and when they perform well in the relaxation exercise, the soundscape would become quiet and simple. The instruction given was ‘Please relax with deep breathing. The wind sound indicates your breathing. When you are more relaxed and calm, the soundscape will become quieter and more simple’.

4.4. Measurements

Psychological measures included the participant’s self-reports on relaxation and anxiety. The stress/anxiety level was measured by Relaxation Rating Scale (RRS) and State-Trait-Anxiety-Inventory, State-subscale (STAI-S). The RRS is a simple self-reported instrument that is used to assess the degree of subjective relaxation experience (Lesage, Berjot, and Deschamps 2012). The RRS only has one question that requires the participant to rate his/her level of relaxation on a Likert-type scale with one being ‘not relaxed at all’ and nine being ‘totally relaxed’. The higher score indicates that the participant is more relaxed. The STAI-S is a 20-item self-report survey, which requires an individual to rate how he/she feels ‘at this moment’. Higher scores indicate a high level of anxiety (Spielberger 2010; Prinsloo et al. 2013).

Physiological measurements included average heart rate (HR), HRV (LF%), respiration rate, and skin conductance responses. The bio-signals were recorded using a NeXus-10 system (MindMedia, the Netherlands). The average HR and HRV (LF%) are measured by the same PPG sensor used for the UnWind biofeedback system. When a user places the finger on the PPG sensor, the BVP signal is detected by illuminating the skin with the light from a LED and then measuring the amount of light reflected to a photodiode. The sample rate is 500 Hz. The IBI data were extracted from the BVP waveform and then analysed in Kubios software to obtain the average HR and HRV index, namely the percentage of power in the low frequency ranging from 0.04 Hz to 0.15 Hz, (LF%). One point should be noted that SDNN₁₆ was calculated in real-time for biofeedback in the UnWind and the LF% was analysed with Kubios for HRV analysis in post hoc. The respiration trace was measured by a strap-type respiration sensor that was placed at the abdominal position. The strap was adjusted so that there was a slight tension when the participant fully breathed out. Respiration rate (RSP-R) (cycles per minute) was then derived from the respiration trace. The electrodes of the skin conductance sensor were strapped around the finger pads of the middle and ring fingers. The sensors were placed on the palm side of the fingers. The skin conductance (SC) data were recorded with a sampling rate of 256 Hz. In Ledalab software, SC data are deconvolved by the general response shape, which results in a large increase of temporal precision. Then the continuous decomposition analysis (CDA) performs a decomposition of SC data into continuous signals of phasic and tonic activity (Benedek and Kaernbach 2010). We calculated the amount of skin conductance response (SCRs) with the phasic signal, as one index of arousal.

A follow-up interview was conducted at the end of the experiment. The interview was loosely structured with a focus on three questions: ‘What did you like about the experience? What did you not like about the experience? Any other comments?’. The interview data are used to support the interpretation of the quantitative data and provide indications of psychological states.

4.5 Procedure

The experiment followed the procedure shown in Figure 4. The participants were assigned to one of the four conditions by a computer-generated list of random numbers. The experiment consisted of a baseline session, stress session, and a relaxation session. During the baseline session, the participants sat quietly on the chair and relaxed with eyes closed for 10 minutes. During the stress session, the participants completed a mentally challenging task which consists of a Stroop colour-word test and a Mirror-tracing test. During the relaxation session, the participants were encouraged to perform a relaxation exercise with/without biofeedback assistance. After each session, the participants completed the STAI and RRS surveys. Finally, all physiological sensors were detached, and the follow-up interview was conducted. All participants were tested individually in a small testing room furnished with a recliner chair, rug, lamps, and biofeedback equipment. In all conditions, the participants wore an acoustic noise-cancelling headphone (Bose, QuietComfort 25) to block the noise in the environment and listen to the music.

Figure 4. The procedure of the experiment.

5. Results

Within each condition, we compared the measures between the stress session and relaxation session by using a paired t-test. The results are shown in Table 3. Then for each condition, we calculated the percent changes of the measures in relaxation session (relative to their levels during the stress session) and used ANOVA to test which of the conditions worked the best and to test whether the differences between the conditions are significant. Finally, a two-way ANOVA (General Linear Model/ univariate analysis in SPSS) was conducted to compare the main effects of music and biofeedback and the interaction effect between music and biofeedback on the relaxation exercise.

Table 3. The results of the measures in four conditions.

		NS		NM		NSBFB		NMBFB
		Stress	Relaxation	Stress	Relaxation	Stress	Relaxation	Stress	Relaxation
HR	Mean	74.88	71.76	77.69	73.19	74.11	70.45	73.88	68.39
	Std.D	11.71	14.91	5.77	7.76	10.08	11.71	10.74	9.37
	Std.E	3.70	4.72	1.82	2.45	3.19	3.72	3.39	2.96
	Sig	0.057		0.007		0.12		0.001
SCRs	Mean	7.04	3.78	6.67	3.19	7.50	3.99	5.36	1.58
	Std.D	3.34	1.87	3.05	2.03	3.52	2.99	2.20	1.38
	Std.E	1.05	0.59	0.96	0.64	1.11	0.94	0.69	0.44
	Sig	0.002		0.000		0.000		0.001
LF%	Mean	58.81	58.89	60.39	53.47	65.38	74.72	54.32	81.13
	Std.D	10.50	18.67	14.11	20.63	10.39	20.69	20.28	10.69
	Std.E	3.32	5.90	4.46	6.52	3.28	6.54	6.41	3.38
	Sig	0.988		0.247		0.134		0.011
RSP-R (cycle per minute)	Mean	20.16	17.05	20.43	14.41	21.44	7.28	21.27	8.98
	Std.D	2.36	4.3	2.97	4.55	5.43	2.49	5.76	3.91
	Std.E	0.75	1.47	0.94	1.44	1.72	0.79	1.82	1.24
	Sig	0.021		0.002		0.000		0.000
RRS (score)	Mean	4.10	6.20	4.7	7.7	3.4	7.0	4.20	7.50
	Std.D	1.79	1.81	1.89	1.05	1.65	1.05	1.39	1.50
	Std.E	0.57	0.57	0.59	0.34	0.52	0.33	0.44	0.47
	Sig	0.014		0.000		0.000		0.000
STAI (score)	Mean	43.10	36.50	39.70	28.90	46.20	35.40	49.9	32.5
	Std.D	7.92	7.18	7.62	5.63	9.73	7.73	11.05	10.67
	Std.E	2.51	2.27	2.40	1.78	3.07	2.45	3.49	3.37
	Sig	0.009		0.001		0.000		0.001

5.1. Physiological measures

5.1.1. Heart rate (HR)

As shown in Table 3, heart rate for the participants in all conditions decreased during the relaxation session (NM, −5.9 ± 5.4%; NMBFB, −7.3 ± 3.5%; NS, −4.8 ± 7.1%; NSBFB, −4.9 ± 9.1%). The HR decrease was only significant in two music conditions (p < .01), which suggests that the sedative music has the potential to reduce the cardiac responses to stressors. Figure 5 shows the percent decrease in HR during the relaxation session (relative to their levels during the stress session) for each of the four conditions. There was no significant difference between conditions as determined by one-way ANOVA (F(3, 36) = 0.282, p = .838).

Figure 5. Simple Boxplot of percent changes of the Heart Rate (HR) in four conditions.

5.1.2. Skin conductance responses (SCRs)

The skin conductance signal constantly changes within an individual respondent, depending on their hydration, skin dryness, or autonomic regulation. Normally, in the resting period after the stress, the arousal activities (SCRs) will gradually decline. We also observed this trend in our results of SCRs. As shown in Table 3, the SCRs decreased significantly in all conditions (NM, −51.3 ± 25.9%; NMBFB, −56.9 ± 28.4%; NS, −44.2 ± 20.2%; NSBFB, −50.8 ± 20%). The SCRs’ decreases in two music conditions were more significant than non-musical conditions. Figure 6 shows the percent decrease in SCRs during the relaxation session. There was no significant difference between conditions as determined by one-way ANOVA [F(3, 36) = 1.572, p = .213].

Figure 6. Simple Boxplot of percent changes of the SCRs in four conditions.

5.1.3. Respiration rate (RSP-R)

As shown in Table 3, the respiration rate of the participants decreased significantly during the relaxation session in all conditions. Especially in the biofeedback conditions (NS-BFB, NM-BFB), for most participants, their respiration rate dropped significantly below the ten bpm. Figure 7 shows the percent changes in RSP-R in four conditions. There was a significant difference for the four conditions by one-way ANOVA [F(3, 36) = 15.2, p < .01]. Post hoc comparisons using the Tukey HSD test indicated that the decrease in RSP-R for the biofeedback conditions (NS-BFB, −64.7 ± 14.4%; NM-BFB, −56.2 ± 19.5) were significantly greater than the non-biofeedback conditions (NS, −16.1 ± 19.3%; NM, −29.5 ± 19.8%). This result suggests that the HRV biofeedback through UnWind was more effective in encouraging slow breathing during the relaxation exercise. A two-way ANOVA further confirmed the significant effect of biofeedback on the reduction of RSP-R, F(1,36) = 41.9, p < .01. However, the interaction effect between music and biofeedback was not significant, F(1, 36) = 3.54, p = .068.

Figure 7. Simple Boxplot of percent changes in RSP-R in four conditions.

5.1.4. Percent of the power of IBI data in low frequency (LF%)

Table 3 shows that only in NM-BFB condition, the LF% showed a significant increase (p < .05) during relaxation session (relative to its level during the stress session). As shown in Figure 8, the LF% showed a small decrease during relaxation session in the non-biofeedback conditions (NS, −0.6 ± 33.6%; NM, ̶ 26 ± 44%), but an increase in biofeedback conditions (NSBFB, 5.8 ± 28.2%; NMBFB, 30.7 ± 30.7%). One-way ANOVA shows a significant difference in LF% changes for the four conditions [F(3, 36) = 4.52, p < .01]. Post hoc comparisons using the Tukey HSD test indicated that the LF% changes for NM-BFB condition (30.7 ± 30.8) was significantly different from the NM (−26.2 ± 44%) condition. Moreover, a two-way ANOVA shows a main effect of the biofeedback factor in HRV-LF% [F(1, 36) = 8.0, p < .01], and a significant interaction effect between music and biofeedback on HRV-LF% [F(1, 36) = 5.54, p < .05]. These results suggest that the combination of sedative music and biofeedback technique were effective in enhancing the HRV during the relaxation exercise.

Figure 8. Simple Boxplot of percent changes of the LF% in four conditions.

5.2. Psychological self-report

5.2.1. Relaxation rating scale (RRS)

Table 3 shows the self-report of RRS in the relaxation session was significantly higher than in the stress session. As shown in Figure 9, the participants in the two music conditions reported a larger increase in RRS than the non-music conditions relatively (NS, 2.10 ± 2.18 vs NM, 3.0 ± 1.56; NSBFB, 2.9 ± 0.74 vs NMBFB, 3.3 ± 1.64). However, one-way ANOVA shows that there was not a significant difference between the conditions at the p < .05 level among the conditions [F(3, 36) = 1.46, p = .24].

Figure 9. Simple Boxplot of score changes of the RRS in four conditions.

5.2.2. State-Trait-Anxiety-Inventory (STAI)

Table 3 shows the participants reported a decreased STAI score after the relaxation session in all conditions (NS, ̶ 6.6 ± 6.4; NM, ̶ 10.8 ± 6.5; NSBFB, ̶ 10.8 ± 3.4; NMBFB, ̶ 17.4 ± 11.6). Figure 10 shows the changes of STAI scores for four conditions. There was a significant difference between the conditions as determined by one-way ANOVA [F(3,36) = 3.48, p < .05]. A Tukey post hoc test revealed that the decrease of STAI in NM-BFB condition was significantly greater than in NS condition. This result suggests that music biofeedback can be more effective in reducing the subjective anxiety after the stress than listening to the nature soundscape alone. A two-way ANOVA shows the main effect of both factors: music [F(1,36) = 5.1, p = .03] and biofeedback [F(1,36) = 5.1, p = .03] factors in self-report anxiety level. However, the interaction effect between music and biofeedback on STAI was not significant, F(1, 36) = .25, p = .62.

Figure 10. Simple Boxplot of score changes of the STAI in four conditions.

5.3 Interview data

5.3.1. NS condition

The responses indicated that listening to pre-recorded nature sounds made the participants relaxed. 7/10 participants stated that nature sounds brought them with some natural pictures in mind. They expressed that these images led to a pleasant experience and an enhanced relaxation. The nature soundscapes created some imagery feel, such as some feelings described in the interview ‘It made me feel as if I was in a forest’ and ‘I felt myself sitting by the river and watching the birds sing around the trees’. The participant’s preferences on specific nature sounds might greatly influence the relaxation experience. For instance, one participant stated that ‘the sound of the water stream and birdsong are very relaxing, give me a feeling of peace’. However, two participants thought the birdsong of silvereye and owl was annoying and disturbing. Also, 3/10 participants suggested giving the users more selections of nature sounds or ambience sounds, such as the sound of the sea or the coffee shop ambience.

5.2.2. NM condition

The responses indicated that the combination of sedative music and nature sounds was calming and smoothing for most of the participants. 8/10 participants stated that the slow rhythm of the music (Erik Satie’s Gymnopedie No.1) made them calm down and relax quickly. 3/10 participants thought the music helped to create an atmosphere and set the mood which was mentioned in the interview as ‘pleasant’, ‘beautiful’, and ‘happy’. However, 2/10 participants thought the frequent bird sounds damaged the quiet atmosphere created by the sedative music. They suggested us to reduce the frequency and the types of birdsong. 3/10 participants hoped to select the music which they felt relaxing for them. A wider collection of music was suggested, such as the Anasazi flute, classical guitar, and more new piano music (i.e. Yiruma’s River Flows in You).

5.2.3. NSBFB condition

The merits of NSBFB condition are similar to NS condition as described above. The responses also indicated that the biofeedback through the changing wind sound could make the participants focus on their breathing and become more aware of their heart rhythm. 6/10 participants stated that the sense of controlling the wind sound with the breathing regulation was relaxing. 3/10 participants mentioned that when the nature soundscape was becoming increasingly quiet along with the deep breathing, they felt a sense of accomplishment, which made them more relaxed. In contrast, 2/10 participants said that they felt stressed when the changes of the nature sounds were less than or very different from their expectation. Another disadvantage was that the repeated wind sound (6–10 cycles per minute) made some participants feel fatigue and boring after a few minutes exercise.

5.2.4. NMBFB condition

Similar to NM condition, most participants reported that the combination of sedative music and nature sounds was relaxing and calming. 4/10 participants thought this type of music is very suitable for yoga and meditation. 3/10 participants noticed the relationship between the nature sounds and the music. They said when they attempted to breathe slowly and deeply, the nature sounds became quiet and the music seemed to come to the ‘foreground’. In contrast, when they stopped deep breathing practice, the nature sounds became loud and push the music to the ‘background’ which was hard to hear. This motivated them to continue with deep breathing practice. 2/10 participants reported that they were attracted quickly by the music so that they did not pay much attention to the changes of the nature sounds. This weakened the effects of biofeedback. 2/10 participants also mentioned that the nature sounds (particularly the wind sound) reduced the quality of the music. Compared to NSBFB condition, there was no participant reporting a feeling of fatigue and boredom in the relaxation exercise. Similarly to other conditions, the participants also suggested a personalised selection and combination of natures sounds and sedative music.

6. Discussion

In this paper, we have presented a musical interface that combines nature sounds and sedative music as a new form of New-age music for biofeedback display. UnWind allows the users to receive the biofeedback information by listening to music during relaxation exercises. We compared the UnWind musical biofeedback against NS-based auditory biofeedback, music listening and nature sounds listening, with four physiological measures and two psychological measures as dependent variables. Our results confirmed the feasibility of combining nature sounds and sedative music for biofeedback display and showed the positive effects arising from this combination on enhancing HRV and reducing the subjective stress and anxiety. When compared to the two non-biofeedback conditions, biofeedback leads to a lower respiration rate and enhanced HRV (LF%). When compared to the two non-music conditions, the sedative music leads to lower arousal levels (SCRs and HR). There was a significant interaction effect between music and biofeedback on the improvement of HRV during the relaxation exercise. Results from subjective ratings also indicate that musical biofeedback leads to a more significant reduction of anxiety level (STAI).

This study supports the evidence from the previous research on music for relaxation assistance and also shed light on designing a new form of musical biofeedback. As mentioned above, listening to sedative music could help the listener calm down, moderate arousal (Pelletier 2004), and reduce psychological anxiety (Iwanaga, Ikeda, and Iwaki 1996; Labbé et al. 2007). Consistent with the literature, the comparison between music and non-music conditions also shows the beneficial effect of sedative music in lowering arousal activities (SCRs and HR). On the other hand, UnWind suggests a new approach to design musical biofeedback, which does not deal with complex musical structures (Yokoyama et al. 2002; Bergstrom, Seinfeld, and Arroyo-Palacios 2014) or modify the sound effects (Harris et al. 2014; Bhandari et al. 2015), but can also harness the benefits of sedative music. To the best of our knowledge, this is the first musical biofeedback interface that combines nature sounds with sedative music for both presenting biofeedback data and also promoting relaxation.

A musical interface can be based on the ‘parameter-mapping sonification’ (Hermann 2008), where the bio-data is directly mapped to the musical parameters or manipulates the sound effects, such as the quality of music or the perception of the soundscape. In these interfaces, the musical representations of biofeedback data still sound quite different from a well-composed musical piece. Therefore, how to address the aesthetic of musical expressions and create a musically pleasant experience with the musical display remains an important design challenge. Compared to the previous musical biofeedback displays, we saw some strengths of the UnWind.

Firstly, we do not need to manipulate the structure of music for presenting information. Instead, we add a layer of nature sounds as the integrant of the music for feedback display. The two-layered structure of musical interface reduces the difficulty in addressing music expressions and enables us to take advantage of the existing well-composed sedative music. Because the audio output of the musical interface largely depends on the selected music, the listening experience is more predictable and controllable. Secondly, besides the informative functions, nature sounds itself can also create the sensation of experiencing a natural acoustic environment or an imaginary nature scene, which may foster the experience of calmness. Thirdly, the two-layered musical interface makes it possible to update the selection of nature sounds and music for different users and applications. Last but not least, as biofeedback technique is a tool that facilitates the learning of self-regulation, it is likely to cause a certain degree of anxiety during its use for some new users. In these cases, the sedative music could lower the anxiety caused by the biofeedback-assisted learning process. As shown in our results, the participants in NM and NMBFB conditions reported a higher level of relaxation (RRS) and a lower level of anxiety (STAI), respectively.

We think there are still some design issues in UnWind which need to be investigated in the future. For instance, the disharmony or mismatch between nature sounds layer and the music layer might reduce the listening experience. The music layer might interfere with the user’s perception of the information presented by the nature sounds, which may reduce the effectiveness of biofeedback. As shown in our results, the NSBFB condition is relatively more effective than the NMBFB condition in facilitating slow breathing, which suggests that the nature sounds might be more effective in presenting information without the presence of the music. In this study, we have evaluated the UnWind biofeedback system for relaxation assistance, regarding its usability and user experience. For acquiring specialised self-regulation skills, biofeedback-assisted learning requires practice and repetition. Through practice, the users become familiar with their unique psychophysiological responses to stress and learn to control them to a healthy direction. To investigate the effectiveness of UnWind in the assistance of skills learning, it will be necessary to conduct a new user study with a multi-session biofeedback programme in future studies.

From our experience with this study, we have realised the importance of the balance between the functional and experiential aspects of the interface when designing musical biofeedback for relaxation assistance. To achieve this, we suggest an adaptive musical interface where the ‘proportion’ of nature sounds, the ‘apparent degree’ of information layer, can be adjusted based on the performance and the physiological state of the users during relaxation exercises. When the users need the feedback information for ‘revising’ self-regulation, the information layer should be more noticeable and perceptible. On the contrary, when the users have mastered the self-regulation skills and achieved the optimal relaxation state, the link between the nature sounds and feedback information could be weakened, and the information layer could even fade out.

7. Conclusion

We view the outcome of this study as an encouraging indication that the combination of nature sounds and sedative music may be used as a new type of musical biofeedback for relaxation assistance. We have established that the musical biofeedback through UnWind could effectively enhance the HRV by facilitating the breathing regulation and also reduce the physiological arousal and the psychological anxiety. Moreover, the responses from the interview imply that the musical feedback can reduce fatigue and boredom by adding the variation in musical structures.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Bin Yu http://orcid.org/0000-0002-3128-7441

Additional information

Funding

This work was supported by the China Scholarship Council.