Move like everyone is watching: Social context affects head motion and gaze in string quartet performance

Abstract

Ensemble musicians engage with each other visually through glances and body motion. We conducted a case study to test how string quartet musicians would respond to playing conditions that were meant to discourage or promote visually communicative behaviour. A quartet performed in different seating configurations under rehearsal and concert conditions. Quantity of head motion was reduced when musicians' gaze was constrained. Differences in gaze and body motion between musicians reflected their musical roles in the ensemble. Overall, our findings suggest that gaze and motion dynamics vary within and between performances in response to changing musical, situational and social factors.

Keywords:

• Ensemble performance
• interpersonal communication
• body motion
• gaze
• peripheral vision

1. Introduction

As listeners and observers, we are impressed by the high quality of coordination that skilled music ensembles are able to achieve. Successful coordination may take the form of dialogic call-and-response in a group improvisation, a blending of vocal timbres in a choral performance, the patterning of complimentary rhythms in group drumming or the collective shaping of time in a classical string quartet performance. Ensembles of all compositions and genres face the challenge of maintaining coordination despite uncertainty over how the performed music will sound. In the case of a string quartet – the focus of the current study – uncertainty arises primarily in relation to how fellow ensemble members may veer from a mutually decided interpretation.

This case study investigates the function of visually communicative behaviour, including body motion and gaze, during string quartet performance. Our goal is not to show how visual communication supports coordination in sound output; rather, we address the broad questions of how ensemble musicians draw on visual-motor interaction while performing previously rehearsed material, and how their use of visual-motor interaction depends on the social context of their performance. More specifically, we examine body motion and gaze patterns of a string quartet in rehearsal and concert settings. Social context is operationalised through manipulations of visual contact between musicians and the presence/absence of an audience. In the following sections, we discuss the literature on coordination and visual-motor interaction in ensemble playing and literature contrasting musicians' playing styles in rehearsal and concert conditions. This is followed by a discussion of the function of gaze in ensemble performance.

1.1. Coordinating sound through motion

Coordination can be described as an emergent property of group interaction, which arises as group members make small-scale responses to each other's behaviour (Høffding, 2019; van der Schyff et al., 2018; van der Steen & Keller, 2013). Musical communication draws substantially on listening. An extensive literature has described the processes involved in entraining to regularly timed, sounded music as well as the processes involved in anticipating, monitoring, and adapting to the variabilities that emerge in sounded musical output as a result of errors, imprecision in motor output, and expressive nuances (e.g.Keller, 2001; Keller et al., 2014; van der Steen & Keller, 2013; Wing et al., 2014).

A number of studies have additionally tested whether musicians' body motion, ranging from sound-producing actions to communicative gestures, and visual attention towards each other, can help to support successful coordination. A pioneering study on ensemble communication in piano duos was carried out by Williamon and Davidson (2002), who used a combination of observation and interview techniques to explore how the pianists' body motion and gaze patterns changed as they rehearsed a new piece together. A notable finding was that an increasing proportion of signalling gestures and glances between pianists occurred across rehearsals at locations in the music that the pianists identified as ‘important for coordinating the performance and communicating musical ideas’.

Deliberate visual signalling (e.g. exchanging cueing gestures such as exaggerated head nods or conducting-like arm movements) facilitates prediction and helps musicians synchronise when timing is irregular (Bishop & Goebl, 2015, 2018; Kawase, 2014b). Conductors provide this important information for large ensembles (Luck & Sloboda, 2008; Wöllner & Auhagen, 2008; Wöllner et al., 2012), but for small ensembles, the task of performing cueing gestures falls to one of the players – typically, whomever is assuming a leader position at the moment. Cueing gestures are especially important when performers have to synchronise an entrance or a re-entrance following a long pause. Bishop and Goebl (2015) showed that accompanying pianists were able to synchronise with soloists' cueing gestures following long held notes (i.e. fermatas) similarly well in audio-visual and visual-only conditions, suggesting that cueing gestures were the main source of timing information at these moments.

A closer look at the kinematics of instrumentalists' cueing gestures shows that gesture duration and velocity correlate with the gesturer's intended playing tempo (Timmers et al., 2014; Wing et al., 2013). Beat position tends to be communicated through acceleration patterns (Bishop & Goebl, 2018), which is also the case for conductors' gestures (Luck & Sloboda, 2008; Wöllner et al., 2012). For string quartet musicians, the contributions of the head and right (bowing) arm to cueing gestures seem to be the most important to communicating beat position, as occluding these parts of the body reduces other musicians' success at synchronising with the gestures (Wing et al., 2013).

Looking more broadly across the course of a performance, outside of the isolated moments where cueing gestures are needed, there is some evidence that ensemble musicians may synchronise more readily when they can see each other's body movements than when they cannot. Benefits of visual contact have been observed in some singing ensembles (D'Amario et al., 2018; Palmer et al., 2019).

On the other hand, manipulations of visual contact have yielded no notable effects in several studies of piano duos (Bishop et al., 2019b; Bishop & Goebl, 2015; Keller & Appel, 2010), although benefits can emerge when pianists are tasked with playing music that contains sudden tempo changes (Kawase, 2014a, 2014b). In general, the benefits of visual contact are likely to depend on the musical material, the degree to which the musicians deviate from a strictly regular beat, and their familiarity with each other and their shared interpretation.

The question of how ensemble musicians maintain synchronisation in note timing has been a key focus in the literature (Bishop & Keller, in press; Keller et al., 2014), but it is important to note that most types of music require performers to do more than synchronise. Features such as dynamics, articulation, and phrasing must also be coordinated. Furthermore, some music requires performers to de-synchronise their note timing; for instance, by inserting asynchronises for expressive effect or by maintaining distinct metres (i.e. resisting entrainment; see Clayton, 2007). It is therefore of interest to investigate how motor interaction between ensemble members may contribute to other aspects of a performance, such as expressivity.

1.2. Communicative body motion during ensemble performance

A case study of a student string quartet, using observation and interview techniques, highlighted several categories of expressive–communicative gesture functions which occurred as the quartet performed in rehearsal and concert settings (Davidson & Good, 2002). These functions included marking of entrances and exits, marking of dynamics, and coordination of timing and expression. Dynamics were supported by head and arm gestures that varied in size according to how loudly or softly the musicians were aiming to play. The authors also described instances where the musicians seemed to mirror each other's body sway, and identified this as an important means of coordinating timing and expression.

Building on these observations, a number of recent studies have carried out analyses of performers' body movements with the aim of showing how musicians influence each other motorically. These studies focus not on signalling gestures that musicians carry out deliberately, but on features such as quantity of motion, cross-correlations in motion trajectories, or entropy in motion patterns (as a measure of predictability). We can describe these features of body motion as ‘temporally unfolding’, because in contrast to discrete signalling gestures (like cueing gestures), they emerge as meaningful characteristics of motion over longer periods of time. Also in contrast to signalling gestures, temporally unfolding features of body motion are not likely to require substantial attention or monitoring by performers. Focused attention would prove too difficult to sustain over a long period of time, especially as other demands on attention emerge.

The effects of playing together on body motion were assessed in a study with violinists from professional and student string quartets (Glowinski, Mancini, et al., 2013). The violinists from these ensembles showed greater entropy in their head motion when playing solo than when playing (the same passages of music) with their quartet, suggesting that motion became more predictable in the ensemble setting. This increased predictability may be a general feature of motor performance in joint settings, as it has also been observed in people who are carrying out non-musical perceptual-motor tasks (e.g. making speeded motor responses to target stimuli; Vesper et al., 2011). Another communicative feature of body motion that may increase in joint performance settings is the quantity of motion that is produced. Bishop et al. (2019b) found that quantity of head motion increased for duo pianists and clarinettists over the course of an hour-long rehearsal session, reflecting a shift in the musicians' attentional focus from individual note accuracy to group coordination.

Musicians' tendencies to coordinate their head motion or body sway when playing together has been documented in studies of piano duos (Bishop et al., 2019b; Goebl & Palmer, 2009; Keller & Appel, 2010), string quartets (Chang et al., 2017; Glowinski et al., 2012) and a piano trio (Chang et al., 2019). Patterns of coordinated head motion sometimes reflect the leader/follower relationships that emerge between ensemble members. Glowinski et al. (2012) estimated the degree to which string quartet musicians influenced each other's head movement by calculating each musician's ‘driving force’ (a measure based on Granger causality), and found the highest driving force for the 1st violinist, reflecting their role as leader of the group. In another study of string quartet musicians, the role of leader was assigned to different quartet members in different trials (Chang et al., 2017). An analysis of body sway again drawing on Granger causalities showed that assigned leaders influenced their co-performers more than the ‘following’ co-performers influenced the leader or each other.

As we have discussed, changes to the social and musical aspects of a performance setting – such as the presence of co-performers, a greater or lesser familiarity with the musical material, or leader/follower assignments – can prompt changes in the quality (and quantity) of musicians' body motion. Another example comes from a study by Glowinski, Gnecco, et al. (2013), which compared string quartet musicians' head motion between normal playing conditions and a ‘perturbed’ condition in which the 1st violinist introduced new expressive features while playing without prior discussion with the other musicians. Motion features such as head direction and convergence of head motion towards a central point in the middle of the quartet differed between the normal and perturbed playing conditions, indicating that the musicians adopted a different style of motion in response to the 1st violinist's expressive perturbations (see also Glowinski et al., 2015).

1.3. Contributions of visual-motor interaction to performance quality

Drawing this literature together, we might speculate on the function that expressive and communicative body motion has on ensemble playing – that is, the benefits that it offers performance – from the performers' perspective, in terms of musical quality. First, the ability to see co-performers' movements may help with coordinating expressive nuances. Performers' body motion is known to communicate expressivity to audiences (Dahl & Friberg, 2007; Davidson, 1993; Tsay, 2013), and performers' influence on each other's body motion increases as they play with more expressive intensity (Chang et al., 2019). However, how expressive nuances emerge dynamically from the motor interactions between ensemble members remains unclear.

A second possible function of expressive and communicative body motion in ensemble playing is that of social motivation. Musicians' motion may be partially driven by the desire to engage socially with other ensemble members. Body motion provides visual cues that can help musicians monitor each other's attention, perceptions of success, and engagement with the music. Particularly in the context of high-level performance, a great deal of mental and physical effort is needed for musicians to maintain enough control over their motor output to achieve their expressive goals. Even advanced performers will retreat to standard interpretations that can be carried out more automatically, unless they are motivated to introduce alternate expressive nuances and confident that their co-performers will be able to respond to them (Bishop & Goebl, 2020). Musicians who are able to confirm their co-performers' attentiveness by monitoring their body motion may be more willing to invest the extra effort that is needed to produce an inspired performance.

This hypothesised function of musicians' body motion is driven by studies of gaze and motor interaction in piano and clarinet duos (also referred to in the discussion above), which showed that the musicians moved more and spent a greater percentage of performance time watching each other after rehearsing than before (Bishop et al., 2019a, 2019b). Similar findings were observed in rehearsals by piano trios (Vandemoortele et al., 2018). Notably, Bishop et al. (2019b) also showed a decrease in communicative body motion at the end of the rehearsal session, when, following their final performance under free-gaze conditions, the musicians recorded a performance without visual contact. These findings suggest that visual interaction is motivated not by a lack of certainty about how the duo will interpret the music, which occurs at the start of rehearsing a new piece, but by familiarity with the music and co-performer, which is achieved after a period of playing together. Visual interaction furthermore seems to be motivated by the more social context that occurs when the musicians are able to watch each other freely, and dissuaded by the less social context that occurs when visual contact is not possible. The findings of these studies also suggest that visual interaction tends to emerge once performers are familiar enough with the music that they can divert their attention away from the score and focus on the collaboration.

1.4. Ensemble performance in concert conditions

Moelants et al. (2012) compared performances by a vocalist and viola de gamba player under concert and rehearsal conditions, and found that the vocalist changed posture more often and made higher-intensity hand gestures during the concert. Biasutti et al. (2016) analysed video recordings of string quartet performances given under concert and rehearsal conditions, and observed stronger (more emphasised) attacks during the concert. Such findings suggest that high-arousal concert conditions may encourage increased motor activity in skilled performers. In part, this increased motor activity may be driven by a desire to engage with the audience by drawing their attention through visual cues. Indeed, studies of audience perception of musicians' motion during performance have shown that audience judgements of performance quality and expressivity are highly influenced by visual cues (Behne & Wöllner, 2011; Griffiths & Reay, 2018).

On the other hand, some musicians may gravitate towards a more controlled – and perhaps more practiced, less flexible – style of movement in concert conditions (Schaerlaeken et al., 2017). This change might reflect a cautious, conservative mindset in which the musician prioritises accuracy and practiced expressive nuances over spontaneity and creativity (which come with increased risk). A less flexible movement style might otherwise (or also) reflect performance anxiety, which can have substantial physiological effects that might interfere with motor control (Yoshie et al., 2009).

Overall, it remains unclear how musicians respond to concert conditions, and whether the presence of an audience promotes more overt expressions of engagement with the music and other musicians. The conflicting results of the studies listed above suggest that musicians' responses may vary depending on their personality and skill level, as well as on the context of the performance (e.g.who is in the audience, and what are the long-term effects of playing well versus poorly?).

1.5. Visual communication through central and peripheral vision

A secondary aim of the current study was to test whether musicians can establish visual-motor interaction when they have only a peripheral view of each other and are unable to watch each other directly. In ecological performance conditions, ensemble musicians are not always free to watch each other directly. Depending on how they are positioned in the performance space, one musician may have limited or no visual access to another; in addition, they may have to keep their eyes on the score if it is complex or unfamiliar. Thus, the ability to establish motor interactions given only peripheral visual information (i.e. information that falls beyond the fovea and parafovea, or more than 2.5${}^{\circ }$ of radius from fixation) about co-performers' movements could be useful.

While acuity is less in the periphery of the visual field than in the centre, people can extract some rough detail about moving stimuli that are presented peripherally. For example, viewers extract information about walking direction from peripheral stimuli (Thornton & Vuong, 2004), even if stimuli are presented as scrambled point-light displays (Troje & Westhoff, 2006). There is some evidence that peripheral stimuli can trigger ‘weak’ motor resonance: grasping actions that are presented in near peripheral vision can modulate the activation of motor circuits, though these responses are imprecise (Leonetti et al., 2015). It remains unclear how much information musicians are able to extract about the peripherally viewed movements of co-performers, and indeed how much movement information is even available to central and peripheral visual fields during the course of a performance.

Eye movements are, of course, not only a means of obtaining information; gaze direction also conveys information to others about where the viewer is attending. People show a reflex-like tendency to follow others' gaze, facilitating joint attention (Driver et al., 1999). Some studies show that gaze following is especially likely to occur following direct eye contact with the observer – that is, we are more likely to follow someone's gaze if they have just been looking directly at us (Bristow et al., 2007). Faces that are presented looking directly at us are remembered better than faces that are presented with averted gaze (Mason et al., 2004), and we ascribe more sophisticated mental abilities to faces with direct gaze (Khalid et al., 2016). In a collaborative musical setting, direct gaze may serve as an important nonverbal cue that musicians use to attract and monitor each other's attention.

Few studies have examined musicians' gaze patterns in detail. As discussed above, musicians in duos and trios have been found to spend increasingly more time watching each other as they rehearse (Bishop et al., 2019a; Vandemoortele et al., 2018; Williamon & Davidson, 2002). Bishop et al. (2019a) showed that duo musicians watched each other primarily at the start and end of the piece and during a temporally irregular section with no marked metre, although glances sometimes occurred outside of these locations. Another study of piano duos showed that gaze durations at moments of tempo change were higher for pianists who were assigned a leader role than for pianists who were assigned a follower role (Kawase, 2014a). How gaze patterns relate to the different musician roles that naturally occur in a string ensemble, however, remains unclear.

1.6. Current study

This study contributes to the literature with a novel investigation of how string quartet musicians' gaze and body motion patterns change in response to playing conditions that differ in social demands and communication constraints. The playing conditions that we tested were expected to either promote visually communicative behaviour, by providing the musicians with observers, or discourage it, by disrupting normal communication channels and the assumed leadership structure of the group. In contrast to many previous studies of ensemble playing, our focus was not on the quality of coordination in sound output, but on the way the musicians influenced each other through their body motion, and the extent to which the presence of observers (whether co-performers or audience members) prompted more overt body-based between-performer interaction.

The quartet recorded a total of six performances of the same material (Figure 1). The first five were given under ‘rehearsal-like’ conditions, so the musicians were alone in the lab, apart from the experimenters. The sixth performance was given in the same location under concert conditions with an examiner present (and included additional musical material).

Figure 1. Schematic of the performance conditions. V1, 1st violin; V2, 2nd violin; Vla, viola; C, cello.

The performance conditions that we tested are listed below in the order that the quartet completed them.

1. Blind: The musicians played back-to-back so that they were entirely unable to see each other.

2. Score-directed: The musicians faced each other in their normal semi-circle configuration but were instructed to look only at the score, so that they had only peripheral visual access to each other.

3. Normal-rehearsal: The musicians faced each other in their normal configuration and were free to look where they wanted. For our analyses, we used this condition as the reference to which others were compared.

4. Violin-isolated: The 1st violinist played behind a curtain, while the others were in their normal configuration and free to look where they wanted. This condition was intended to disrupt the normal leadership structure of the group by preventing the exchange of visual cues with the 1st violinist (the assumed leader).

5. Replication-rehearsal: A replication of the Normal-rehearsal condition. This condition was included so that we could test for order effects.

6. Concert: The musicians played in their normal configuration and were free to look where they wanted. Their performance in this condition constituted a semester exam, and they were joined in the lab by an audience of about 40 people, which included an examiner, some family members and friends of the musicians, and researchers from our centre.

Our main predictions were as follows.

Prediction 1 (hereafter P1): It was predicted that the quartet would vary their body motion across repeat performances of the same material despite relative consistency in sound output. Typically, high-level musicians endeavour to vary their performances and maintain some level of ‘freshness’ and spontaneity (see Clarke, 2011). However, in the lab, solo musicians have been shown to perform expressive body motion consistently across performances, with especially high consistency during expressively salient passages (Teixeira et al., 2015).

It is unclear whether ensembles would show the same consistency, especially across different visual contact conditions. Our prediction was that points of high expressivity might be affected more by the different playing conditions, resulting in more variability in expressive motion during expressively intense passages than elsewhere in the music.

Prediction 2 (P2): This study investigated the potential function of musicians' body motion as a source of social motivation. We tested the prediction that ‘more social’ visual contact conditions (e.g. where the musicians could watch each other freely) would increase communicative body motion relative to ‘less social’ visual contact conditions (e.g. where the musicians could look only at the score).

Previous research has shown that during early stages of rehearsal, musicians move more similarly when they can see each other than when they cannot, suggesting that these visual-motor interactions are maintained through real-time exchange of visual information (e.g. rather than the result of similar individual responses to the music; Bishop et al., 2019b). The current study was expected to build on this finding by showing that an ensemble playing well-practiced repertoire would also vary their interactivity in response to visual contact conditions. Such a result would indicate that similarity in motion patterns partially emerges from real-time interaction, even for ensembles who have established an expressive script for the performance through extensive rehearsal.

Prediction 3 (P3): We predicted that the musicians would engage in more visual-motor interaction in the concert performance than in the rehearsal performances that were given under free-gaze conditions. This prediction builds on research showing that the high-arousal conditions of live concert or exam performance can encourage more communicative body motion (e.g. Biasutti et al., 2016), and presupposes that given the musicians' experiences as advanced students, they might respond positively to the concert and not show substantial negative effects of performance anxiety.

Prediction 4 (P4): This study tested the prediction that musicians would obtain information about each other's body motion patterns through peripheral vision while focusing only on the score and not seeing each other directly, resulting in more communicative motion during a ‘score-directed’ gaze condition than in a ‘blind’ condition in which musicians had no visual access to each other at all. The Violin-isolated condition was predicted to disrupt the leading/following relationships of the ensemble and result in increased interactivity between the 2nd violinist, violist and cellist.

Visual-motor interaction was quantified in terms of the percentage of performance time that the musicians spent watching each other, their quantity of head motion, and the average distance between their heads. We also conducted exploratory analyses of how consistent the musicians were in their sound output and head motion across performances, partially addressing the question of how tightly coupled sound output and ancillary gestures are. Finally, we made some exploratory analyses of how gaze patterns related to the musicians' different roles in the quartet.

2. Methods

2.1. Participants

The quartet was a student ensemble from a local music academy, who had been playing together as an ensemble for about 6 months. The musicians were 19–20 years old (one female and three male) and had between 13 and 16 years of formal music training. All four musicians had attended the same music school as children, where they had occasionally played in ensembles together over the years. The 2nd violinist and cellist in particular had played in the same quartet for about 11 years. The group had been rehearsing the piece that they performed for the experiment (Haydn) for three months, and the second piece that they performed in the concert (Debussy) for two months. The musicians all provided written, signed informed consent.

2.2. Design

Randomisation of experiment conditions was not possible, so we selected an order of conditions that would allow us to anticipate some order effects. In particular, factors such as anxiety about the concert, fatigue, and boredom with playing the same music several times over could have a negative effect on the intensity of the quartet's expressivity during later conditions. Therefore, we positioned the condition that we expected to be the most difficult (i.e. the Blind performance) first. We included two rehearsal-like conditions in which the musicians could look around freely (i.e. the Normal-rehearsal and Replication-rehearsal) at different points of the experiment, so that we could compare them to check for order effects.

2.3. Materials and equipment

For the first five performances (Blind through Replication-rehearsal), the quartet played the first 68 bars of the String Quartet in B-flat major, Op. 76, No. 4, Allegro con spirito, by Haydn. They selected this excerpt from their repertoire after we asked them to choose something short, dynamic, and which they enjoyed playing together. For the concert, they played the full first and second movements (Allegro con spirito and Adagio) of the Haydn and the full first and second movements of the String Quartet in G minor, Op. 10, by Debussy.

The experiment set-up is shown in Figure 2. The quartet was positioned in one half of the performance space, and were joined by an audience during the Concert condition that filled most of the other half.

Figure 2. Left: The quartet rehearsing in the performance space, with MoCap cameras highlighted. Right: A close-up of a violinist, with body and glasses markers highlighted.

Motion data were recorded using a Qualisys system with 12 Oqus 300/310 cameras. Six reflective markers were affixed to each musician: 1 on the top of the head, 1 on the upper back, and 2 on each arm, positioned above and below the elbow. Markers were also placed in the corners of each music stand. Motion data were recorded at 120 Hz.

The musicians wore SMI Wireless Eye Tracking Glasses, which were attached by cable to a portable device. The glasses recorded both gaze and pupil diameter at 60 Hz on individual Android devices. Each set of glasses had 4 markers attached in a unique configuration (2 slightly above the top of the glasses frame and 2 sticking downwards from the corners) so that they could be tracked as a ‘rigid body’ by the MoCap systems. The glasses attachments that the markers were fixed to were custom-designed and 3D-printed.

The performances were audio- and video-recorded using a Garmin VIRB 360${}^{\circ }$ camera and Canon XF-105. Videos from the Canon camera were recorded within Qualisys Track Manager (QTM) in alignment with the motion capture data.

To align all other data, we recorded an audio-visual signal using a clapperboard (with 2 markers affixed) at the start of each performance. Timestamps for all devices were synchronised retrospectively to this point.

2.4. Procedure

The recording session started with a brief warm-up period. The musicians were then positioned for the first performance, and were instructed to sit still and gaze at a single score note for a one-minute baseline pupillometry recording. The Blind performance was then recorded.

Following this and each subsequent performance, there was a brief pause as we repositioned the musicians for the next condition and reset the recordings. The musicians were allowed to converse during these pauses and occasionally discussed aspects of the previous performance. Baseline pupillometry recordings were taken each time the quartet was repositioned (prior to all but the Normal-rehearsal condition).

We made a longer pause after the Replication-rehearsal performance. During this time, the musicians took a break and the lab was prepared for the concert. Six audience members, including two family members, were fitted with HR sensors. The musicians then had a few additional minutes of rehearsal and completed a final baseline pupillometry recording before the audience came in. Following a short welcome to the audience, the concert was recorded.

2.5. Analysis

Our dependent variables are described in the sections below. In terms of head motion, we examined differences in quantity of motion across the six conditions and over time within conditions. We also examined within-subject/between-performance variability and between-subject/within-performance differences in quantity of motion curves. In terms of gaze data, we examined the percentage of performance time that the musicians spent watching each other, and the percentage of performance time that they had a peripheral view of each other.

As a supplementary analysis, we compared sound level (root mean square (RMS); a correlate of loudness) between performances. This comparison provided some indication of how variable the musicians' sounded performances were. The sound level curves also serve as general representations of piece structure.

2.5.1. Audio data

We used the Python package Madmom (Böck et al., 2016) to extract sound level (RMS) values for audio recordings, using a frame size of 2048 samples with 50% overlap. Sound level curves were smoothed with a convolution-based method, using a Hamming window of 50 samples.

To evaluate how similarly the musicians played across performances, smoothed sound level curves were temporally aligned with the sound level curve of the Normal-rehearsal performance (our reference condition) using dynamic time warping (‘dtw’ package in R). The DTW algorithm gives a normalised (per observation) distance measure that represents the difference between aligned curves. We used this measure as an indication of how similar/different the other performances were in loudness fluctuations to the Normal-rehearsal performance (hereafter referred to as ‘between-performance difference’, since higher values indicate a greater difference).

2.5.2. Head motion data

All analyses of head motion were completed in R (R Core Team, 2018). Performances were defined as beginning 5 s prior to piece onset and concluding 5 s following the final note onset. We set these 5 s ‘buffer’ periods so that we would be sure to capture motion relating to the musicians' preparatory (cueing) gestures and final release gestures.

Head position data contained some slight noise (Bishop & Jensenius, 2020), so data in x-, y-, and z-dimensions were individually smoothed using a Savitzky–Golay filter (‘prospectr’ package; Stevens & Ramirez-Lopez, 2020) with a polynomial order of 3 and window size of 41. Velocity was computed as the numerical derivative of smoothed position data using a central differences formula and smoothed slightly using the same method (order = 3, window = 21). We then computed the norm of smoothed 3D velocities, and Quantity of Motion (QoM) was calculated as the sum of normalised velocities at each second interval.

QoM across visual conditions. We used generalised linear modelling (GLM) to test for differences in QoM between the Normal-rehearsal and all other performances. A gamma distribution was specified since QoM data were positively skewed. To achieve enough data points for statistical analysis, we calculated mean QoM values per piece section. Piece sections were manually identified based on major structural features in the score (phrase boundaries and textural changes), and correspond to the following: bars 1–21 (Section 1); bars 21–36 (Section 2); bars 36–49 (Section 3); bars 50–59 (Section 4) and bars 60–68 (Section 5). Thus, the GLM included a data point for each combination of piece section, performance and musician.

Stability of head gestures over time. To estimate how much musicians varied their head motion across performances (i.e. how stable their motion patterns were), we compared QoM curves (see Appendix for illustrations of per-performance/musician QoM curves). Every performance was compared to every other performance using a windowed approach (window size 10 s, 50% overlap; window size was set to approximate phrase length at the quartet's starting tempo). Mean squared errors (MSEs) were computed within windows by averaging per-second squared errors.

QoM and musical structure. We also examined the relationship between head motion and musical structure. QoM was calculated per piece section (using the structurally determined sections described above), and GLM was used to test for differences between sections. Again, a gamma distribution was specified. Section 1 was used as the reference level against which all subsequent sections were compared, giving us an indication of how QoM changed relative to the start of the piece.

Interperformer interaction. As an overall measure of interperformer interaction, we calculated the sum of distances (vector lengths) between performers' head markers. Four distances were computed: between the 1st and 2nd violinist, 2nd violinist and violist, violist and cellist, and cellist and 1st violinist. Distances between each pair of musicians were normalised to a 0–1 range within each performance. Summed distances (which took a value between 0 and 4) were then again normalised to a 0–1 range within each performance.

The sum of distances represented the size of the perimeter of the polygon defined by the four heads. Smaller perimeter size occurred when the performers' heads were closer together (e.g.because they were all leaning forward). GLM compared perimeter sizes between the Normal-rehearsal and other conditions. We omitted the Blind and Violin-isolated performances from this analysis, since the musicians were configured differently.

To evaluate relationships between individual musicians' patterns of motion, we used DTW to compare the QoM curves between all combinations (pairs) of musicians (six pairs total). We used the normalised (per observation) distance values that are output by the DTW algorithm as an indication of the similarity/difference between musicians' motion patterns, with higher values indicating a greater difference. GLM was then used to assess the magnitude of difference (1) per musician, indicating how distinctively each musician moved, and (2) between various pairings of musicians.

For the former assessment, the 1st violinist was compared to all other musicians, with the expectation that the others might follow his movements, decreasing the apparent distinctiveness of those movements. For the latter assessment, the 1st violin/2nd violin difference was compared to all other pairings; given that the violinists played the same instrument and often have especially musically complimentary parts, they might be expected to move most similarly.

2.5.3. Gaze data

Gaze data were manually coded for all but the Blind performance (including the Score-directed performance, so that we could confirm whether the musicians followed the instructions and kept their eyes on the score). A ‘gaze target’ was identified for each sample of data, based on where the person was looking at the time. There were seven possible targets for each musician: the score, the musician's left-hand fingers, the musician's bow/bowing hand, the three other musicians, and ‘elsewhere’ (anywhere else in the room).

We adopted a fairly lenient gaze coding strategy. Coding was done by visually comparing the position of musicians' fixation points against the recorded scene video. We documented the time that musicians spent looking at different gaze targets in continuous ranges, without excluding blinks. For example, if the cellist fixated on the 1st violinist at 01:00.000 and started shifting her gaze towards the score at 01:01.000, then for that full second she was documented as watching the 1st violinist, even if she blinked during that time. We allowed for a small margin of error around each potential target to account for possible imprecision in fixation points. We also allowed for the possibility that musicians occasionally fixated outside the spatial range of gaze positions that the glasses could capture. For example, if the cellist fixated on the 1st violinist, but then her gaze drifted upwards outside the capture range of the glasses before drifting back down to the 1st violinist, the time that her gaze was outside the capture range was counted in the time spent watching the 1st violinist.

To verify the accuracy of our manual coding, we ran a comparative coding analysis using the area of interest (AOI) labelling function that is available in eye tracking analysis software BeGaze, produced by SMI. This was done for the cellist's Normal-rehearsal gaze data. AOIs were marked around the violinists, violist, and the score. The results were proportional to our manual coding results but documented less looking time for each gaze target, indicating that the automated analysis provides a more conservative estimation of gaze target looking times. We decided to use our manual coding strategy for the remainder of the recordings because we found that the automated analysis underestimated looking times.

Using our per-sample gaze target codes, we calculated the percentage of each performance that the musicians spent watching each gaze target. We also calculated the number of glances that the musicians made towards each gaze target (i.e. the number of entries into the area of interest that defined each target).

Our analysis of gaze excluded the 1st violinist in the Violin-isolated performance, as he was visually isolated from the rest of the ensemble. Due to poor gaze tracking (probably due to sweat and slippage of the glasses), we also had to omit the 2nd violinist from the Concert.

2.5.4. Estimation of peripheral visual field

We estimated the boundaries of the musicians' near peripheral visual fields using the following procedure. First, a new origin was set at the binocular gaze vector origin of the viewing musician (i.e. midway between the left and right eyes), and all MoCap markers in the capture volume were translated with respect to that origin. New axes were defined according to the position of the viewing musician's glasses, with the x-axis (horizontal) defined by the top two markers, the y-axis (vertical) defined by the two leftmost markers, and the z-axis (depth) defined as orthogonal to the new x- and y-axes. All MoCap markers were then rotated according to these axes. This translation-rotation brought the gaze vector (recorded by the glasses) into the same coordinate space as the MoCap markers.

Next, vectors were defined between the viewing musician's gaze vector origin and the head marker of each other musician. The vertical angle between each of these ‘head’ vectors and the viewing musican's gaze vector was then calculated using the projection of both vectors on the y–z plane (vertical; where x = 0). Similarly, the horizontal angle between each head vector and the gaze vector was calculated using the projection of both vectors on the x–z plane (horizontal; where y = 0).

Finally, for each head vector, the vertical and horizontal angles were then compared against two criterion angle values: 60${}^{\circ }$ in the horizontal direction and 25${}^{\circ }$ in the vertical direction. These can be taken as an approximation of the limits of the near peripheral visual field in people with normal peripheral vision (Simpson, 2017). Where both the horizontal and vertical angles for a head vector fell below these criterion values, the corresponding musician was said to be within the viewing musician's near peripheral visual field.

Due to unreliable tracking of the 2nd violinist's and violist's glasses markers, we report analyses of peripheral visual fields only for the 1st violinist and cellist.

3. Results

3.1. Sound level and timing of performances

We calculated sound-level curves partially as a way of illustrating the structure of the piece, and partially to see how similarly sound level and timing were maintained across performances (given that head motion and gaze were expected to vary).

Sound level (RMS) curves for the Normal-rehearsal and the performances that were most similar (Replication-rehearsal) and least similar (Concert) are given in Figure 3. The plots show smoothed curves that have not yet been aligned, so it is possible to see that the Concert performance was slower than the Normal-rehearsal, finishing slightly later (note also that this slowing is only apparent after about the first 20 s). This temporal offset is in addition to the differences in RMS magnitude that were captured by the between-performance difference measure (calculated using aligned performances). Overall, the quartet was very consistent in their global tempo across performances, as performance durations were all within 2 s of each other (Table 1). This high consistency in timing and dynamics should be kept in mind as between-performance differences in head motion and gaze are discussed below.

Figure 3. Smoothed sound level (RMS) curves for the Normal-rehearsal (black), Replication-rehearsal (red) and Concert (blue). Vertical black lines indicate piece section boundaries.

Table 1. Between-performance comparisons of sound level (RMS) curves.

	Performances
	Blind	SD	NR	VI	RR	Concert
Between-performance difference	1426.2	922.0	–	912.7	825.6	1743.0
Performance duration (s)	131.1	129.5	130.2	130.5	129.5	130.3

Notes: Lower between-performance difference values indicate higher similarity to the Normal-rehearsal. SD, score-directed; NR, normal-rehearsal; VI, violin-isolated; RR, replication-rehearsal.

3.2. Quantity of motion across visual conditions

We predicted that QoM would be higher in performances with free-gaze than in performances with constrained gaze (P2, P4), and that increased QoM would be particularly encouraged in the Concert (P3). Figure 4 shows mean QoM per performance, with data averaged across musicians. GLM showed higher QoM in the Normal-rehearsal than in the Blind, $t(119)=2.62$, p = .01, and Score-directed performances, $t(119)=3.56$, p<.001. The other performances did not differ from the Normal-rehearsal. Thus, the restrictive conditions of the Blind and Score-directed performances seem to have reduced head motion.

Figure 4. QoM per performance. B, blind; SD, score-directed; NR, normal-rehearsal; VI, violin-isolated; RR, replication-rehearsal; C, concert.

Figure 5 shows QoM per performance separately for each musician. Some of the musicians (e.g. the 2nd violinist) varied more substantially between performances than others (e.g. the cellist). It is also notable that only the 1st violinist moved less in the Concert than in the Normal-rehearsal and Replication-rehearsal.

Figure 5. QoM per performance for each musician individually. Abbreviations of performances are listed in Figure 4. (a) 1st Violin, (b) 2nd Violin, (c), Viola and (d) Cello.

3.3. Timecourse of head motion

As a secondary exploratory analysis, we examined the timecourse of QoM profiles in order to identify locations in the piece where QoM was consistently higher or lower, and where musicians were more or less variable in their movements across performances (addressing P1).

QoM and musical structure. Figure 6 shows average QoM values per piece section. GLM revealed higher QoM in the second, $t(119)=7.96$, p<.001, third, $t(119)=4.64$, p<.001, fourth, $t(119)=7.81$, p<.001, and fifth sections of the piece, $t(119)=6.46$, p<.001, compared to the first section.

Figure 6. QoM per piece section. The numbers above the plot indicate mean sound level (RMS) per section.

A notable structural difference between Section 1 and later sections is its dynamic level (specified as piano in the score), so we made a between-section comparison of sound level, which was lowest in Section 1 (0.24), somewhat higher in Sections 3 and 5 (0.34 and 0.33), and highest in Sections 2 and 4 (0.58 and 0.59). However, per-second sound level and QoM values correlated only slightly ($R^2=.13$), suggesting that differences in dynamics are only partially related to differences in QoM.

Stability of head gestures over time. Figure 7 shows MSEs, which indicate between-performance/within-subject similarity in QoM curves, averaged across pairs of performances at each 10 s window. GLM revealed higher MSEs in the second, $t(103)=6.27$, p<.001, third, $t(103)=2.95$, p<.001, fourth, $t(103)=6.44$, p<.001 and fifth sections of the piece, $t(103)=3.40$, p<.001, compared to the first section. Interestingly, MSE curves tended to peak for all musicians towards the end of each piece section, perhaps reflecting more flexible periods of expressivity.

Figure 7. MSEs indicating within-subject/between-performance variability in QoM curves. Vertical black lines indicate piece sections.

Per-musician MSE curves were relatively similar in shape, suggesting that periods of high and low variability tended to co-occur across musicians. The 1st violinist and cellist showed lower MSEs (105.6 and 103.6) than the 2nd violinist and violist (116.1 and 135.7), though these differences did not reach significance at $\alpha =.008$ (following Bonferroni adjustment).

3.4. Interperformer motor interaction

We predicted that interperformer interaction would decline in conditions where the musicians could not see each other freely (P2, P3, P4). GLM showed that the quartet's perimeter size was smaller in the Normal-rehearsal than in the Score-directed, $t(549)=2.92$, p = .004 and Concert performances, $t(549)=5.19$, p<.001 (alpha = .01; Figure 8). Thus, the musicians spent more time with their heads closer together (perhaps leaning forward) during the Normal-rehearsal than in the more ‘constrained’ Score-directed performance (2) and Concert (6). (Recall that perimeters were normalised per performance, so these differences do not simply indicate a closer seating arrangement.)

Figure 8. Normalised perimeter sizes indicating the average ‘closeness’ of musicians' heads. SD, score-directed; NR, normal-rehearsal; RR, replication-rehearsal; C, concert.

Figure 9 shows the average between-performer difference value that was calculated for each musician, representing how distinctive their motion was, and how differences compared across pairings of musicians. Differences have been averaged across performances for both of these plots.

Figure 9. Between-performer (normalised) differences in head motion. V1, 1st violin; V2, 2nd violin; Vla, viola; Cel, cello.

The cellist had a greater average difference value than the 1st violinist, $t(359)=2.58$, p = .01, indicating that her motion was, on average, more distinct than his. Difference values for the 2nd violinist and violist did not differ from those of the 1st violinist.

The between-performer difference of the 2nd violinist and cellist was significantly greater, $t(179)=3.784$, p<.001, and the difference of the 1st violinist and cellist was marginally significantly greater, $t(179)=1.95$, p = .05, than the 1st violin/2nd violin difference. All other pairings were similar to the 1st violin/2nd violin difference.

3.5. Gaze direction across visual conditions

We predicted that the musicians would spend more time watching each other during the Concert than during the other free-gaze conditions (P3). We also expected their gaze patterns to reflect to the traditional leader/follower structure of the classical quartet (i.e., where the 1st violinist is the leader). The percentages of performance time that the musicians spent looking at the score and each other are shown for each performance in Figure 10. In the Score-directed condition, the musicians were mostly able to follow the instructions and keep their eyes on the score; there was only one between-performer glance recorded (the cellist glanced once at the 1st violinist for about 200 ms). Generally, in the other conditions, the musicians spent between 70 and 90% of their time looking at the score, though this varied between performers; for example, in the Normal-rehearsal and Replication-rehearsal, the cellist spent only about 40% of her time looking at the score.

Figure 10. Percentages of performance time that the musicians spent looking at the score and each other across performances. Note that in the Score-directed performance, one between-performer glance was recorded by the cellist (who looked briefly at the 1st violinist).

Table 2 shows how much time each musician spent watching each other musician (with percentages averaged across performances). Overall, the 1st violinist was watched the most (the others spent 12.5% of their cumulative time watching him), followed by the 2nd violinist (7.7%), the cellist (4.6%), and then the violist (3.8%). In contrast, the 1st violinist spent very little time watching the other musicians, and indeed never once looked at the 2nd violinist.

Table 2. Percentages of performance time that the musicians spent watching each other (averaged across Normal-rehearsal, Replication-rehearsal and Concert performances).

		Viewing musician
		1st violin	2nd violin	Viola	Cello
Viewed musician	1st violin	–	6.5%	3.0%	26.2%
	2nd violin	0%	–	5.0%	16.3%
	Viola	3.1%	6.9%	–	2.1%
	Cello	0.2%	11.9%	2.5%	–

T-tests were run using per-piece section percentages to determine whether the 1st violinist was the ‘most watched’ musician by a significant margin. The violist, $t(50)=4.0$, p<.001, and cellist, $t(66)=3.2$, p = .002, were watched significantly less than the 1st violinist, but no difference was found between the 1st and 2nd violinist, $t(68)=2.3$, p = .03 ($\alpha =.02$).

Table 3 shows gaze data in terms of the total numbers of glances that the musicians made towards each other during the experiment. Chi-square tests were performed on this frequency data individually for each musician ($\alpha =.01$). These showed a marginally significant result for the 1st violinist, ${\chi }^2(1)=6.40$, p = .001, and significant results for the 2nd violinist, ${\chi }^2(2)=18.41$, p<.001 and cellist, ${\chi }^2(2)=81.62$, p<.001, indicating within-subject differences in how often these musicians looked at each of their co-performers. The 1st violinist looked most often at the violist, the 2nd violinist looked most often at the cellist, and the cellist looked most often at the 1st violinist. The duration of co-performer-directed fixations also provides some indication of how the musicians distributed their attention towards each other (Table 4).

Table 3. Number of glances that each musician made towards their co-performers across the experiment (including Normal-rehearsal, Replication-rehearsal and Concert performances).

		Viewing musician
		1st Violin	2nd Violin	Viola	Cello
Viewed musician	1st violin	–	10	18	133
	2nd violin	0	–	16	107
	Viola	9	18	–	19
	Cello	1	37	11	–

Table 4. Average fixation duration (in s) for each combination of viewing and viewed musician (using data from Normal-rehearsal, Replication-rehearsal and Concert performances).

		Viewing musician
		1st violin	2nd violin	Viola	Cello
Viewed musician	1st violin	–	1.72	0.65	0.77
	2nd violin	0	–	1.40	0.79
	Viola	1.35	1.17	–	0.60
	Cello	0.63	1.29	1.20	–

The musicians spent somewhat less time watching the score in the Normal-rehearsal than in subsequent performances; however, t-tests comparing the Violin-isolated, Replication-rehearsal and Concert performances with the Normal-rehearsal (using percentages computed per piece section) indicated no significant differences at $\alpha =.02$. Notably, the musicians divided their visual attention between the score and their co-performers similarly in the Concert and rehearsal conditions.

3.6. Estimation of peripheral visual field

A secondary analysis was done to estimate how often the 1st violinist and cellist had a peripheral view of their co-performers. Table 5 shows the percentages of the Score-directed, Normal-rehearsal, Replication-rehearsal and Concert performances that the 1st violinist and cellist had each other musician in their near peripheral visual field. The Score-directed performance gives us an estimate of how visually accessible the musicians are to each other when they are all looking only at their scores. According to our estimates, both the 1st violin and the cello had little or no peripheral visual access to the musician sitting directly beside them. Percentages generally increased in the Normal-rehearsal, Replication-rehearsal and Concert, when the musicians had the freedom to look around.

Table 5. Percentages of Score-directed (SD), Normal-rehearsal (NR), Replication-rehearsal (RR) and Concert (C) performances that the 1st violinist and cellist was estimated to have each other musician in their near peripheral visual field.

		Viewing musician
		1st Violin				Cello
		SD	NR	RR	C	SD	NR	RR	C
Viewed musician	1st violin	–	–	–	–	77%	82%	74%	73%
	2nd violin	0%	2%	1%	2%	95%	95%	92%	95%
	Viola	3%	16%	18%	14%	0.2%	44%	37%	25%
	Cello	49%	60%	82%	53%	–	–	–	–

4. Discussion

This case study provided a venue for examining the dynamics of visual-motor behaviour in string musicians during quartet performance. We examined head motion and gaze patterns across a series of conditions that were designed to disrupt (Blind, Score-directed, Violin-isolated), enable (Normal-rehearsal, Replication-rehearsal) or promote (Concert) visually communicative behaviour. We tested four predictions: (1) that the musicians would vary their body motion across repeat performances of the same material, especially during periods of high expressivity; (2) that the musicians would engage in more visual-motor interaction in performances where they could watch each other freely than in conditions where they could not; (3) that the musicians would engage in more visual-motor interaction in the concert than in the rehearsal performances that were given under free-gaze conditions; and (4), that the musicians would engage in more visual-motor interaction in the Score-directed condition, where a peripheral view of each other was possible, than in the Blind condition, and further, that the Violin-isolated condition would heighten interactivity between the 2nd violinist, violist and cellist.

Our main findings can be summarised as follows. QoM was lowest in the Blind and Score-directed performances, suggesting that the constrained conditions of these performances discouraged communicative motion. Musicians' abilities to see each other peripherally in the Score-directed performance was insufficient to encourage more movement. The musicians differed slightly in how stable their head motion was across performances, but all tended to move more variably at specific points in the piece, suggesting moments of greater expressive flexibility. Notably, QoM magnitudes and variability were lowest in the first section of the piece, and both increased in later sections. We found some evidence that musicians may have influenced each other's motion; for example, the average closeness of their heads differed between conditions. The cellist's head motion was most distinctive, perhaps, reflecting her distinct musical role in the ensemble.

4.1. Heightened interactivity in normal-rehearsal and concert performances

We did not observe substantial differences between the Normal-rehearsal condition and the Concert condition. Thus, our hypothesis that the presence of an audience and examiner would encourage greater visual-motor interaction between musicians (more movement, more time spent watching each other) was not supported. We likewise saw no evidence of the musicians ‘freezing up’, or interacting less with each other during the Concert than during the Normal-rehearsal.

This lack of effect is in contrast to previous studies that have shown higher-intensity and more emphasised body movements when musicians perform in front of an audience (Biasutti et al., 2016; Moelants et al., 2012). Since these studies, as well as our own, comprise case studies, there are several possible explanations for the difference in results relating to individual differences in personality and experience level among ensemble members. For example, the musicians in the current study may still be developing their personal performance styles and less flexible in their range of expressive motion and communicative behaviour than a more experienced quartet would be. Our quartet may also have been restrained in their performance behaviour because the concert also constituted their semester exam; perhaps they would have performed more freely in a non-exam concert setting.

However, we did see differences in the intensity of visual-motor interaction across rehearsal-like conditions, suggesting that our quartet did respond to the different conditions with varying degrees of interactivity. We therefore propose an alternative explanation for the similarity of the Normal-rehearsal and Concert performances.

Prior research has discussed how music performances are always shaped by the unique set of circumstances that surround them (van der Schyff et al., 2018). The circumstances surrounding the Normal-rehearsal performance may have been such that the quartet was motivated to intensify their body-based interactions relative to the Repetition-rehearsal. The Normal-rehearsal came immediately following two unusual conditions, where the musicians may have felt constrained by their odd seating arrangement (Blind, performance 1) and their inability to look around as they would like (Score-directed, performance 2). On top of these constraints, the musicians were playing for the first time in an unfamiliar lab setting. The Normal-rehearsal, performance 3, may have come as a relief to them, as they were finally able to perform in a more free and comfortable way. We do see some decrease in musicians' QoM as well as reduced time spent watching each other between the Normal-rehearsal and Replication-rehearsal (performance 5), suggesting that the Normal-rehearsal may have encouraged increased communicative behaviour compared to the Replication-rehearsal. The musicians also reported feeling slightly more nervous during the Replication-rehearsal, as it was the final performance before the Concert.

This enhanced interactivity in the Normal-rehearsal performance effectively constitutes an order effect, as it seems to have emerged as a result of the contrast between the Normal-rehearsal and previous conditions. While empirical studies usually aim to avoid order effects, in this case, it highlights how subtle changes in performance context and constraints can have a marked effect on musicians' experience and performance behaviour. This is an important and novel finding that supports our general hypothesis that musicians respond to more ‘social’ playing conditions with an increase in interactive behaviour (see Section 1.3). Indeed, we can now build on this hypothesis and suggest that the sociality of different playing conditions is relative, and that musicians may respond to perceived differences in sociality across performance environments. This finding also adds to a growing collection of studies that show how ensembles collectively adapt their performance behaviour to accommodate situational factors such as which channels of communication are available (Bishop & Goebl, 2015; Goebl & Palmer, 2009) and how well formed the ensemble's interpretation is (Bishop et al., 2019b; Glowinski et al., 2015). Further research should test how ensembles respond to subtle differences in perceived sociality.

4.2. Examining the role of the 1st violinist

Looking at the individual QoM results, we see some differences between musicians in how they responded to the concert setting. In particular, the 1st violinist moved less during the Concert than during the Normal- and Replication-rehearsals, while the other musicians moved similarly across the two conditions (if not more in the Concert than in the Replication-rehearsal). This finding is somewhat at odds to the results of the piano duet study conducted by Goebl and Palmer (2009), where pianists who were assigned the role of leader moved their fingers higher than did pianists who were assigned the role of follower. Pianists' finger motion, while directly involved in sound production, is also of potential communicative value to a duet partner. The reduced head motion that we observed on the part of the 1st violinist may have been a stress response, as he also had substantially larger pupil diameters during the Concert compared to the other musicians (Bishop et al., 2021). This consistent elevation of pupil diameter is suggestive of increased arousal and mental effort. Davidson and Good (2002) also noted that the 1st violinist in their study was especially anxious during the concert performance.

The 1st violinist in the current study was also distinctive in his gaze behaviour. He spent very little time watching his co-performers, in contrast to the other musicians. On the other hand, he was watched by his co-performers more than any other musician. Davidson and Good (2002) likewise found that the musicians in the quartet that they studied watched the 1st violinist more than they watched each other. This unbalanced pattern of gazing likely reflects the traditional role of the 1st violinist as the leader of the quartet. As leader, the 1st violinist may be expected to minimise the extent to which they adapt to their co-performers, while the others adapt to them (though see Konvalinka et al., 2010). Leader/follower affects on visual attention have also been shown in piano duos, where followers looked at leaders for longer intervals than leaders looked at followers (Kawase, 2014a).

The Violin-isolated performance examined the effects of visually isolating the 1st violinist from the rest of the quartet. QoM across the group decreased slightly (though not significantly) and the musicians spent somewhat less time watching each other, compared to the Normal-rehearsal condition. QoM and gaze patterns were similar, however, to the Replication-rehearsal condition. Thus, the visual absence of the 1st violinist seemed neither to deter nor promote communicative movement. While the effects of visually isolating the 1st violinist have not been studied before, research on body motion in string quartets has shown that other quartet members are quick to adapt their body motion to interpretive changes that are spontaneously introduced by the 1st violinist (Glowinski, Gnecco, et al., 2013). Thus, a quartet may readily respond to cues given by the 1st violinist, but can play relatively undisrupted if no visual cues are given.

In general, further study with combined qualitative and quantitative methods is needed to examine the role of the 1st violinist in classical string quartets. First violinists may feel greater accountability for the success of the performance (and potentially greater stress and anxiety) as a result of their traditional role as ensemble leader, and this may affect their expressive body motion and the way they interact with their co-performers.

4.3. Strength of visual-motor interactions across musician pairs

Our measures of interperformer motor interaction suggested that the strength of musicians' influence on each other varied across performances and was strongest in the Normal-rehearsal. Out of the performances in which musicians were seated facing each other, they spent the most time with their heads close together (i.e. average normalised perimeter was smallest) in the Normal-rehearsal. This measure of motor interaction is similar to a measure used by Badino et al. (2014), who calculated the distance from each performer's head to a central point that was defined by the musicians (the ‘ear’ of the quartet; see also Glowinski et al., 2012, 2015; Glowinski, Gnecco, et al., 2013). As we discussed earlier, Glowinski, Mancini, et al. (2013) showed increased convergence to the ear during ‘perturbed’ playing conditions, when the 1st violinist was tasked with introducing new expressive nuances during the performance. Our finding is in line with the suggestion, discussed above, that the Normal-rehearsal may have encouraged increased interaction between musicians, compared to what we might normally see in rehearsal-like conditions.

We also made a novel evaluation of the strength of interaction within different pairings of musicians (in terms of QoM curve similarity), using the 1st violinist/2nd violinist pair as a baseline. Greater similarity in QoM curves occurred between the violinists and violists than between these musicians and the cellist. The distinctiveness of the cellist's motion might relate to her unique musical role in the ensemble and the constraints of her instrument – cellists are more fixed in position than violinists and violists, but freer to separate head motion from sound-producing motion, as the head does not support the instrument. Previously, Chang et al. (2017) showed that the direction of information flow between string ensemble members can vary when leadership roles are manipulated. The current study did not manipulate leadership assignments, but showed that interperformer relationships might vary depending on the different roles that emerge naturally during quartet performance. Further research is needed to determine whether the cellist's distinctiveness is generalisable across ensembles.

It is interesting to note that the cellist moved most distinctively, given that she spent more time watching her co-performers than anyone else. Indeed, she differed most from the 1st and 2nd violinists despite spending a cumulative 42% of performance time, on average, watching the two of them. Overall, three of the four musicians spent a substantial amount of time watching their co-performers. The 1st violinist, while the most watched of all the musicians, looked almost exclusively at the score.

4.4. Role of peripheral vision in interperformer interaction

Our results do not give a clear indication of how much motion information musicians were able to extract from their peripheral visual fields. QoM was reduced in the Score-directed performance compared to later performances with free-gaze, and interperformer interaction was also reduced, as suggested by a greater average quartet perimeter. Together, these results could indicate a poor exchange of motor information through peripheral vision. Performers might also have felt constrained or preoccupied by the instruction to restrict their gaze, leading to a lack of motivation to communicate visually. In follow-up discussions with the musicians, they said that they sometimes deliberately try to position their gaze in order to see their co-performers peripherally. More controlled study would be needed to determine how much musicians influence each other through peripherally viewed motion.

4.5. Head motion and musical structure

The musicians demonstrated varying flexibility in QoM across the course of the piece, with peaks in flexibility (i.e. periods of high between-performance/within-performer variability) occurring at the ends of Sections 2 and 4 – the sections where average QoM was highest. The end of Section 2 is a high-energy passage with fortissimo markings, which contrasts the quiet start of Section 3. The end of Section 4 is likewise a high-energy, loud passage which contrasts the quiet start of Section 5. The climactic passages found in Sections 2 and 4 may allow for greater flexibility in expressive body motion than we see elsewhere in the piece.

It is important to note that QoM and sound-level values were not strongly correlated, so the increased QoM that we saw in these climactic sections was not exclusively a result of playing louder. Rather, increased QoM (and increased motion flexibility) could relate more generally to an increase in emotional intensity. Research with solo musicians has also examined the consistency of ancillary motion across repeat performances of the same material. Solo clarinettists were found to produce more consistent trajectories with the bell of their instrument at points of particular expressive importance (Teixeira et al., 2015), a finding that is in contrast to our quartet's tendency to move more variably at such points. The variability in motion patterns that we observed likely reflects the different performance conditions that we tested (which may have elicited variable degrees of musical engagement from the musicians) and the ensemble rather than solo setting.

Prior research has shown that ensemble musicians' motor interaction varies according to the expressive intensity of their performance, with greater interactivity occurring during performances that are more expressive (Chang et al., 2019). Quartet musicians have also been shown to respond to unexpected expressive nuances, introduced by the 1st violinist, with changes in motion dynamics that signal increased attention to the 1st violinist (Glowinski et al., 2015). The findings of the current study make a novel contribution to this literature by showing that body motion at expressively significant moments is most affected by changes in playing conditions.

4.6. Limitations

We are cautious about generalising the results of this case study. Body motion and gaze are highly individualistic among performing musicians, and factors such as personality, skill level, style of training and within-ensemble social factors are likely to influence musicians' ability and willingness to interact in expressive/communicative ways, as well as their ability to pick up and respond to fluctuations in each other's behaviour. Furthermore, classical string quartets have a rather prescribed structure of performer roles (particularly when performing music of the classical period), which is in contrast to the less defined roles of some other types of ensembles.

An additional limitation was our inability to randomise the order of performance conditions. As discussed above, the order that we chose resulted in an inflated degree of interperformer interaction in the Normal-rehearsal; although we would also note that this is an interesting effect which shows how even subtle changes in performance conditions result in musicians engaging differently with each other. Finally, it should be noted that the ensemble might have behaved differently during this experiment than they do outside the lab as a result of the unfamiliar conditions and potentially uncomfortable equipment that they were wearing (eye tracking glasses and jackets with markers).

We would also note that this study suffered from some data loss in both motion capture and gaze recordings. A string quartet presents a sizable challenge for successful motion tracking because the presence of four moving bodies (plus their instruments and music stands) creates many sources of possible marker occlusion. Our setup was complicated by the fact that the musicians were sharing the lab space with an audience of 40 people, and therefore positioned closer to the edge of the capture volume than would have been ideal. While the musicians' head markers were well tracked, the glasses markers of the 2nd violinist and violist were not tracked well enough to provide usable data. This was most likely because of how these musicians were positioned near the edge of the capture volume. Ideally, for best capture results, the quartet would be positioned in the centre of the capture volume. We also failed to obtain usable gaze data from the 2nd violinist during the concert performance. To prevent data loss with eye tracking glasses, it is necessary to monitor the recordings and stop and re-calibrate if tracking is not successful. This was not possible during the concert, but would be advisable during normal lab recording sessions.

4.7. Conclusions

In conclusion, this case study has shown that members of a string quartet respond to changes in performance condition – whether constraining or motivating – with changes in quantity of motion and gaze. The musicians demonstrated a clear drive to visually engage with each other when it was possible to do so (i.e. when the performance conditions allowed it). They demonstrated this drive in both rehearsal and concert conditions, suggesting that they do not need the pressure of an audience to trigger heightened engagement. We would surmise that the musicians in the current study derive a sense of togetherness from visual-motor interaction which enhances their experience of performing together even if the effects on coordination are minimal. Our results are also suggestive of how patterns of visually communicative behaviour may differ between musicians according to their role in the ensemble, and become more or less variable across different parts of a piece. Future studies should test how factors such as social dynamics and expertise affect ensemble musicians' visual-motor interaction.

Acknowledgments

We are grateful to Assoc. Prof. Werner Goebl (Dept. of Music Acoustics–Wiener Klangstil, University of Music and Performing Arts Vienna) and Dr. Tetsuto Minami (Toyohashi University of Technology) for the loan of their eye tracking glasses. We also wish to thank the Borealis String Quartet for their enthusiastic participation in this project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the University of Oslo and the Research Council of Norway through its Centres of Excellence scheme, project number 262762.