Improvising through the senses: a performance approach with the indirect use of technology

ABSTRACT

This article explores and proposes new ways of performing in a technology-mediated environment. We present a case study that examines feedback loop relationships between a dancer and a pianist. Rather than using data from sensor technologies to directly control and affect musical parameters, we captured data from a dancer’s arm movements and mapped them onto a bespoke device that stimulates the pianist’s tactile sense through vibrations. The pianist identifies and interprets the tactile sensory experience, with his improvised performance responding to the changes in haptic information received. Our system presents a new way of technology-mediated performer interaction through tactile feedback channels, enabling the user to establish new creative pathways. We present a classification of vibrotactile interaction as means of communication, and we conclude how users experience multi-point vibrotactile feedback as one holistic experience rather than a collection of discrete feedback points.

KEYWORDS:

• Vibrotactile
• interactive systems
• dance
• feedback loop
• haptic
• audiovisual

Acknowledgements

We would like to thank the dancers that took part in the evaluation process, Hayley Barker and KJ Mortimer and the dancer Helen Calcutt and pianist Niccolò Granieri that took part in the case study. We also like to thank the reviewers for their valuable feedback and comments.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Tychonas Michailidis is a research fellow at Solent University in Southampton, UK. His research interests focus on Human-Computer Interaction, haptics, sensor technologies and music. In particular, his research examines how vibrotactile and haptic feedback experience can advance interactive systems, as well as how it can be used as a communication tool in the arts and have real world applications.

James Dooley an artist based in Birmingham UK, James’ audiovisual interaction design examines approaches to combining audio, visual and environmental elements in ways that produce emergent structures and forms. The individual components of his work play with the boundary between autonomy and synchrony, revealing their hidden connections and creating a sense of place.

Niccolò Granieri PhD student at the Integra Lab of the Royal Birmingham Conservatoire in Birmingham, UK, Granieri’s research focuses on the study and implementation of expressive microgestures in piano playing in the creation of an accessible and interactive digital music environment.

Balandino Di Donato Research Assistant at Goldsmith, University of London (UK). His research focuses on Music and Human-Computer Interaction; specifically on finding ways to generate and transform audiovisual mediums through biosignals.

ORCID

Tychonas Michailidis http://orcid.org/0000-0002-7196-3811

James Dooley http://orcid.org/0000-0002-7972-6888

Niccolò Granieri http://orcid.org/0000-0002-0477-798X

Balandino Di Donato http://orcid.org/0000-0001-6993-2445

Notes

1. https://www.myo.com.

2. http://x-io.co.uk.

3. Vibrating amplitude (G) refers to the intensity of the vibration of motors.

4. https://cycling74.com.

5. Similar results are also proposed in the work of Frid et al. (2014).

6. Excerpts from two sessions with the dancers as well as part of the evaluation process can be viewed online https://youtu.be/n1×0fVHA2iw.

7. The visual content realized using the GEM library for Pure Data were generated through mapping the piano sound's acoustic properties mainly the amplitude and frequency of the piano control the length, colour and position of the lines.

8. Excerpts from the case study between the dance and the pianist can be viewed online https://youtu.be/oxxiF0y7hFY.