The Handy Metaphor: Bimanual, touchless interaction for the internet of musical things

Abstract

We describe the development of a new ubiquitous music metaphor for creative action involving the use of hand gestures, Handy. The metaphor was deployed through two proof-of-concept, low-cost prototypes based either on motion sensors connected to an embedded micro-controller (Handy H) or by repurposing the webcam of a portable computer (Handy S). The evaluations showed positive results (slightly better with musicians than with non-musicians) in the following factors: ease of use and productivity (Handy H); engagement, productivity and collaboration support (Handy S). We discuss the implications of this outcome, targeting the interaction with the Internet of Musical Things.

Keywords:

• Ubiquitous music
• IoMusT
• timbre interaction
• ubimus
• internet of musical things
• touchless interaction
• bimanual interaction

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 http://www.micropik.com/PDF/HCSR04.pdf.

3 https://www.youtube.com/watch?v=rJle7n4Xw1Q&t=618s.

2 https://www.youtube.com/watch?v=x-wFs-2bhBY.

4 https://www.leapmotion.com/.

5 http://idialab.org/chromachord-a-virtual-musical-instrument/.

6 As an attempt to explore alternatives to the acoustic-instrumental paradigm, we adopt the concept of Gelassenheit for usage in creative musical practices. Gelassenheit is a term coined by Heidegger (1966). 1. Calculative thinking is understood as a ‘scientific-artistic method’ involving measuring, collecting data and replicating results. 2. Meditative thinking is the aptitude to be open to unpredictable actions and unexpected events, i.e. to the mystery of life (Heidegger, 1966). Heidegger proposes Gelassenheit strategies as new forms of thinking to be achieved through openness to experience. From an artistic perspective, Aliel (2017) considers it as a process of adaptation and modification of self-reflective strategies (Aliel, 2017). The absence of control acts as an impulse to unexpected results, placing the artists in a frame of mind adaptive to new contexts. With reduced control, the artists also expand their procedural choices, introducing divergent conceptions that fall beyond their preexisting knowledge. Thus, meditative thinking is not univocally bound to a product (as it is usually the case in calculative thought-forms). It highlights the particularities of each artistic experience (see also Dewey, 1934/2005 as a predecessor of part of these ideas).