Physical modelling for interactive installations and the performing arts

Abstract

This article explores new artistic possibilities through four different examples of digital environments where live performance and virtual physical models come together. Physical models (specifically mass-spring systems) are used to enhance mappings between human movement and digital rendering. In the case of multi-modal rendering, physical models improve the cross-modal coherence. Given a physical model's capacity to simulate physical behaviours, they can also be used to visualise or to simulate human movement qualities, and even to simulate a dancer's virtual partner. More applications certainly remain to be tested, but the present study opens new avenues for the exploration of the complex and physically plausible behaviours of mass-spring systems in the context of both artistic practice and art pedagogy.

Keywords:

• computer graphics
• augmented performance
• digital scenography
• physical models
• mass-spring systems

Notes on contributors

Sarah Fdili Alaoui is postdoctoral researcher in the School of Interactive Arts and Technology at Simon Fraser University in Vancouver, Canada. Previously, she was a researcher at IRCAM, LIMSI-CNRS and University Paris-Sud. Her research concerns the study of dance gestures and their use in interaction design. She is involved in several collaborations with dancers and choreographers, to create interactive systems for dance performance and pedagogy. Sarah holds an engineering degree in Computer Science and Applied Mathematics from ENSIMAG and a Masters in Applied Mathematics from the University Joseph Fourier. She received a PhD in Computer Science from the University Paris-Sud on the analysis and representation of dance gestures, where she studied the contribution of dance movement qualities in human–computer interaction. Sarah has also been trained in classical and contemporary dance and has performed in contemporary dance companies and improvised dance collectives. Sarah is currently completing a qualification as a Certified Movement Analyst at the Laban/Bartenieff Institute for Movement Studies (LIMS). She is interested in bridging scientific and experiential research in the movement-based arts to radically alter the study of movement as a modality, strategy and method for transforming our understanding of human knowledge and cognition.

Cyrille Henry is working at the intersection of art, computer science and scientific research. His work incorporates the use of sensors, movement analysis, physical modelling and image processing as well as real-time audio-visual synthesis. He previously worked as head of the hardware department of La Kitchen for four years (2001–2005) with the aim of creating sensor interfaces and to develop their uses in an artistic context (performing arts, interactive installation, contemporary music). Cyrille holds a Masters in Physics from the University du Maine. Since 2006, he has been independent developer and artist in the collective chdh. He collaborates with various dance companies and artists to create elaborate devices for digital installations and interactive stages.

Christian Jacquemin is a Professor in Computer Science at the University of Paris-Sud and co-leads the Virtuality, Interaction, and Design and Art (VIDA) project at CNRS-LIMSI with Nathalie Delprat. From 1987 to 2000, he was working in Natural Language Processing. Since 2000, his work is focused on interactive 3D graphics and its applications for information visualisation, virtual and mixed realities, and visual arts. He is involved in several collaborations on the artistic application of interactive graphics (theatre, art installations, sound and graphic design, etc.). His current research interests include information visualisation (visual analytics), interactive 3D graphics and audio, advanced graphics rendering and applications to arts. He has collaborated with several artists and designers on the realisation of augmented reality environments for art installation, theatre plays and multimedia performances. He has published at major conferences in computational linguistics, information retrieval, information visualisation, multimedia and digital and performing arts.

Notes

1. The Nine Evenings, a series of avant-garde live performances and new technologies organised by Robert Rauschenberg and Billy Kluwer in October 1966, together with earlier experiences such as Maurice Bejart's CYSP 1 ballet with Nicolas Shöffer's interactive sculpture in June 1956 have shown how entangled performing arts and interactive media can be, to the point that the digital work has become a full partner in performance design.

2. A set of experiments in collaboration with dancer and choreographer Marion Cavaillé have investigated the interest of physical models on stage, both for improvisation and for live performance. In the set-up, physical models were presented on a vertical screen and the dancer was improvising in front of the screen, watching the visuals and trying to improvise with them. Semi-directed interviews with the dancer have shown that she could ‘play’ with the models and progressively build up a choreographic relationship with the model as long as it could be perceived as responsive and at the same time endowed with autonomy (see Chiselling Bodies section below).

3. Documentation of Effet papillon performance by Contour Progressif in collaboration with Cyrille Henry. Accessed 1 July 2014. http://www.contour-progressif.net/projets/spectacle/effet-papillon

4. Documentation and videos of DS/DM installation. Accessed 1 July 2014. http://www.researchcatalogue.net/view/?weave=1585 and http://sarah.alaoui.free.fr/spip.php?article1

5. Documentation and video of the rehearsal of the performance Chiselling Bodies. Accessed 1 July 2014. http://sarah.alaoui.free.fr/spip.php?article34

6. Artist collective chdh. Accessed 1 July 2014. www.chdh.net

7. See Verplank, Mathews, and Shaw (2000) for more information about scanned synthesis.

8. Documentation and videos of the album vivarium. Accessed 1 July 2014. http://www.chdh.net/vivarium

9. Documentation and videos of the performance egregore. Accessed 1 July 2014. http://www.chdh.net/egregore