Creative engagement in interactive immersive environments

Abstract

This paper presents two independent case studies selected from n = 11, an 11-year-old male with Down's Syndrome and another 12-year-old male with Autistic Spectrum Disorder (ASD). An audiovisual immersive interactive environment was developed to encourage creative interaction and expression from the participants. The feedback was via a microphone attached to a sound processor to capture any vocal utterances or speech, additionally the child's gestures were captured through a digital camera and mapped to computer vision algorithms to create a ‘moving painting’, and then further to a MIDI sound source that responded to the movements of the child. Results indicated increased engagement spans whilst in the environment, atypical communicative responses in the case of the ASD child; highly expressive gestural movements that vividly illustrated aspects of the theory of movement as postulated by Laban (1963) and Laban and Lawrence (1974); and increased vocal utterances.

Keywords:

• expressive gesture
• engagement
• interaction
• multisensory environments

Notes

www.informus.dist.unige.it

Alesis Airsynth™-www.alesis.com

www.korg.com

The Aura Interactor was originally developed for use with computer games but is no longer manufactured.

http://www.dfki.de/∼kipp/anvil

http://www.unige.ch/fapse/emotion/resmaterial//GAQ_English.pdf