Go-with-the-Flow: Tracking, Analysis and Sonification of Movement and Breathing to Build Confidence in Activity Despite Chronic Pain

REFERENCES

• Aung, M. S. H., Kaltwang, S., Romera-Paredes, B., Martinez, B., Singh, A., Cella, M., … Bianchi-Berthouze, N. (in press). The automatic detection of chronic pain-related expression: Requirements, challenges and a multimodal dataset. IEEE Transactions in Affective Computing.
   Google Scholar
• Bengtsson, S. L., Ullén, F., Ehrsson, H. H., Hashimoto, T., Kito, T., Naito, E., … Sadato, N. (2009). Listening to rhythms activates motor and premotor cortices. Cortex, 45, 62–71. doi:10.1016/j.cortex.2008.07.002
   PubMed Web of Science ®Google Scholar
• Botvinick, M., & Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature, 391, 756. doi:10.1038/35784
   PubMed Web of Science ®Google Scholar
• Bradley, M. M., & Lang, P. J. (1999). International affective digitized sounds (IADS): Stimuli, instruction manual and affective ratings (Tech. Rep. No. B-2). Gainesville: The Center for Research in Psychophysiology, University of Florida.
   Google Scholar
• Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. doi:10.1191/1478088706qp063oa
   Google Scholar
• Breivik, H., Collett, B., Ventafridda, V., Cohen, R., & Gallacher, D. (2006). Survey of chronic pain in Europe: Prevalence, impact on daily life, and treatment. European Journal of Pain, 10, 287. doi:10.1016/j.ejpain.2005.06.009
   PubMed Web of Science ®Google Scholar
• Bruckner, H., Theimer, W., & Blume, H. (2014). Real-time low latency movement sonification in stroke rehabilitation based on a mobile. Proceedings of the IEEE 2014 International Conference on Consumer Electronics.
   Google Scholar
• Camurri, A. (1995). Interactive dance/music systems. Proceedings of the ICMC 1995 International Computer Music Conference.
   Google Scholar
• Castagné, N., & Cadoz, C. (2005). A goals-based review of physical modelling. Proceedings of the ICMC 2004 International Computer Music Conference.
   Google Scholar
• Cepeda, M., Carr, D., & Lau, J. (2006). Music for pain relief. Cochrane Database of Systematic Reviews 2. doi:10.1002/14651858.CD004843.pub2
   Google Scholar
• Della Volpe, R., Popa, T., Ginanneschi, F., Spidalieri, R., Mazzocchio, R., & Rossi, A. (2006). Changes in coordination of postural control during dynamic stance in chronic low back pain patients. Gait & Posture, 24, 349–355. doi:10.1016/j.gaitpost.2005.10.009
   PubMed Web of Science ®Google Scholar
• De Vignemont, F., Ehrsson, H. H., & Haggard, P. (2005). Bodily illusions modulate tactile perception. Current Biology, 15, 1286–1290. doi:10.1016/j.cub.2005.06.067
   PubMed Web of Science ®Google Scholar
• Donaldson, L. (2009). 150 years of the annual report of the chief medical officer: On the state of public health 2008. London, UK: Department of Health.
   Google Scholar
• Dubus, G. (2012). Evaluation of four models for the sonification of elite rowing. Journal on Multimodal User Interfaces, 5, 143–156. doi:10.1007/s12193-011-0085-1
   Web of Science ®Google Scholar
• Dubus, G., & Bresin, R. (2013). A systematic review of mapping strategies for the sonification of physical quantities. Plos One, 8, e82491. doi:10.1371/journal.pone.0082491
   PubMed Web of Science ®Google Scholar
• Effenberg, A. O. (2005). Movement sonification: Effects on perception and action. IEEE Multimedia, 12, 53–59. doi:10.1109/MMUL.2005.31
   Web of Science ®Google Scholar
• Effenberg, A. O., Fehse, U., & Weber, A. (2011). Movement sonification: Audiovisual benefits on motor learning. BIO Web of Conferences, 1, 00022.
   Google Scholar
• Effenberg, A. O., Weber, A., Mattes, K., Fehse, U., & Mechling, H. (2007). Motor learning and auditory information: Is movement sonification efficient? Journal of Sport & Exercise Psychology, 29, 66.
   Google Scholar
• Felipe, S., Singh, A., Bradley, C., Williams, A., & Bianchi-Berthouze, N. (2015). Roles for personal informatics in chronic pain. Proceedings of the 9th International Conference on Pervasive Health Computing Technologies for Healthcare.
   Google Scholar
• Gatchel, R. J., Peng, Y. B., Peters, M. L., Fuchs, P. N., & Turk, D. C. (2007). The biopsychosocial approach to chronic pain: Scientific advances and future directions. Psychological Bulletin, 133, 581–624. doi:10.1037/0033-2909.133.4.581
   PubMed Web of Science ®Google Scholar
• Ghez, C., Rikakis, T., Dubois, R. L., & Cook, R. (2000). An auditory display system for aiding interjoint coordination. Proceedings of the ICAD 2000 International Conference on Auditory Display.
   Google Scholar
• Goto, M. (2007). Active music listening interfaces based on signal processing. Proceedings of the IEEE 2007 International Conference on Acoustics, Speech, and Signal Processing.
   Google Scholar
• Gromala, D., Tong, X., Choo, A., Karamnejad, M., & Shaw, C. (2015). The virtual meditative walk: Virtual reality therapy for chronic pain management. Proceedings of the CHI 2015 Conference on Human Factors in Computer Systems. New York, NY: ACM.
   Google Scholar
• Hammal, Z., & Cohn, J. F. (2012) Automatic detection of pain intensity. Proceedings of the ICMI 2012 International Conference on Multimodal Interaction. New York, NY: ACM.
   Google Scholar
• Harding, V., & Williams, A. (1995). Extending physiotherapy skills using a psychological approach: Cognitive-behavioural management of chronic pain. Physiotherapy, 81, 681–688.
   Google Scholar
• Hasegawa, S., Ishijima, S., Kato, F., Mitake, H., & Sato, M. (2012). Realtime sonification of the center of gravity for skiing. Proceedings of the AH 2012 Augmented Human International Conference. New York, NY: ACM.
   Google Scholar
• Hermann, T. (2008). Taxonomy and definitions for sonification and auditory display. Proceedings of the ICAD 2008 International Conference on Auditory Display.
   Google Scholar
• Hermann, T. (2011). Model-based sonification. In T. Hermann, A. Hunt, & J. G. Neuhoff ( Eds.), The sonification handbook (pp. 399–428). Berlin, Germany: Logos Verlag.
   Google Scholar
• Hermann, T., & Hunt, A. (2005). Guest editors’ introduction: An introduction to interactive sonification. IEEE Multimedia, 12, 20–24. doi:10.1109/MMUL.2005.26
   Web of Science ®Google Scholar
• Hermann, T., & Zehe, S. (2011). Sonified aerobics—Interactive sonification of coordinated body movements. Proceedings of the ICAD 2011 International Conference on Auditory Display.
   Google Scholar
• Holden, M. (2005). Virtual environments for motor rehabilitation: Review. CyberPsychology & Behavior, 8, 187–211. doi:10.1089/cpb.2005.8.187
   PubMedGoogle Scholar
• Höner, O., Hermann, T., & Grunow, C. (2004). Sonification of group behavior for analysis and training of sports tactics. Proceedings of the ISon 2004 International Workshop on Interactive Sonification.
   Google Scholar
• Hunt, A., & Wanderley, M. M. (2002). Mapping performer parameters to synthesis engines. Organised Sound, 7, 97–108. doi:10.1017/S1355771802002030
   Google Scholar
• Hunt, A., Wanderley, M. M., & Kirk, R. (2000). Towards a model for instrumental mapping in expert musical interaction. Proceedings of the ICMC 2000 International Computer Music Conference. Berlin, Germany: ICMA.
   Google Scholar
• Jansen-Kosterink, S. M., Huis in T Veld, R. M. H. A., Schönauer, C., Kaufmann, H., Hermens, H. J., & Vollenbroek-Hutten, M. M. R. (2013). A serious exergame for patients suffering from chronic musculoskeletal back and neck pain: A pilot study. Games for Health Journal, 2, 299–307. doi:10.1089/g4h.2013.0043
   PubMed Web of Science ®Google Scholar
• Juslin, P. N., & Sloboda, J. A. (2001). Music and emotion: Theory and research. Oxford, NY: Oxford University Press.
   Google Scholar
• Juslin, P. N., & Västfjäll, D. (2008). Emotional responses to music: The need to consider underlying mechanisms. Behavioral Brain Sciences, 31, 559–575. doi:10.1017/S0140525X08005293
   PubMed Web of Science ®Google Scholar
• Kaltwang, S., Rudovic, O. & Pantic, M. (2012). Continuous pain intensity estimation from facial expressions. In G. Bebis et al. (Eds.), Advances in visual computing (Vol. 7432, pp. 368–377). Heidelberg, Germany: Springer.
   Google Scholar
• Kenyon, G. P., & Thaut, M. H. (2005). Rhythmic-drive optimization of motor control. In M. H. Thaut (Ed.), Rhythm, music and the brain: Scientific foundations and clinical applications (pp. 85–112). New York, NY: Routledge Chapman & Hall.
   Google Scholar
• Kerns, R. D., Sellinger, J., & Goodin, B. R. (2011). Psychological treatment of chronic pain. Annual Review of Clinical Psychology, 7, 411–434. doi:10.1146/annurev-clinpsy-090310-120430
   PubMed Web of Science ®Google Scholar
• Kleiman-Weiner, M., & Berger, J. (2006). The sound of one arm swinging: A model for multidimensional auditory display of physical motion. Proceedings of the ICAD 2006 International Conference on Auditory Display.
   Google Scholar
• Lavie, N. (2005). Distracted and confused?: Selective attention under load. Trends in Cognitive Sciences, 9, 75–78. doi:10.1016/j.tics.2004.12.004
   PubMed Web of Science ®Google Scholar
• Lee, A. S., Cholewicki, J., Reeves, N. P., Zazulak, B. T., & Mysliwiec, L. W. (2010). Comparison of trunk proprioception between patients with low back pain and healthy controls. Archives of Physical Medicine and Rehabilitation, 91, 1327–1331. doi:10.1016/j.apmr.2010.06.004
   PubMed Web of Science ®Google Scholar
• Legrain, V., Damme, S. V., Eccleston, C., Davis, K. D., Seminowicz, D. A., & Crombez, G. (2009). A neurocognitive model of attention to pain: Behavioral and neuroimaging evidence. Pain, 144, 230–232. doi:10.1016/j.pain.2009.03.020
   PubMed Web of Science ®Google Scholar
• Lenti Boero, D., & Bottoni, L. (2008). Why we experience musical emotions: Intrinsic musicality in an evolutionary perspective. Behavioral and Brain Sciences, 31, 585–586. doi:10.1017/S0140525X08005396
   Web of Science ®Google Scholar
• Lewis, G. N., & Rosie, J. A. (2012). Virtual reality games for movement rehabilitation in neurological conditions: How do we meet the needs and expectations of the users? Disability & Rehabilitation, 34, 1880–1886. doi:10.3109/09638288.2012.670036
   PubMed Web of Science ®Google Scholar
• Liu, G.-Z., Huang, B.-Y., & Wang, L. (2011). A wearable respiratory biofeedback system based on generalized body sensor network. Telemedicine and E-Health, 17, 348–357. doi:10.1089/tmj.2010.0182
   PubMed Web of Science ®Google Scholar
• Mancini, M., Ach, L., Bantegnie, E., Baur, T., Berthouze, N., Datta, D., & Wagner, J. (2014). Laugh when you’re winning. In Y. Rybarczyk (Ed.), Innovative and creative developments in multimodal interaction systems, proceedings of 9th IFIP WG 5.5 international summer workshop on multimodal interfaces, eNTERFACE 2013 (Vol. 425, pp. 50–79). Lisbon, Portugal: Springer.
   Google Scholar
• Martel, M. O., Thibault, P., & Sullivan, M. J. L. (2010). The persistence of pain behaviors in patients with chronic back pain is independent of pain and psychological factors. Pain, 151, 330–336. doi:10.1016/j.pain.2010.07.004
   PubMed Web of Science ®Google Scholar
• Matsubara, M., Kadone, H., Iguchi, M., Terasawa, H., & Suzuki, K. (2013). The effectiveness of auditory biofeedback on a tracking task for ankle joint movements in rehabilitation. Proceedings of the ISON 2013 Interactive Sonification workshop.
   Google Scholar
• Meng, H., & Bianchi-Berthouze, N. (2014). Affective state level recognition in naturalistic facial and vocal expressions. IEEE Transactions on Systems, Man and Cybernetics Part B: Cybernetics, 44, 315–328.
   PubMed Web of Science ®Google Scholar
• Menzer, F., Brooks, A., Halje, P., Faller, C., Vetterli, M., & Blanke, O. (2010). Feeling in control of your footsteps: Conscious gait monitoring and the auditory consequences of footsteps. Cognitive Neuroscience, 1, 184–192. doi:10.1080/17588921003743581
   PubMed Web of Science ®Google Scholar
• Nazemi, M., Mobini, M., Kinnear, T., & Gromala, D. (2013). Soundscapes: A prescription for managing anxiety in a clinical setting. Proceedings of the CHI 2013 Conference on Human Factors in Computer Systems. New York, NY: ACM.
   Google Scholar
• Olugbade, T. A., Aung, M. S. H., Marquardt, N., De C. Williams, A., & Bianchi-Berthouze, N. (2014). Bi-modal detection of painful reaching for chronic pain rehabilitation systems. Proceedings of the ICMI 2014 International Conference on Multimodal Interaction. Istanbul, Turkey: ACM.
   Google Scholar
• Olugbade, T. A., Bianchi-Berthouze, N., Marquardt, N., & Williams, A. (2015). Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain. Proceedings of the ACII 2015 International Conference on Affective Computing and Intelligent Interaction. Xi’an, China: IEEE.
   Google Scholar
• Paraskevopoulos, I. T., Tsekleves, E., Craig, C., Whyatt, C., & Cosmas, J. (2014). Design guidelines for developing customised serious games for Parkinson’s Disease rehabilitation using bespoke game sensors. Entertainment Computing. doi:10.1016/j.entcom.2014.10.006
   Google Scholar
• Pauletto, S., & Hunt, A. (2006). The sonication of EMG data. Proceedings of the ICAD 2006 International Conference on Auditory Display.
   Google Scholar
• Peretz, I., & Zatorre, R. J. (2005). Brain organization for music processing. Annual Review of Psychology, 56, 89–114. doi:10.1146/annurev.psych.56.091103.070225
   PubMed Web of Science ®Google Scholar
• Perri, M. A., & Halford, E. (2004). Pain and faulty breathing: A pilot study. Journal of Bodywork and Movement Therapies, 8, 297–306. doi:10.1016/S1360-8592(03)00085-8
   Google Scholar
• Rabenstein, R., & Trautmann, L. (2001). Digital sound synthesis by physical modelling. Proceedings of the ISPA 2001 International Symposium on Image and Signal Processing and Analysis.
   Google Scholar
• Rainville, J., Hartigan, C., Martinez, E., Limke, J., Jouve, C., & Finno, M. (2004). Exercise as a treatment for chronic low back pain. The Spine Journal, 4, 106–115. doi:10.1016/S1529-9430(03)00174-8
   PubMedGoogle Scholar
• Roby-Brami, A., Van Zandt-Escobar, A., Jarrassé, N., Robertson, J., Schnell, N., Boyer, E., … Bevilacqua, F. (2014). Toward the use of augmented auditory feedback for the rehabilitation of arm movements in stroke patients. Annals of Physical and Rehabilitation Medicine, 57, e4–e5. doi:10.1016/j.rehab.2014.03.015
   Google Scholar
• Rocchesso, D., Bresin, R., & Fernstrom, M. (2003). Sounding objects. IEEE Multimedia, 10, 42–52. doi:10.1109/MMUL.2003.1195160
   Web of Science ®Google Scholar
• Romera-Paredes, B., Aung, M. S. H., Pontil, M., Williams, A. C. D. C., Watson, P., & Bianchi-Berthouze, N. (2013). Transfer learning to account for idiosyncrasy in face and body expressions. Proceedings of the IEEE 2013 International Conference on Automatic Face and Gesture Recognition.
   Google Scholar
• Rosati, G., Rodà, A., Avanzini, F., & Masiero, S. (2013). On the role of auditory feedback in robotic-assisted movement training after stroke. Computational Intelligence and Neuroscience. Article ID 586138.
   PubMed Web of Science ®Google Scholar
• Rosser, B., & Eccleston, C. (2011b). The current state of healthcare apps for pain: A review of the functionality and validity of commercially available pain-related smartphone applications. The Journal of Pain, 12, P9. doi:10.1016/j.jpain.2011.02.034
   Google Scholar
• Rosser, B. A., & Eccleston, C. (2011a). Smartphone applications for pain management. Journal of Telemedicine and Telecare, 17, 308–312. doi:10.1258/jtt.2011.101102
   PubMed Web of Science ®Google Scholar
• Rosser, B. A., McCullagh, P., Davies, R., Mountain, G. A., McCracken, L., & Eccleston, C. (2011). Technology-mediated therapy for chronic pain management: The challenges of adapting behavior change interventions for delivery with pervasive communication technology. Telemedicine and E-Health, 17, 211–216. doi:10.1089/tmj.2010.0136
   PubMed Web of Science ®Google Scholar
• Schaffert, N., Mattes, K., & Effenberg, A. O. (2010). Listen to the boat motion: Acoustic information for elite rowers. Proceedings of ISON 2010 Interactive Sonification workshop. Stockholm, Sweden: KTH.
   Google Scholar
• Schmitz, G., Kroeger, D., & Effenberg, A. O. (2014). A mobile sonification system for stroke rehabilitation. Proceedings of the ICAD 2014 International Conference on Auditory Display.
   Google Scholar
• Schönauer, C., Pintaric, T., Kaufmann, H., Jansen-Kosterink, S., & Vollenbroek-Hutten, M. (2011). Chronic pain rehabilitation with a serious game using multimodal input. Proceedings of the ICVR 2011 International Conference on Virtual Rehabilitation. Zurich, Switzerland: IEEE.
   Google Scholar
• Singh, A., Klapper, A., Jia, J., Fidalgo, A., Tajadura-Jimenez, A., Kanakam, N., … Williams, A. (2014). Motivating people with chronic pain to do physical activity: Opportunities for technology design. Proceedings of the CHI 2013 Conference on Human Factors in Computer Systems. New York, NY: ACM.
   Google Scholar
• Smith, B. H., & Torrance, N. (2011). Management of chronic pain in primary care. Current Opinion in Supportive and Palliative Care, 5, 137–142. doi:10.1097/SPC.0b013e328345a3ec
   PubMedGoogle Scholar
• Sullivan, M. J. L. (2008). Toward a biopsychomotor conceptualization of pain: Implications for research and intervention. The Clinical Journal of Pain, 24, 281–290. doi:10.1097/AJP.0b013e318164bb15
   PubMed Web of Science ®Google Scholar
• Tajadura-Jiménez, A., Furfaro, E., Bianchi-Berthouze, N., & Bevilacqua, F. (2015). Sonification of virtual and real surface tapping: Evaluation of behavior changes, surface perception and emotional indices. MultiMedia, 22, 48–57.
   Google Scholar
• Tajadura-Jiménez, A., Larsson, P., Väljamäe, A., Västfjäll, D., & Kleiner, M. (2010). When room size matters: Acoustic influences on emotional responses to sounds. Emotion, 10, 416–422. doi:10.1037/a0018423
   PubMed Web of Science ®Google Scholar
• Tajadura-Jiménez, A., Väljamäe, A., Toshima, I., Kimura, T., Tsakiris, M., & Kitagawa, N. (2012). Action sounds recalibrate perceived tactile distance. Current Biology, 22, R516–R517. doi:10.1016/j.cub.2012.04.028
   PubMed Web of Science ®Google Scholar
• Timmers, R., Marolt, M., Camurri, A., & Volpe, G. (2006). Listeners’ emotional engagement with performances of a Scriabin étude: An explorative case study. Psychology of Music, 34, 481–510. doi:10.1177/0305735606067165
   Google Scholar
• Tracey, I., & Bushnell, M. C. (2009). How neuroimaging studies have challenged us to rethink: Is chronic pain a disease? The Journal of Pain, 10, 1113–1120. doi:10.1016/j.jpain.2009.09.001
   PubMed Web of Science ®Google Scholar
• Turchet, L., Pugliese, R., & Takala, T. (2013). Physically based sound synthesis and control of jumping sounds on an elastic trampoline. Proceedings of the ISon 2013 Interactive Sonification Workshop.
   Google Scholar
• Turk, D. C. (2002). Clinical effectiveness and cost-effectiveness of treatments for patients with chronic pain. The Clinical Journal of Pain, 18, 355–365. doi:10.1097/00002508-200211000-00003
   PubMed Web of Science ®Google Scholar
• Turk, D. C., & Rudy, T. E. (1987). IASP taxonomy of chronic pain syndromes: Preliminary assessment of reliability. Pain, 30, 177–189. doi:10.1016/0304-3959(87)91073-6
   PubMed Web of Science ®Google Scholar
• Varni, G., Mancini, M., Volpe, G., & Camurri, A. (2011). A system for mobile active music listening based on social interaction and embodiment. Mobile Networks and Applications, 16, 375–384. doi:10.1007/s11036-010-0256-4
   Web of Science ®Google Scholar
• Vidyarthi, J., & Riecke, B. E. (2013). Mediated meditation: Cultivating mindfulness with sonic cradle. Proceedings of the CHI 2013 Conference on Human Factors in Computer Systems. New York, NY: ACM.
   Google Scholar
• Vlaeyen, J. W. S., De Jong, J., Geilen, M., Heuts, P. H. T. G., & Van Breukelen, G. (2002). Management of chronic pediatric diseases with interactive health games: Theory and research findings. The Clinical Journal of Pain, 18, 251. doi:10.1097/00002508-200207000-00006
   PubMed Web of Science ®Google Scholar
• Vogt, K., Pirrò, D., Kobenz, I., Holdrich, R., & Eckel, G. (2009). PhysioSonic - Evaluated movement sonification as auditory feedback in physiotherapy. Proceedings of the CMMR/ICAD.
   Google Scholar
• Volpe, G., & Camurri, A. (2011). A system for embodied social active listening to sound and music content. ACM Journal on Computing and Cultural Heritage, 4, 1–23. doi:10.1145/2001416
   Google Scholar
• Wallis, R. I., Ingalls, T., Rikakis, T., Olsen, L., Chen, Y., Xu, W., & Sundaram, H. (2007). Real-time sonification movement for an immersive stroke rehabilitation environment. Proceedings of the ICAD 2007 International Conference on Auditory Display.
   Google Scholar
• Wellner, M., Schaufelberger, A., & Riener, R. (2007). A study on sound feedback in a virtual environment for gait rehabilitation. Proceedings of the IWVR 2007 International Workshop on Virtual Rehabilitation.
   Google Scholar
• Williams, A., Eccleston, C., & Morley, S. (2012). Psychological therapies for the management of chronic pain (excluding headache) in adults. Cochrane Database of Systematic Reviews.
   Google Scholar
• Wolpert, D. M., & Ghahramani, Z. (2000). Computational principles of movement neuroscience. Nature Neuroscience, 3, 1212–1217. doi:10.1038/81497
   PubMed Web of Science ®Google Scholar