Segmenting and Parsing Instrumentalists' Gestures

Abstract

This article presents a segmentation model applied to musician movements, taking into account different time structures. In particular we report on ancillary gestures that are not directly linked to sound production, whilst still being entirely part of the global instrumental gesture. Precisely, we study movements of the clarinet captured with an optical 3D motion capture system, analysing ancillary movements assuming that they can be considered as a sequence of primitive actions regarded as base shapes. A stochastic model called the Segmental Hidden Markov Model is used. It allows for the representation of a continuous trajectory as a sequence of primitive temporal profiles taken from a given dictionary. We evaluate the model using two criteria: the Euclidean norm and the log-likelihood, then show that the size of the dictionary does not influence the fitting accuracy and propose a method for building a dictionary based on the log-likelihood criterion. Finally, we show that the sequence of primitive shapes can also be considered as a sequence of symbols enabling us to interpret the data as symbolic patterns and motifs. Based on this representation, we show that circular patterns occur in all players' performances. This symbolic step produces a different layer of interpretation, linked to a larger time scale, which might not be obvious from a direct signal representation.

Acknowledgements

We would like to thank Norbert Schnell for his fruitful advice as well as Delphine Bernardin and Erwin Schoonderwalt for creating the database. A work partially supported by NSERC, Canada and MDEIE, Quebec. Finally, we would like to thank the COST IC0601 Action on Sonic Interaction Design (SID) for their support in the short-term scientific mission at IDMIL, Montreal.

Notes

¹ Information is to be understood as the accuracy of the model to generate the input data.