Event-based Multitrack Alignment using a Probabilistic Framework

Abstract

This paper presents a Bayesian probabilistic framework for real-time alignment of a recording or score with a live performance using an event-based approach. Multitrack audio files are processed using existing onset detection and harmonic analysis algorithms to create a representation of a musical performance as a sequence of time-stamped events. We propose the use of distributions for the position and relative speed which are sequentially updated in real-time according to Bayes’ theorem. We develop the methodology for this approach by describing its application in the case of matching a single MIDI track and then extend this to the case of multitrack recordings. An evaluation is presented that contrasts ourmultitrack alignment method with state-of-the-art alignment techniques.

Keywords:

• performance
• audio analysis
• software

Acknowledgements

The authors would like to thank Sebastian Ewert for assisting in the evaluation study and to Simon Dixon for advice on the methodology.

Notes

1 http://www.mpi-inf.mpg.de/resources/MIR/SyncRWC60/

2 http://code.soundsoftware.ac.uk/projects/bayesian-multitrack-matching

3 Purchased from http://www.elasticmax.co.uk/

4 www.zplane.de

5 http://code.soundsoftware.ac.uk/projects/bayesian-multitrack-matching

Additional information

Funding

The authors would like to thank the Royal Academy of Engineering and the EPSRC for funding this research.