Improved Score-performance Matching Using Both Structural and Temporal Information from MIDI Recordings

Abstract

Automated score-performance matching is a complex problem due to the use of expressive timing by performers and the presence of notes that are unspecified in the score, such as performance errors and ornaments. Automated matchers typically use performance data extracted from MIDI recordings. For the most part, these algorithms use structural information, such as pitch and chronological succession, but do not use timing information. As a result, most matchers cannot deal satisfactorily with ornamented performances or performances that exhibit extreme variations in tempo. The matcher presented here relies both on structural and temporal information, allowing it to generate an accurate match even for heavily ornamented performances. Hand-made score-performance matches on a corpus of 80 MIDI recordings of organ performances of two pieces were used as ground truth data for a comparison with matcher results. The matcher achieved a nearly perfect accuracy rate. In addition, the matcher performed equally well or better than matchers previously described in the literature on a set of piano performances of two pieces by Chopin, thus demonstrating its versatility and robustness. We also propose a heuristic for the identification of ornaments and errors that is based on perceptual principles, and which could theoretically be amenable to empirical study. Finally, this matcher is designed to accommodate multi-channel MIDI recordings of performances from keyboard instruments with multiple manuals, such as organ or harpsichord. This feature makes it a potentially valuable tool for the investigation of ensemble performances of MIDI instruments.

Acknowledgements

This research was supported by fellowships from the Social Sciences and Humanities Research Council of Canada and from the Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT) to Bruno Gingras, as well as a grant from the Natural Sciences and Engineering Research Council and a Canada Research Chair awarded to Stephen McAdams.

Notes

¹‘Untied’ notes (Repp, 1996a) are treated as repetitions.

²Note that, according to this definition, the registral space of a monophonic melody is unbound.

³These threshold values were adjusted empirically so as to optimize the categorization of unmatched performance notes into errors and ornaments. Generally speaking, increasing these threshold values will increase the proportion of unmatched performance notes identified as errors relative to those identified as ornaments.

⁴Heijink, Windsor, et al. (2000) reported a total of eight misinterpreted notes out of 5642 notes for the structure matcher, for an error rate of 0.1%. The figure of 5642 notes is computed by adding the total number of notes in the performances to the total number of notes in the score. However, in analysing the data provided by Heijink et al. using our methodology, we arrive at a total of 10 misinterpreted notes for the structure matcher. All analyses are available upon request.