Musical Organics: A Heterarchical Approach to Digital Organology

Abstract

Gaining a comprehensive understanding and overview of new musical technologies is fraught with difficulties. They are made of digital materials of such diverse origins and nature, that they do not fit comfortably into traditional organological classifications. This article traces the history of musical instrument classifications relevant to the understanding of new digital instruments, and proposes an alternative method to the centuries-old tree-structure of downwards divisions. The proposed musical organics is a multidimensional, heterarchical, and organic approach to the analysis and classification of both traditional and new musical instruments that suits the rhizomatic nature of their material design and technical origins. Outlines of a hypothetical organological informatics retrieval system are also presented.

Keywords:

• organology
• classification
• musical instruments
• NIME
• musical organics
• information retrieval

1. Introduction

Since the beginning of recorded history, we find technologies of music-making at the forefront of human technics; as artefacts, musical instruments can be seen as an individual culture’s ‘crowning achievements’ (Nettl, 2005, p. 382). The expertise required to build instruments such as lutes, flutes, organs or pianos, traditionally placed music technologies at the cusp of human ingenuity and technical knowledge. With the advent of electricity, the evolution of musical instruments has developed alongside technological progress and scientific knowledge. These changes have been connected to developments in musical culture, equally in terms of new performance technologies, aesthetics, music theory, social practices, and architectural spaces. A profound technological shift took place with the advent of electronic instruments in the late nineteenth century; gaining a stronger cultural resonance in the latter half of the twentieth century, equally in modern, contemporary, and popular musics. The history the analogue synthesizer’s innovation, in the 1960s, is a good example of the complex process of adoption required for new music technologies to become stabilised as instruments among other instruments (Pinch & Trocco, 2002). Furthermore, the commercialisation of digital music technologies in the 1970s, a new wave of energy and innovation transformed the field of musical instruments. Often referred to as digital musical instruments (DMIs), these new music technologies are part of an industry that is exceptionally dynamic and fast changing, and their development should be seen in conjunction with an interdisciplinary research field called ‘NIME’ (New Interfaces for Musical Expression—see www.nime.org). Here, new musical instruments are studied from the perspectives of music, performance, ergonomics, psychology, engineering, software design, digital signal processing, and more. New controllers, sensors, mapping techniques, feedback actuators, motors, machine learning, and digital signal processing techniques are continually applied in devices of emergent research, some of which reach the commercial marketplace through a convoluted process of innovation and marketing.

In the current rapidly developing era of computation, musical instruments are designed using the latest technologies, both in terms of software and hardware. The field is continually reticulating, branching, but also clustering around technologies that find resonance in musical practice. We observe how the evolution of a particular instrument tends to halt at a certain point; it becomes a node in which the technology concretisises (Simondon, 2016), and becomes a stable reference for composers, performers, educational institutions, as well as the media and the general public. New instruments are thus either pulled into tradition, repertoire, and educational establishments, or rejected, put on hold, forgotten. Instruments that become part of musical tradition develop slowly, and change is of minute degree, rarely of essence. An unarticulated contract is forged between composers, performers, and instrument makers, constituting their complex and multidimensional relationship. In order to understand these relationships and how the evolution of an instrument stabilises, we might apply actor–network theory (Latour, 2005), and examine how each of the roles is part of more integrated networks: institutional, educational, cultural, aesthetic, commercial, and technological. The composer–interpreter–instrument maker triangle is part of a complex hierarchy and power structure that, from the lens of musical instruments, involves educational institutions, guilds, societies, associations, publishers and labels, manufacturers, collections, concert halls, festivals, the media, and commercial retailers. New digital instruments destabilise these structures and form new economic constellations, vocations, and professions.

There is a clear demand for establishing organisational principles for these new digital instruments. Inventors want to learn from each other (McPherson et al., 2016; Paine, 2010), performers benefit from a stronger contextualisation of their musical practice, musicologists need a terminology to analyse and reference developments in the field, and composers wish to understand the instrumentation principles of these new technologies. Critical analytics of DMIs can be helpful to all of the above. The research field that has traditionally dealt with the study and classification of musical instruments is called organology. Organologists have presented a plethora of useful approaches to classifying and sorting musical instruments, equally for comprehensive musicological knowledge, and for the spatial considerations of outlining a book about instruments, or organising a museum’s instrument collection. The organisational principles are many and they differ amongst the world’s cultures, often focusing on the material substrata of instruments and their vibrational function. Originating in the classical Indian Nāṭyaśāstra system, the museum classification of Mahillon in 1880, and the system designed by Hornbostel and Sachs in 1914, we now typically operate with classificatory divisions defined as idiophones, membranophones, chordophones, and aerophones.

The above-mentioned Hornbostel–Sachs system (1914) is the most universally accepted classification scheme, and, albeit imperfect, it is widely used in musicological literature, as well as in museum collections. In 1940, Sachs introduced the electrophone category as a response to new musical materialities, such as oscillators, filters, pickups, and amplifiers (Sachs, 2006). The electrophone category has proved insufficient for today’s context, and a number of authors have engaged with the issue, as we will see later. However, the problem is extremely complex as the field of new electronic instruments has dramatically increased in size, activities, and technological solutions since the 1940s, both in the analogue and digital domains. Because of disperse origins, digital music technologies cannot easily be made a subset of the electrophone, nor would a category such as the digiphone really work—and we will explore that problem in this article.¹ The difficulty we are faced with is illustrated by the following question: what constitutes the organisational principles of digital instruments? We are surely not only interested in their physical materiality, as the plastic, rubber or metal they are made of do not produce the sound they emit.² Instead, we discover organisational principles relating to the type of sound-producing algorithms, performer gestures, perception-modalities of sensors, or mapping strategies, to name but a few. The frameworks we apply to understand instrumental qualities from these principles form a critical analytics of musical instruments, and the musical organics system described in the last section of this article supports such multi-perspective analytics.

The difficulty in attempting to continue the classificatory strategies of the nineteenth and early twentieth centuries for the new material reality of digital musical technologies is evident. For this reason, much of the work attempting to add digital instruments to existing organological classifications include the word ‘towards’ in the title, as the authors typically acknowledge that these are only the first steps towards a new or improved organology. But the journey often ends there. The intention is not to poke fun at what might appear to be half-finished jobs, but rather acknowledge the useful attempts and ingenious solutions proposed. These frameworks can serve as analytic tools even if the aim of comprehensiveness is abandoned. However, in this article I hope to admit defeat even before writing the word ‘towards’ (no matter how tempting), and an alternative strategy to tackle the problem is presented: that of musical organics.

This article briefly surveys key historical classifications in organology, before introducing more recent attempts to deal with comprehensive classifications of digital instruments. The notion of musical organics is then proposed as an approach benefiting the organology of new digital instruments. This is not a classification system designed with considerations of physical space or printed books: it is rather a philosophical concept, engaging with the problems of classifying DMIs, and proposing a dynamic architectural information-space applying modern media technologies, where classifications of musical instruments can be built on-the-fly using a flexible information retrieval system.

2. Classifying instruments

All musical cultures have ways of understanding their instruments that involve sorting them into meaningful categories. What could be considered a useful classification in one culture might be of little relevance in another, and we often find that extra-instrumental concerns, such as mythology, societal structure, cosmology, or religious function play part in defining the principles of categorisation. Kartomi (1990) has written a fine ethnomusicological account of organological classification schemes across distinct musical cultures. Kvifte (2005) continues this work and reassesses the problems of organology in the electronic age, emphasising the analysis of playing technique, but with a reference to the evolution of organological classification schemes. Most often these schemes focus on the material substance and morphology of the instruments, for example in the Sanskrit Nāṭyaśāstra, written between 200 BCE and 200 CE, where instruments are grouped into the categories of ‘stretched’ (e.g. the vina string instrument); ‘covered’ (e.g. drums); ‘hollow’ (e.g. flutes); and ‘solid’ (e.g. bells and cymbals). This is a coherent system—unlike the common division used in Western orchestral instrumentation into string, wind, brass and percussion sections (where string refers to the vibrational material, wind to the excitation force, brass to the material type and percussion to the human action)—and it is the inspiration for Mahillon’s classification introduced below, and thus various systems still in use today, for example the Hornbostel–Sachs.

Western organology also traces its origins back to Hellenic thought, where both Plato and Aristotle talk about animate instruments (organon psychon or the human voice) and inanimate instruments (organon apsychon or string and wind instruments). Both had a strong preference for the human vocal instrument (with Plato effectively banning inanimate instruments from his ideal city-state), and this is a view that persisted for centuries to come (Kartomi, 1990, p. 108). A more systematic classification, which prevailed throughout the medieval period, was presented by the Neoplatonic philosopher Porphyry, in the third century CE, but he is known for the explication of Aristotle’s Categories into a classificatory tree structure called Arbor Porphyriana (the Porphyrian Tree). In his work on organology, Porphyry divided musical instruments into three categories: wind, string and percussion. This system was often combined with cosmological and theological explanations of music, such as Boethius’s sixth century division into musica mundana, musica humana and musica instrumentalis (as the third category was later called), a classification later adopted by Athanasius Kircher, whose Musurgia Universalis from 1650 became a key musicological treatise, influencing composers such as Bach and Beethoven (Devlin, 2002).

In the Renaissance, the focus shifted from the cosmos back to human musicians and their instruments. Indeed, Kircher based his work on the writings of Gioseffo Zarlino, a sixteenth century composer, instrument maker, and musicologist, who designed an influential and comprehensive classificatory scheme built upon Porphyry’s system. Zarlino divided instruments into natural and artificial categories, depending upon whether they related to the movement of the heavenly bodies or not, but also into instromenti mobili (for variable pitches, such as violin or trombone) or instromenti stabili (for fixed pitches, such as harp or a flute). However, it was with the work of Michael Praetorius, in 1619, that we find the first modern systematic organological approach, in his work De Organographia, dedicated to musicians and instrument makers (Restle, 2008, p. 259). Praetorius’s book was beautifully illustrated, and in it we find instruments divided into wind, strings, and percussion. The book was very influential with composers and performers and its popularity was not superseded until Berlioz published his manual of instrumentation in 1855 (see Figure 1).

Fig. 1. A woodcut from Praetorius’ De Organographia, a key organological work published in 1619.

Between 1880 and 1892, Victor-Charles Mahillon, a curator at the Musée instrumental du Conservatoire Royale de Musique in Brussels, published a five-volume catalogue of musical instruments. The preface of the first volume was called Essai de classification méthodique de tous les instrumentes anciens et modernes (Mahillon, 1880), and here Mahillon developed a scheme of coherent organisational principles that would work for the museum’s large collection of instruments. The scheme was a tree-branch divisionary system, divided into the classes of autophones, mebranophones, chordophones, and aerophones. In addition to the period’s imperialism and the acquisition of cultural items from across the globe, at play was also nineteenth century scientism, exemplified by the work of Comte, Darwin, Marx, Freud, Grimm and von Humboldt, where methods of collecting, measuring, describing, analysing and classifying were vigorously applied in the study of cultural change and natural evolution, both central concepts of the nineteenth century scientific practices. The use of scientific instruments, as epistemic tools for thinking about the world, also affected the musical thinking of the time, and technologies such as chronometers, tuning forks, sirens and metronomes provided a new platform for the development of compositional approaches (Jackson, 2011).

Building on Mahillon’s system, the classification designed by Erich M. von Hornbostel and Curt Sachs in 1914 has been the most popular scheme for categorising musical instruments in the twentieth century. Redefining some of the taxonomical divisions, and also introducing logical numbering based on the Dewey Decimal System (known to most people as the system used for classifying books in library collections), Hornbostel and Sachs were under no illusion that their classificatory system would be coherent and logical at all times, stating that the ‘objects to be classified are alive and dynamic, indifferent to sharp demarcation and set form, while systems are static and depend upon sharply-drawn demarcations and categories’ (von Hornbostel & Sachs, 1961, p. 4). The system enables a tracing down to the unique features of individual instruments, through logical divisions at each hierarchical level. For example, the numerical denominator of 111.242.222 would refer to sets of hanging bells with internal strikers (1 = idiophone; 11 = struck idiophone; 111 = idiophones struck directly; 111.2 = percussion idiophones; 111.24 = percussion vessels; 111.242 = bells; 111.242.2 = sets of bells; 111.242.22 = sets of suspended bells; 111.242.222 = sets of clapper bells). A system with such nuanced cataloguing is clearly beneficial for museums, historians, anthropologists, and musicologists, but it might not reflect the reality of actual musical practice (see Figure 2).

Fig. 2. A common tree-like organological classification. Here, showing the top categories of the Hornbostel–Sachs Systematik from 1916.

Sachs’ fifth category, the aforementioned electrophones introduced in 1940, was divided into instruments with electronic action (51), electromechanical action (52), and electroacoustic action (53). This addition was sufficient plasterwork at the time, but with digital instruments, software and diverse mappable controllers, this fix has long crumbled under the weight of new innovations. In their article ‘Demystifying and Classifying Electronic Music Instruments’, Bakan, Bryant, Li, Martinelli, and Vaughn (1990) address the problems of Hornbostel and Sachs’ four-branched organology by proposing a considerable addition to the system. Not persuaded by Sach’s definition of the electrophone category, they suggest a rethinking where electric and amplified instruments are sent back to their acoustic siblings (where the electric guitar becomes a subcategory of the chordophone guitar, but with an added ‘E’ at the end, or: 321.322-E). In this system, the electrophone category is used exclusively for instruments that generate sounds electronically. For example, the famous Yamaha DX7 digital FM synthesizer gets the classification of 512.231 K-Ps/Ua-My-T-MIDI.³

Margaret Birley and Arnold Myers (Birley & Myers, 2015) have recently published a revision of the Hornbostel–Sachs classification, which has now been taken into use by the MIMO (Musical Instruments Museums Online) consortium. Considering that the Hornbostel–Sachs system is widely used internationally, this is a welcome and timely project, updating the system to be more inclusive of non-Western instruments, and in particular, expanding the electrophone category. The MIMO website is already an impressive resource for musical instruments (see www.mimo-international.com). However, from the perspective of someone embedded in the field of NIME research, the electrophone category is still very limited and needs further work. The weakness of the category relates to the unorthodox status of electronic instruments, which—in particular new DMIs or NIMEs—have a history of existing outside the inscribed musical tradition (for reasons too complex to be dealt with here), illustrated by the fact that they do not tend to be awarded much space or presence in museum collections of musical instruments.

Another recent attempt to attend to the problems of the Hornbostel–Sachs system can be found in a recent paper by Weisser and Quanten (2011), who provide two alterations to the system, the first introducing two versions of timbre modifiers as important organological concerns, the second adding a modular syntax to the electrophone category (represented by symbols such as +, *, and =). This addition to the electrophone class was by request of the MIMO team. The article criticises Bakan et al.’s (1990) sweeping generalisations, and proposes a more modular approach that represents the complexity of twenty-first century instruments. In the final section of the paper the authors come to the conclusion that perhaps downwards macrotaxonomies might not be ideal for the complexity we are currently faced with, considering also that any ‘classification system reflects its inventor’s ideas about the objects of his or her research as much as—if not more than—the way the inventor organizes the objects’. (Weisser & Quanten, 2011, p. 140). In the last section of the article, Quanten speculates whether we could devise a framework with more holistic approaches, moving away from tree-like classifications.

3. Trees grow upwards: On typologies and microtaxonomies

The organological classifications described in the previous section are all characterised by their downwards logical divisions. This organisational approach is comprehensive, with the primary function of addressing the spatial considerations of fitting a large amount of data or material objects into their respective locations. However, other classifications are not necessarily so concerned with this overall perspective, suggesting a bottom-up approach where the unique instrument is described in detail and seen in the context of its musical culture. In 1947, Hans Heinz Dräger developed a method of microtaxonomical organology—detailed inspection and description of instruments—added as ‘clusters of variables’ to the end nodes found in the Hornbostel–Sachs scheme. Also including the electrophone category, Dräger’s parameter clusters focused on the instrument’s: (a) appearance, (b) tone production, (c) mono- and polyphonic capacity, (d) musical flexibility, (e) tone duration, dynamic range and loudness, (f) range, melodic possibilities, (g) tone colour range, (h) tone colour, (i) the performer (Dräger, 1948). The system has been criticised for being too complex (Kunst, 1959, p. 61) and is not in wide use, although clusters as parameter spaces have been applied to some degree in the NIME literature, as we will see later (e.g. Birnbaum, Fiebrink, Malloch, & Wanderley, 2005; Magnusson, 2010; Spiegel, 1992).

In 1969, Elschek and Stockman published a paper on an upward typology of musical instruments (Elschek & Stockman, 1969). They suggest a distinction between a classification and a typology, where classification is seen as downwards, based on a single criterion for division, whereas typology is upwards, focusing on the whole instrument, and subsequently grouping it with other instruments of similar nature. For Elschek and Stockman organology is primarily an empirical study with a strong historical element, as opposed to the more distant logical systematics. Elschek also developed a graphical system of signs for describing the instrumental functionality of aerophones (Elschek, 1969). With a grounded focus on the characteristics of the instrument itself, Elschek’s approach was to create a language, both conceptual and symbolic, to describe and explain the unique functionality of the instruments. The idea of using graphical means to describe instruments is further explored by Mantle Hood, who, in 1971, came up with a system called organography. The organograms were inspired by choreographic Labanotation (Laban, 1975), and are a method of describing the key element of the instrument. The collection of organograms illustrate symbolically the lowest terminal entries of the Hornbostel–Sachs scheme and are a useful contribution to organological science, although they have been criticised for being difficult to learn and laborious to prepare (Kartomi, 1990, p. 186) (see Figure 3).

Fig. 3. Mantle Hood’s organography enabled a visual description of musical instruments in their functional details. Here, displaying a composite symbol for a pair of fictitious drums.

In the work of Heyde (1975), an emphasis is placed on the evolution and relationships between instruments, conceptualised via analysis he defined as a ‘natural system of classification’ inspired by genetics. This is a complex system, but involves classes of abstraction levels, for example, the ‘formal class’, where he distinguishes between technological and human elements in instruments using the terms of technomorph and anthropomorph. Here, prototypical instruments are only anthropomorphic, as in whistling or beat boxing; typical instruments are both technomorphic and anthropomorphic, such as the piano and the saxophone; and exotypic instruments have only technomorphic qualities, as in musical automata and some software. An interesting feature of Heyde’s system is the ‘system class’ wherein he describes the functional elements of instruments, dividing their ‘carrier elements’ into: initiator, intermediary, transformer, intermediate transformer, modulator, amplifier, resonator, and coupler (Kvifte, 2005, p. 46).

Recent studies in organology broaden the scope of the field, often emphasising the cultural context of musical instruments (Dawe, 2001; Qureshi, 2000); lived organology based on stories, historical meanings and relationship with the sacred (Hooshmandrad, 2004); or the ‘social life’ of musical instruments (Bates, 2012). Bates applies actor–network theory in analysing how instruments are more than simple objects applied in music making, but actually serve as complex actor-networks of meaning, history, and agency. A related approach is taken by Tresch and Dolan (2013) in their ‘Toward a New Organology: Instruments of Music and Science’, where they apply the same type of organological classification to both scientific and musical instruments—one based on ethics. Building their system on the later work of Foucault, they propose the following categories for instrumental ethics: (1) material disposition, or what kind of assemblage the instrument is; (2) mode of mediation, or how the instrument’s action is seen as autonomous or passive, modifying or transparent, hidden or visible; (3) map of mediations, or musical context, material conditions, protocols and institutions; (4) the instrument’s telos, or its ends. Here they ask: what is the meaning of the instrument? Why is it used? How does it impact on its context? Like Bates, Tresch and Dolan apply actor–network theory when describing how ethical concepts can be applied to inanimate objects.

Other systems include Kurt Reinhard’s structured approach to analysing the stylistic properties of instruments. Reinhart found the Hornbostel–Sachs system sufficient for classifying the morphology of musical instruments, but presented a system that added concerns of note production (whether mono- or polyphonic, whether the notes are continuous or die away, and whether the notes can be dynamically changed during play). Playability and context was important for Reinhard and he suggested that loudness and timbre should also be graded in the analysis of instruments. It is also relevant to mention here two projects developed in the early 1970s, which used computers for database entries, by Michael Ramey and William Malm, respectively. For both Ramey and Malm, the use of relational databases would extend the Hornbostel–Sachs system with diverse additional classifications, such as ornamentation, vibration sources, timbre, tuning data, cultural functions, and performer behaviour. Malm’s system entailed an open database for entering additional categories, and, interestingly, one of Malm’s primary database entries were hologram files of the instruments. This is relevant in today’s context of new virtual reality technologies, where 3D representations of instruments can be rendered—and easily recorded with holographic capturing software that works on regular cameras, for example on mobile phones. At the time, in the early 1970s, Ramey’s system was visionary in that it understood the information storage and retrieval potential of computer use. He claimed that the system could implement Dräger’s detailed classification, but also ‘extend the boundaries of such classification to encompass any criteria which the researcher might find useful’. (Ramey quoted in Kartomi, 1990, p. 186).

Finally, to add yet another novel non-hierarchical approach to these upwards classifications, the multidimensional scalogram analysis proposed by Lysloff and Matson (1985), forms an interesting alternative to traditional tree-based classifications. Lysloff and Matson reject the logic of hierarchical taxonomies and propose a structured system of variables with which instruments can be analysed, and subsequently visualised using graphical scalogram analysis, where instruments are represented as points in 3D parameter space that changes depending on which parameters are studied. This visual representation is sufficiently understandable on printed paper, but could become extremely interesting in a dynamic virtual reality system where the representation could morph between parameter configurations, with appropriate haptic interfaces for navigation.

4. New classifications for NIMEs

The dynamic research field known as New Interfaces for Musical Expression (NIME) is one of the key contexts for the research and development of new musical instruments and related technologies.⁴ The design of digital instruments is clearly very different from the production of acoustic instruments, due to the heightened epistemic dimension inscribed in the materialities of digital systems (Magnusson, 2009). These new materialities are often hardware and software technologies manufactured for purposes other than music, but appropriated for nimes. Without a tradition or the established composer–performer–instrument maker constellation, new DMIs do indeed follow the innovation and speed of high-tech culture, as opposed to the slower development of acoustic instruments, engendering a sense of novelty, curiosity, but also a certain alienation experienced by performers and audience alike.

Examples of the new materialities of digital instruments include: accelerometers that measure gestural movements in three dimensions; photocells that sense luminosity; infrared sensors detecting heat; face detection algorithms that can perceive facial expressions; 3D motion-sensing range cameras designed for computer games tracking whole-body movements; biosensors perceiving bodily states and brain interfaces tracking cognitive activity; and satellites tracking geographic locations. Haptic feedback in instruments is implemented with motors or solenoids. These materials appear as nodes with a clear agency as they are interwoven into complex techno-cultural structures.⁵ They are often borrowed from clearly defined utilisation contexts, such as the use of thumbsticks and pads in gamepad controllers; the accelerometer functionality of a Wii controller; the 3D camera of the Kinect; printed circuit board technologies; the medical context of electromyographic and brain interfaces; or the telecommunication and military context of TCP/IP and GPS. In all of these cases the knowledge in design and use of materials does not derive from the field of instrument design, but rather appropriated from other technological domains.

The above are examples of material instances applied in new musical instruments. Although obvious, it has to be noted that the computer itself is not a musical instrument but a meta-technology that has been adopted for such use. As such, much of the technical expertise of the digital luthier (Jordà, 2005) is not related to the acoustic properties of materials, but rather factors of human–machine ergonomics. A new type of knowledge is required to build, test, perform, analyse and understand these new instruments; a knowledge that does not derive from the age-old tradition of making acoustic instruments. In lieu of the traditional luthier’s oral transmission of knowledge—which includes maintaining a relationship with composers and performers, as well as an intuitive understanding of physical materials and the acoustics of architectural performance spaces—the digital luthier applies knowledge and techniques deriving from product design, human–computer interaction, computer games, web design, ergonomics, science fiction and even virtuosic sports such as skateboarding or karate. Furthermore, the programming languages used (say C, C++, Lisp, Java, JavaScript) are not made for musical purposes, although higher level musical environments have been created using these. Similarly, the communication protocols used, such as serial, TCP/IP, USB, HID, and more, are adopted for specific uses and built upon (MIDI, for example uses the serial protocol and OSC uses both TCP/IP and UDP/IP). Finally, we could mention software libraries, such as neural networks, machine learning, computer vision, motion tracking, dynamic mapping, GUI frameworks (sliders, buttons, knobs), etc., that are applied as blackboxes (Latour, 1994) into the design of music software and hardware instruments.

It is evident that the complexity of the new materials, the heterogeneous knowledge required, and the nature of the materials applied from extra-musical fields render the situation of the digital luthier quite unique. The fact that DMIs are not made of materials that resonate, and that learning to play them involves understanding how human gestures can be mapped to sound, points to the limitations of traditional organological classifications. The instruments, the design concerns, the performance contexts, and the sound generation, are simply too heterogeneous to fit the existing classification schemes. People have therefore sought to redefine what a musical instrument really is, how the concept of instrumentality is transformed with the digital, and how we might attempt to group and classify the new instruments. In his book ‘Instruments and the electronic age’ (2005), Tellef Kvifte, by emphasising analysis of playing technique over instruments, shifts the meaning of the term organology from instrumental organs to human organs, nodding towards the turn to the body in the social science, arts, and humanities research.⁶

In a paper asking what constitutes instrumentality in new digital musical devices, Cance, Genevois, and Dubois (2009) ask what classifies instrument as such? They apply cognitive linguistic research and discourse analysis to the field, analysing both English and French language use related to new instruments, and conclude that instruments are not defined as being hardware devices or software, but rather qualify as such as a consequence of how users interact with them. Sarah Hardjowirogo (2017) further explores the construction of instrumental identity, presenting seven potential criteria for an object to classified as a musical instrument: (1) Sound production, (2) intention/purpose, (3) Learnability/virtuosity, (4) Playability/control/immediacy/agency/interaction, (5) Expressivity/effort/corporeality, (6) ‘immaterial features’/Cultural Embeddedness, (7) Audience perception/liveness. She points to the cultural embeddedness of musical instruments, gesticulating towards a strand in ethnomusicological research ranging from Johnson’s (1995) focus on the form, function and meaning of instruments, arguing for a wider scope in instrumental research, to Bates (2012) application of actor–network theory to the discussion of the social life of musical instruments.

Other organological attempts have pointed to instruments as compositional devices, pointing to the machines we use to create our music with, for example in Douglas Kahn’s article ‘Track Organology’, where he states that our instruments are ‘currently pitched for composition, writing, and accumulation, not for performance, speech, and improvisation. They are laggard and methodological like a pen, not fast and first-draft like a tongue’ (Kahn, 1990, p. 74). Considering the age of Kahn’s article, this is an understandable notion, as most digital music technologies at the time were studio simulators, not instruments for live performance. This has changed, and in a more recent work Kim and Seifert (2017) argue for a classification that would include analysing the degree of interactivity and agency in new instruments, and they present a taxonomy of interactivity, depending on how autonomous the agency of the instrument is. Similarly, Zénouda (2012) presents an organology aimed at audio games, introducing a typology based on what kind of musical control the player has (and note the similarities in playing a game and playing an instrument, if indeed there should be a distinction).

The critical analytics of musical instruments are multi-perspective. For example, Levitin, McAdams, and Adams (2002) provide an organological analysis of control parameters in new musical instruments. Their approach focuses on the sound itself, on classifying the segments of a single musical note (or event), in order to provide a language that can help in the design of new controllers. Jordà (2005), in turn, introduces criteria such as playability, progression, and learnability. This relates to how diverse output instruments can have, and their identity. Is the instrument good for improvisation? Is it expressive? Can it be easily controlled and allow for virtuosic playing? In a NIME paper published the same year, Birnbaum et al. (2005) introduce a visual representation of a dimension space of digital musical devices. Their approach is phenomenological; it focuses primarily on the performer’s body and what the instrument can offer in the performance context. Their categories are: required expertise, musical control, feedback modalities, degrees of freedom, inter-actors, distribution in space, and role of sound. These are set up in a sevenfold visual dimension-space that enables us to compare instruments via simple graphs. Having worked on a theory of musical instruments as epistemic tools (Magnusson, 2009), I found Birnbaum et al.’s system to be lacking the epistemic dimensions that allow us to analyse the conceptual and music-theoretical content of musical instruments. My response was published in a NIME paper (Magnusson, 2010) that analyses how musical instruments are inscribed with knowledge, how theory is encapsulated in their design, and how users engage with this embedded theory. The categories in the epistemic dimension space were: expressive constraints, autonomy, music theory, explorability, required foreknowledge, improvisation, generality, and creative-simulation. The paper used the same visual representation as Birnbaum et al., primarily as it was meant to be complimentary to their work. In both Birnbaum et al. and in my own work, there are statements to the effect that the systems presented are neither final nor exhaustive.⁷ (see Figure 4).

Fig. 4. The dimension spaces of Birnbaum et al. (2005) on the left and Magnusson (2010) on the right.

In a 2006 NIME paper, Kvifte and Jensenius (2006) propose a terminology for describing instruments. They point out that the level of details differs whether applied to a listener, a performer, or a constructor (instrument builder). The features to be analysed include gestural, technical, and musical parameters, all depending on the level of specificity. The paper presents diverse useful models and mapping charts for analysing instruments, but it was not meant to be fully comprehensive, and the authors wrote another paper for the same conference focussing on gestural descriptions.

At the following year’s conference, Hurtado and I reported on a survey (Magnusson & Hurtado, 2007) probing people’s conceptions of acoustic, electric, and digital instruments as an organological study, but without the aim of constructing a classificatory scheme. We were primarily interested in the language used to describe the difference between the acoustic and the digital. Also at NIME, in 2010, Garth Paine presented a study towards a taxonomy of interfaces for electronic music, based upon a survey created as part of the TIEM (Taxonomy of Interfaces for Electronic Music performance) project (Paine, 2010). The questionnaire had the following sections: (1) general description, (2) design objectives, (3) physical design, (4) parameter space, (5) performance practice, (6) classification. A wide-reaching engagement with users, asking how they understand and classify their own instruments, seems like a promising method for gathering research data from the field, but problems arise when attempting to generate a coherent system from the disparate data. The author acknowledges that the approach has weaknesses in this regard, and that as such it is bound to be incomplete.

Finally, at NIME 2016 in Brisbane, key contributors to the field ran a workshop entitled NIMEhub, focussing on how to archive instrument designs (McPherson et al., 2016). The idea was to create a database that would benefit designers and instrument makers, as knowledge would be shared between practitioners (equally the successes and the mistakes), facilitating collaboration, archiving older designs for possible reuse, reducing duplication efforts, promoting easier fabrication, detailed documentation, and supporting the reproducibility of studies. This is clearly a beneficial project for the field, but the authors acknowledge the problem of classification when creating the database for such a repository. One such online repository is currently in development, MusHack (https://muzhack.com), but here the focus is on embedded hardware only, currently ignoring controllers and software instruments.

We have here expounded the problems of organological classifications of new DMIs and looked at diverse approaches in defining the properties through establishing schemes of critical analytics. These systems are useful on their own and can be applied as conceptual schemes within the musical organics system presented below. However, they cannot be considered exhaustive or comprehensive as the material nature, gestural sensing, mapping strategies, synthesis algorithms, aesthetics, and other aspects of digital instruments, are so multidimensional and complex that traditional tree-like classification schemes do not work as smoothly as when applied to acoustic instruments. This article suggests an alternative approach. But before presenting the theoretical underpinnings of the musical organics framework, we would benefit from exploring the history of classification related to musical instruments, and also engage with more recent notions of non-hierarchical and dynamic classification.

5. On trees, roots, and labyrinths

In her book on concepts and classifications of musical instruments, Kartomi (1990) discusses how human beings seek intellectual security by categorising the world and dividing it into manageable sections, and she demonstrates how classification systems always depend on the unique cultural and historical perspective of the classifier. Although classifying attempts can be found in all cultures, we observe how the tendency to classify intensifies in the European Renaissance, particularly with the birth of empirical science. Burke (2000) illustrates how tree structures became key metaphors in the sixteenth century, and applied to curricula, libraries, museums, and encyclopaedias (until they became alphabetised in the early seventeenth century). The nineteenth century saw further tendencies to measure and categorise, applying scientific instruments in all domains of knowledge. The desire was to simplify and understand, to define a common framework and language with which to communicate. Kartomi references anthropologist Lévi-Strauss who writes ‘Any classification is superior to chaos’ (Lévi-Strauss, 1966, p. 15), implying that it is not necessarily the scientific world view that is of importance but rather the practical and psychological function it effects. Friedrich Nietzsche, deeply sceptical of the concept of truth with a capital T, developed an epistemological approach he called ‘perspectivism’, which rejects objectivity in favour of a multitude of perspectives that form the conditions of truth and value judgements. Predating twentieth century phenomenological approaches, Nietzsche disapproved of how the sciences have rejected empirical experience for objective language and they ‘do this by means of the pale, cool, gray, conceptual nets which they threw over the colourful confusion of sense, the rabble of the senses’ (Nietzsche, 2004, Section 14).

Bowker and Star (2000) have argued that classification represents a key issue in the design of computer systems. In order to represent the state of the world in an information system, we perform a process of reduction that involves deciding which qualities of objects and their relations we deem relevant to our system (see also Hui, 2016). Such reduction is inevitably contingent and messy, with an abundance of ethical and political implications. Natural objects, artefacts, and human actions are displaced into a representational schema resulting in strata of complexities and interdependencies that condition our thinking about the world. For Bowker and Star a good classification system exhibits the following properties: (1) the classificatory principles are consistent and unique; (2) the categories are mutually exclusive; and (3) the system should be complete, and provide a total coverage of the world it describes (Bowker & Star, 2000, p. 10). These criteria are clearly not attainable when attempting to classify musical instruments: is the piano a string instrument or a percussion instrument? Is the wind harp a wind or string instrument? The wind chime an aeorophone or an idiophone? The world of musical instruments is fuzzy and its entropy increases with digital instruments. Instead of attempting to establish complete categorical systems of mutually exclusive and consistent categories, we might rather embrace the chaotic nature of the world, the fuzziness of concepts, and the fluid continuity that exists between what we often frame as discrete categorisations and descriptors.

Whence this desire to classify? we might ask, and, as often before, we trace the development of ideas back to the early Greek philosophers. For example, we find the Isagogue, written by the Neoplatonist Porphyry in the third century CE, a key work throughout the Middle Ages on the subject of classification and theory of definition. The Isagogue is a commentary on Aristotle’s work on ontology, the Categories from his Organon, where he introduces the distinction between classes and objects, and proposes ten classificatory predicaments of substance, namely: being, quantity, quality, relation, place, time, posture or attitude, having or possession, action, and affection.

In his ‘From the Tree to the Labyrinth’, Umberto Eco (2014) shows how Porphyry is the first to project Aristotle’s categories in terms of a tree, attributing that conception to Porphyry’s Neoplatonic worldview. For Eco, the problem with Porphyry’s interpretation is that he ‘delineates a single tree of substances, whereas Aristotle uses the method of division with a great deal of caution and, we might add, a great deal of skepticism’. (Eco, 2014, p. 6). Eco, eager to escape the constraints of the tree-metaphor, points out that Aristotle, in On the Parts of Animals, ‘gives the impression of being prepared to construct different trees depending on which problem he is dealing with, even when it comes to defining the same species’. (Eco, 2014, p. 6). Eco states that it is Porphyry’s exposition that establishes the tree-based dictionary-semantics into medieval thought via Boethius, who translated the Isagogue from Greek to Latin in the sixth century CE. Boethius happens to be the author of a five volume De institutione musica, a key medieval work on music theory, and whose categories of instruments we mentioned in an earlier section of this article (see Figure 5).

Fig. 5. Ramon Llull’s Arbor Scientiae, from his work Ars Magna, from 1295. There is a clear influence of Aristotle, Porphyry’s Isagogue, and Boethius’s work.

Eco traces medieval learning through analysis of two distinct organisational knowledge schemes: the dictionary (semantics) and the encyclopaedia (empirical knowledge). The tree metaphor is omnipresent, perhaps culminating in Ramon Lull’s thirteenth–fourteenth century epistemology, which includes, amongst others, the following tree representations: Arbor scientiae (tree of science), Arbor impreialis (tree of government), Arbor moralis (tree of morality), and Arbor celestialis (tree of astrology and astronomy). With Francis Bacon’s 1620 Novum Organum (Scientae) we begin to detect ideas about open repertories, or systems of knowledge that can dynamically evolve with the use of improved scientific instruments. Such encyclopaedic approach is also to be found in Bacon’s New Atlantis novel from 1627, which includes a famous speculative organology of future musical instruments:

We have also sound-houses, where we practise and demonstrate all sounds, and their generation. We have harmonies which you have not, of quarter-sounds, and lesser slides of sounds. Divers instruments of music likewise to you unknown, some sweeter than any you have, together with bells and rings that are dainty and sweet. We represent small sounds as great and deep; likewise great sounds extenuate and sharp; we make divers tremblings and warblings of sounds, which in their original are entire. We represent and imitate all articulate sounds and letters, and the voices and notes of beasts and birds. We have certain helps which set to the ear do further the hearing greatly. We have also divers strange and artificial echoes, reflecting the voice many times, and as it were tossing it: and some that give back the voice louder than it came, some shriller, and some deeper; yea, some rendering the voice differing in the letters or articulate sound from that they receive. We have also means to convey sounds in trunks and pipes, in strange lines and distances. (Bacon, 1850, p. 214)

Eco is not interested in musical instruments in his piece; the focus is on how, in modernity, tree metaphors are replaced with those of labyrinths or maps. He discusses Athanasius Kircher’s 54 fundamental categories which were supposed to be represented by iconograms, but does not mention Kircher’s work on combinatorics, based on Llull’s ontology, nor his extensive writing on musical instruments. Indeed, Kircher is omnipresent in organological literature, and his work Musurgia Universalis (1650) is one of the key works of seventeenth century musicology. Here, Kircher presents the Arca Musarithmica, a machine technology that makes generative music (as we would call it today) possible. In a later work, Phonurgia Nova (1673), Kircher presents musical automata, Aeolian harps, and transmission of sound over a distance. Similarly, in the work of Kircher’s pupil, Gaspar Schott, we find a related obsession with musical automata, for example in the work Mechanica hydraulico-pnevmatica from 1657. Twenty years later, in Magia universalis naturae et artis, Schott describes various sonic instruments that both extend the range of the voice and hearing, as well as outlining tuning systems based on mathematical calculations (not to mention the infamous cat piano). However, it is Schott’s taxonomy of 44 fundamental classes (learned from someone whose name he says he forgot), which Eco finds interesting, and we could list but a few of these:

1. Elements (fire, wind, smoke, ashes, Hell, Purgatory, centre of the earth). 2. Celestial entities (stars, thunderbolts, the rainbows). 3. Intellectual entities (God, Jesus, speech, opinion, suspicion, soul, stratagem, or ghost). 4. Secular statuses (emperor, barons, plebs). 5. Ecclesiastical states. 6. Artificers (painters, sailors). 7. Instruments. 8. Affections (love, justice, lust). 9. Religion .... 14. Brute animals. 15. Birds. 16. Fish and reptiles. 17. Parts of animals. 26. Metals and coins. 27. Various artifacts. 28. Stones. 29. Jewels. 30. Trees and fruit. 31. Public places. 32. Weights and measures. 33. Numerals. 39. Time. 40. Adjectives, and so on … (Eco, 2014, p. 40)

This is a fantastical collection of classes, perhaps humorous, and certainly becomes so in the poetic exegesis of Jorge Louis Borges, who, in his essay ‘The Analytical Language of John Wilkins’, describes a

certain Chinese encyclopaedia entitled Celestial Empire of benevolent knowledge wherein it is written that ‘animals are divided into: (a) belonging to the emperor, (b) embalmed, (c) tame, (d) sucking pigs, (e) sirens, (f) fabulous, (g) stray dogs, (h) included in the present classification, (i) frenzied, (j) innumerable, (k) drawn with a very fine camelhair brush, (l) et cetera, (m) having just broken the water pitcher, (n) that from a long way off look like flies.’ (Borges, 1964, p. 103)

Borges ingenuity and extensive scholarship is well known, but he is particularly well woven into the argument of this article with his comments on classification, as one of the key works of his seventeenth century protagonist, the historical Bishop Wilkins, is called ‘An Essay towards a Real Character and Philosophical Language’ (Wilkins, 1668). This essay describes the attempt at creating a universal language, built on a classification scheme based on a family of symbols akin to Hook’s organography (and note the word ‘towards’ appearing in the title).

In The Order of Things (French: Les mots et les choses), Michel Foucault describes his shattering laughter when he read the above passage in Borges, as the familiar landmarks of his thought are broken up, by the realisation that it is impossible to ‘think that’. (Foucault, 1989, p. xvi). Foucault explains the unease of his laughter: it was based on the suspicion that there is a more serious level of disorder than that of the encyclopaedia, ‘I mean the disorder in which fragments of a large number of possible orders glitter separately in the dimension without law or geometry, of the heteroclite’ (Foucault, 1989, p. xix). In his work, Foucault demonstrates what he defines as three distinct epistemes in recent European history, those of the Renaissance, Classical and modern periods. It is in the Classical episteme that we find this obsession with ordering, with identity and difference, lists and tables, classes, categorisations and taxonomies. It favours the ‘table, a tabula, that enables thought to operate upon the entities of our world, to put them in order, to divide them into classes, to group them according to names that designate their similarities and their differences—the table upon which, since the beginning of time, language has intersected space’. (Foucault, 1989, p. xix)

Foucault points to the disturbing nature of heterotopias, which many of Borges’ worlds could be described as, because they break down the linguistic order, they undermine language, and dissolve our myths. But the heterotopia, those ‘possible orders [that] glitter’ are precisely what contemporary classificatory solutions are moving towards, clearly influenced by the role of network theory in both the physical and social sciences (Castells, 1996), and perhaps particularly by the non-hierarchical structure of the brain as a function of its neural net structure. Already in 1945, computer scientist Warren McCulloch wrote a piece on the heterarchy of the brain’s topology (McCulloch, 1945), inspiring further work in cybernetics, network theory, and distributed intelligent systems. In the practical example of working cybernetic feedback systems, where outputs become input parameters, we clearly see how the tree structure breaks (see Figure 6).

Fig. 6. A change of metaphors: in place of a tree, with branches extending out from a common trunk, we now find a rhizomatic root, lacking a centre.

We are discerning a move from tree-based hierarchies to root-based heterarchies—from symbolic to cybernetic control. Eco applies the metaphors of labyrinths, which will replace the dictionary and encyclopaedic models of traditional classification schemes, naming three specifically: the unicursal labyrinth (with only one possible path); the Irrweg labyrinth, with diverse alternative choices; and the network, where each point can be connected with another. (Eco, 2014, p. 52). For Eco, the network is a solution to the ‘vertigo’ one is faced with when realising that knowledge can never be systematically organised (as opposed to Foucault’s Renaissance and Classical epistemes), and we conclude with a labyrinth which is not ‘ordered by clear binary disjunctions’ (Eco, 2014, p. 36). Here, Eco pulls in Deleuze and Guattari’s notion of the rhizome, defined thusly by the pair:

unlike trees or their roots, the rhizome connects any point to any other point, and its traits are not necessarily linked to traits of the same nature; it brings into play very different regimes of signs, and even nonsign states. [...] It is composed not of units but of dimensions, or rather directions in motion. It has neither beginning nor end, but always a middle (milieu) from which it grows and which it overspills. [...] Unlike the tree, the rhizome is not the object of reproduction: neither external reproduction as image-tree nor internal reproduction as tree-structure. The rhizome is an antigenealogy. It is a short-term memory, or antimemory. The rhizome operates by variation, expansion, conquest, capture, offshoots. [...] In contrast to centered (even polycentric) systems with hierarchical modes of communication and preestablished paths, the rhizome is an acentered, nonhierarchical, nonsignifying system without a General and without an organizing memory or central automaton, defined solely by a circulation of states. (Deleuze & Guattari, 1987, p. 21)

How would a rhizomatic organological framework of instrument analysis work? How should it be encoded? Who would encode it? Using which classifiers and systems? A heterarchical database of musical instruments with ‘multiple entryways’ and ‘in constant modification’ would need to be digital, stored in modern database models, and one that is called into form by the user’s request. Like the rhizome, musical organics ‘pertains to a map that must be produced, constructed, a map that is always detachable, connectable, reversible, modifiable, and has multiple entryways and exits and its own flight’. (Deleuze & Guattari, 1987, p. 21) The next section will investigate the possibilities of such a system.

6. Towards musical organics

It has emerged that tree-like classifications are not fully coherent and functional for traditional instruments, and it is clear that in the analysis of digital instruments they break completely. A new approach is required that supports probes (searches or queries) into repositories of digital instruments from a multiplicity of perspectives: materials (e.g. plastic, metal, glass, fibre, cloth); sensors (e.g. ultrasound, CMOS, bend, potentiometers); sound (e.g. physical models, FM, additive, concatenative, granular, sampling); mapping (e.g. one-to-one, one-to-many, many-to-one, convergent, learned, evolutionary, stochastic); gestures (e.g. wave, hit, stroke, pluck, shake, strike, bow, blow); reuse of proprioceptive skills (such as the trained playing of keyboard, strings, wind and percussion); manufacturer (e.g. of sensors, chips, motors), and many more, including cultural context, musical style, and other areas that have been, or indeed will be, called for as extensions to the existing organological classifications. This system would build on earlier descriptive organologies, but support interpretive organologies that ask ‘why and how’ questions, offer explanations, and put the queries into historical and musicological contexts (Heyde, 2001) (See figures 7 and 8).

Fig. 7. The halldorophone. An example of how twenty-first century instruments contain the elements of the acoustic, the electronic, and the digital in one and the same device.

Fig. 8. The Karlax controller. A new instrument with 55 individual parameter controls. New instruments are now going beyond their acoustic counterparts, reusing skills (finger control of keys), but applying other gestural inputs, such as an accelerometer for detecting movement, rotation sensor for twist, etc.

To classify is first to decide what we deem as relevant to our current interests and then ‘cast the conceptual net’ of Nietzsche. This is the area of ontology, described by Aristotle in his Metaphysics as the discipline that studies being as being, and the attributes that necessarily belong to being (Book IV, chapter 1). Ontology has a long history, but it is primarily a philosophical study of entities, their functions and relations. Computer scientists, in their attempt to transcribe and represent the physical world and digital objects (Hui, 2016), have come up with their own ontologies of representations that are of key importance in software engineering. For Eco, the computer science approach to ontology has been disappointingly tree-like, but defined by a prominent computer scientist as ‘a specification of a representational vocabulary for a shared domain of discourse—definitions of classes, relations, functions, and other objects’ (Gruber quoted in Eco, 2014, p. 60). What Eco finds useful in the computer science approach to ontology is that here the intention is not to be complete, but simply to cover the domain it is designed to represent. ‘In this sense, an ontology, however clumsy and ingenuous it may be, is the local representation of a portion of encyclopaedic knowledge relevant for the purposes of a given universe of discourse’. (Eco, 2014, p. 61).

The approach proposed here under the name of musical organics is not a new classification system, but a heterarchical method of analysing, archiving and representing instruments. This is a methodology of looking: of re-searching, investigating, probing; of comparing and recontextualising; of explaining transitions and transductions in the evolution and design of musical instruments. Musical organics is a rhizomatic system (the ‘organics’ connotation is indeed appropriate),⁸ that is to be implemented primarily as an information retrieval search system with an open API (application programming interface), clear protocols, and small set of open standards. This means that anyone can design a front-end interface for it, whether they are common database representations, GPU accelerated graphical libraries in JavaScript, or new virtual reality presentation technologies. The API would enable a plug-in structure, so that people could write ‘probes’ (a query, a search, a Nietzschean perspective) into the database. For example, none of the existing classifications contain an ontological category or field for the number of players required to play an instrument. Here, we might find that only one person can play the jaw harp, minimum two are required for the txalaparta, the piano can be played by many performers (e.g. Rachmaninoff’s piece Romance for 6 hands), and a digital system might have multiple performer interfaces for the same instrument. The number-of-players field might be interesting to some researchers yet of no interest to others, and this requires the musical organics system to be fluid and flexible in design, respecting Deleuze and Guattari’s suggestion that such a system ‘is open and connectable in all of its dimensions; it is detachable, reversible, susceptible to constant modification’ (Deleuze & Guattari, 1987, p. 10).

What is produced by a probe into the musical organics system would be a presentation of objects, relations, qualities, quantities, metaphors, imaginaries, all serving the unique query that is asked. These presentations are not built by hand, but pulled out of the online data mines for representation in diverse visualisation clients. With today’s potential for information retrieval, machine learning and new database technologies, we can analyse, compare, connect and synthesise data in larger spatial domains and at faster speeds than ever conceivable before, often significantly outperforming human experts. We are effectively able to create a hyper-dimensional dynamic organism in the form of an interconnected growing repository that will not only contain descriptions of the instruments’ properties, but also pictures, sounds, 3D models, videos of the instruments in performance, and more. This allows for very personal uses of the system. Having worked on a ‘search’ that produces a classification, the user would be able to save that constellation as a plugin to the system, which could be studied and improved upon by other users (see Figure 7).

Such a knowledge representation system would be based upon what DeLanda (2002), in his work on Deleuze, calls a ‘flat ontology’, defining it in the following terms: ‘…while an ontology based on relations between general types and particular instances is hierarchical, each level representing a different ontological category (organism, species, genera), an approach in terms of interacting parts and emergent wholes leads to a flat ontology, one made exclusively of unique, singular individuals, differing in spatio-temporal scale but not in ontological status’. (DeLanda, 2002, p. 47). This type of ontology would be problematic in traditional classification schemes, but less so in systems using databases that can be probed by search queries resulting in dynamic, machine-generated constellations of presentation. In order to describe these digital instruments, their objectivity and their semiotic relations to other digital objects, we might resolve to what Harman and Bogost call ‘ontography’, which is a ‘general inscriptive strategy… that uncovers the repletedness of units and their interobjectivity’ or ‘an aesthetic set theory, in which a particular configuration is celebrated merely on the basis of its existence’ (Bogost, 2012, p. 38). Both Bogost and Harman are inspired by what they call ‘Latour litanies’, or a ‘group of items loosely joined not by logic or power or use but by the gentle knot of the comma’, constructed to create the flat ontology that rejects hierarchies of objects. For Bogost, the practice of ontography involves describing the processes of accounting for objects and their relations. Ontographic writing is cataloguing, pointing to ‘the couplings and chasms’ between things (Bogost, 2012, p. 50). Such ontographies can be machine generated and classified, rendered into interpretative frameworks for human researchers. Indeed, the machines can prepare the data, but to interpret and understand it, we still need human organologists with their hermeneutic skills (see Figure 8).

A musical organics system would be a three-tiered system that incorporates traditional specialist classifications, user contributions (or ‘folksonomy’), but importantly also going beyond trees and metadata to actual machine analysis of content, where the system can ‘excavate’ properties, such as textual descriptions, music information retrieval timbral similarities of the instruments, or image and video materials. This could be represented in many different presentational schemes, for example, using 2 or 3D visualisation, or sonification. What is at stake is what Wolfgang Ernst defines as an informatised organisation of knowledge, one which ‘generates diagrams, which, incidentally, is also Deleuze’s term for the Foucauldean new archive’. (Ernst, 2015, p. 10). What Ernst is pointing to here is that new machine information retrieval technologies offer a study of large data structures that are not based on human tagging of metadata or classification:

What is being digitally ‘excavated’ by the computer is a number of information patterns which human perception perceives as ‘text’, ‘sound’ or ‘images’. Contrary to traditional semantic research hermeneutics, an active, audio-visual, coded archive will no longer list text, sound and image sequences according to their authors, subjects, and metadata only. Instead, algorithmically driven digital data networks will allow verbo-audio-visual sequences to be systematized according to genuinely signal-parametric notions (mediatic rather than narrative topoi), revealing new insights into their informative qualities and operative aesthetics. (Ernst, 2015, p. 10)

The primary reason for suggesting that in a musical organics system there is a clear separation between the data stored, how it is parsed, and how it is (re)presented; is that the data does not change, but our methods of probing into the database will benefit from new information retrieval techniques, and new systems of data representation, for example, with new augmented or virtual reality technologies including haptics. This representation of data through graphical form is something Johanna Drucker has written extensively about in her work on graphesis (Drucker, 2014), which is a study of the visual production of knowledge. The projects Drucker and colleagues have been developing at SpecLab (Speculative Computing Laboratory) at UCLA have been practical investigations into the digital humanities and in particular projects of information representation (Drucker, 2009). In ‘Performative Materiality’ Drucker discusses interpretative interfaces as dynamic systems that can pull data into views in order to support acts of interpretation, that are generative and iterative, capable of producing new knowledge, as opposed to returning selected results from a pre-existing data-set. Drucker advises that we

shift from the univocal to polyvocal, introduce point of view systems within the interface so that all views are from the position of an observer, not assumed to be independent, autonomous. Create fragmented and correlated points of view that resist self-evident reification. Create environments that are constellationary, so that diagrammatic relations can be used to re-order familiar conventions through acts of generative, performative engagement. (Drucker, 2013).

The SpecLab projects are in many ways similar in spirit to the work of another experimental laboratory in the humanities and digital studies, the Pompidou’s Institute for Research and Innovation (IRI—see www.iri.centrepompidou.fr), lead by Bernard Stiegler and Vincent Puig. In Puig (2014), some of the IRI projects are described, most of them involving a practical element (a truly practice-based philosophical approach), for example exploring the bottom-up creation of metadata to understand complex data structures, or what Puig calls folksonomy (as opposed to the top-down taxonomy). Stiegler has engaged with the techno-logical perspective embedded in cybernetics and sees technology itself as an epiphylogenetic condition of the human—an essential part of human nature (Stiegler, 1998). For Stiegler, the tendency to categorise and create classes constitutes a rationality that underpins human language, and continuing Derrida’s grammotology, he calls the process of transducing or transcribing objects or events from the analogue domain into the discrete domain grammatisation. Interestingly, in his recent work Stiegler also applies the term organology, but with the prefix ‘general’,⁹ aimed at signifying how technologies become extended organs, instruments for performance and thought, as well as social activities:

the thinking of grammatization calls for a general organology, that is, a theory of the articulation of bodily organs (brain, hand, eyes, touch, tongue, genital organs, viscera, neuro-vegetative system, etc.), artificial organs (tools, instruments and technical supports of grammatization) and social organs (human groupings … social systems in general). (Stiegler, 2010, p. 34)

If the musical organics classification approach is dynamic, open and flexible, it could indeed engage with the three levels of Stiegler’s general organology, where organologists would incorporate the study of bodily organs in music making, in particular learning, proprioception, kinaesthetic, collaboration, skills, virtuosity. They would clearly also study artificial organs, our prosthetic musical organs, the technologies through which we express music, the instruments. This is what traditional organology has focussed on, but this new organology would include broader technological contexts such as phonographic, notational, and ergonomic technologies. Finally, musical organics would include social organs: the modes through which we collaborate, communicate, share and enjoy music—the way we musick (Small, 1998) in the broad sense.

7. Conclusion

This article has presented musical organics as a methodological approach for studying and classifying instruments, including DMIs (which are the primary cause of the current classificatory problems). It has articulated the problems with taxonomic, top-down, tree-like classification of musical instruments in favour of a dynamic system of rearrangeable data. Musical organics is not a defined classification system, an implementation, or a technical specification of musical instrument data: it is a philosophical attempt to rethink classificatory strategies and provide a theoretical underpinning for actual practical work. Musical organics explores and frames the ecosystems of musical technics as a reticulated web—one that hybridises older organologies, continually borrowing, referencing, appropriating, and representing the techniques incorporated in human technological production and performative movement. This is a heterarchical organology, referencing the soil, plants, flowers, and clearly the rhizome itself.

Although a concrete design awaits future work, the musical organics approach would suggest a threefold structure:

(1)	The search domains (online search engines, aggregators, article repositories, collections, and databases) where data repositories can be registered as part of the search.
(2)	The open search-API of data models. The API will support semantic web standards and common interoperable standards and protocols for online repositories (e.g. OAIS—Open Archival Information System, and SWORD—Simple Web-service Offering Repository Deposit), applying information retrieval techniques and machine learning, feeding probes that are returned in commonly used data interchange formats (such as JSON or XML). Users would be able to contribute their own ontological categories, or meta-tagging in a folksonomy style, as well as plugins of new critical analytics schemes.
(3)	The representational engines that display (textually, visually, sonically) the search results. These change over time, applying new developments in aesthetics and media technologies, especially in terms of information display.

The aim of this article has not been to provide the technical specification of this system, but rather to discuss the philosophical underpinning of its design in a historical context. However, it is worth reasoning about why it is important to split the system up into these three parts, namely in order to make it future-proof, as the individual technological elements of the system (i.e. data, search, and representation) change over time and should be able to evolve independently. This can be achieved if the communication protocols between the three parts of the system are well defined.

Although musical organics might add a novel analytical instrument to the field of organology, it is not a new way of thinking. Indeed, we can trace the roots of heterarchical information organisation to Bush, 1945 article in The Atlantic Monthly called ‘As We May Think’, where an information machine, called Memex, is described that enables the user to access all the world’s literature, personal exchange, and multimedia data via one device. Personalising research, the user ‘builds a trail of his [sic] interests through the maze of materials available to him [sic]’. (Bush, 1945). The dynamic and networked method of relational thinking, incorporated in the notion of musical organics, is well supported by machine technologies and has a long history in computer science and AI.

This article has surveyed some historical organological classifications. The musical organics system proposed here is not intended to be comprehensive, final or exhaustive, but one that might evolve, fork out and re-branch, with modular packages that can be applied and added, akin to software plugins, synthesiser modules, or code libraries. A machine information retrieval supported search and classification mechanism is never going to do the job ‘on its own’; the need for hermeneutics, of human interpretation and understanding of context will always be required. The computational system is only there to do the hard labour. Furthermore, older classification systems are not to be abandoned, as many of the existing systems contain ingenious solutions and perspectives on what is relevant in instrument design, but none of them have proven sufficient on their own for the analysis of digital instruments. In order to establish a platform that can support diverse approaches to organological analysis, to support tradition, share knowledge, and build a repertoire, the proposed system needs to be dynamic, open source, distributed and collaborative. This article has attempted to describe a system of critical analytics for musical instruments, arguing that we are at the right time for information retrieval to become part of our extended organs of research, adding AI to our intellectual instruments for organological investigation.

Funding

This work was supported by the AHRC [grant number AH/N00194X/1].

Acknowledgements

Many thanks to my collaborators Sam Duffy and Alex Peverett for their reading and discussion about this work. I was fortunate to be able to discuss this work with Margaret Birley, Tellef Kvifte and Laurie Spiegel, all of whose input was important. Thanks also to Chris Kiefer, and the NIME community in general, for engaging with these ideas, as well as people at the DPASSH conference. The anonymous reviewers provided invaluable comments. This research is part of an AHRC funded research project called Sonic Writing (see www.sonicwriting.org).

Notes

¹ Further additions to the fivefold classifications have been common: Olsen (1980) introduced the corpophone in 1980 for instruments that are part of the human body; Mann’s (2007) physics-based organology adding physiphones (divided into subcategories, such as hydraulophones and plasmaphones); or indeed Deirdre Loughridge and Thomas Patteson’s fictophones (see https://imaginaryinstruments.org), referencing musical instruments that never sounded but exist as ideas (ideophone would suit that, if it wasn’t too similar to idiophone).

² Although from an HCI perspective—which would form part of the aforementioned digiphone studies—this might indeed be of great importance, as the choice between plastic, rubber or metal is likely to yield different musical results, for example in the difference in psychological and cultural response towards these distinct materials. This is an area explored by the Owl Project (www.owlproject.com), who have built digital instruments out of wood for nearly two decades.

³ Certain problems appear in this system with devices such as sequencers or filters that do not actually generate sounds but are essential to many electrophone productions. These are called ‘modifiers’.

⁴ NIME was originally the name of a workshop held at the CHI conference in 2001 on the design of new interfaces for musical expression, but it became an independent conference the year after. There are now university courses called NIME, club events with the title, and people often refer to new musical instruments as ‘nimes’. (See www.nime.org).

⁵ This type of analysis of technological interdependence can be found in various work in the philosophy of technology, for example expressed by Heidegger’s totality of equipment, Simondon’s technical system, Latour’s actor-networks, Deleuze and Guattari’s machinic assemblages, or Hui’s milieu of digital objects.

⁶ We note, of course, the etymology of the words ‘organ’ (Gr. órganon) for instrument, implement, or that with which one works (Gr. ergon), referencing both physical organs as well as instruments. With biosensor technologies we might also question the separation between biological organs and technological organs (cf. discussion of Stiegler’s organology later in this article), where in some cases musical instruments involves the synthesis of both, for example in a brain-sensor musical instrument, where brain signals are translated to musical sound, without corporeal gestures. .

⁷ Both of these papers should ideally have referenced a very interesting ‘Letter to the Editor’ (of the Computer Music Journal) by Laurie Spiegel, called ‘An Alternative to a Standard Taxonomy for Electronic and Computer Instruments’, from 1992. In this undeservedly little known text, Spiegel introduces a multidimensional parameter space, preceding Birnbaum et al. and mine.

⁸ The name also derives from Adolph Bernhard Marx’s section on organology in his Allgemeine Musiklehre, from 1839, where the third part, on musical instruments, is called ‘Organik’, but translated in the English version, published in 1853, as ‘Musical Organics’. This was long before the term ‘organology’ began to be used for the study of musical instruments.

⁹ Stiegler is well aware of the musicological meaning of the term organology, having served as the director of IRCAM (Institut de Recherche et Coordination Acoustique/Musique) in Paris, between 2002 and 2006.