Lobbying for the ear, listening with the whole body: the (anti-)visual culture of sonification

Abstract

Sonification, the transformation of data into sound, is often argued to challenge the “visual culture” of science. Based on an analysis of rhetorical discourses as well as bodily practices within the sonification community, I show that the relationship between sonification and visual culture is in fact more complex and ambivalent: in publications and interviews, sonification researchers blame visual practices for the marginalisation of sound, but also look up to visualisation as a role model. I argue that this delicate balancing act can be regarded as an expression of what historian of science Thomas Kuhn has referred to as the “essential tension” of science between convention and iconoclasm; here: between questioning a scientific status quo (equated with a “visual bias”) and conforming to it. Turning towards the sonic and embodied skills involved in doing sonification work, I show that the different sensory modalities, which seem so neatly bounded in discourses about sonification, are intimately intertwined in practice.

Keywords:

• Sonification
• sound and vision
• visual culture
• embodied skills
• essential tension

In Douglas Adams’ (1988) science-fiction story Dirk Gently’s Holistic Detective Agency, the protagonist Richard describes the advanced visualisation functionalities of his latest project, a spreadsheet program named Anthem:

If you want dancing girls jumping out of the pie chart in order to distract attention from the figures the pie chart actually represents, then the program will do that as well. Or you can turn your figures into, for instance, a flock of seagulls, and the formation they fly in and the way in which the wings of each gull beat will be determined by the performance of each division of your company. (Adams 1988, 23)

Anthem’s most unusual feature, however, is not its capacity for visualising numbers and data, but for “sonifying” them by turning them into sounds of varying pitches and lengths. The corporate world is mesmerised by this musical representation, and Richard becomes fascinated by the idea of applying the same technique to make scientific phenomena, instead of business accounts, audible (24) – much to the chagrin of his boss, who does not find any commercial benefit in “turning the erosion patterns of the Himalayas into a flute quintet” (49).

Although written as a piece of fiction almost three decades ago, Douglas Adams’ story of Anthem in many ways quite accurately describes the phenomenon of “sonification”, the transformation of data into sound. In principle, any kind of data can be made audible – existing examples include the sonification of trends in the stock market, of patterns in environmental data describing microbial populations, of events inside particle accelerators, and of cosmic X-rays.¹ The different sonifications not only represent a multitude of kinds of data, but are used for a variety of purposes, ranging from analytical tools for scientific specialists to musical pieces, from assistive technologies for visually impaired users to means of scientific popularisation aiming at a broad public. In terms of sound, too, sonification is a diverse field: while some sonifications lull the listeners with the sound of orchestral music, others might prompt them to dance to a techno beat or tap their foot to a jazz tune, while yet others stay clear of any musical connotation and instead rely on abstract clicks, blips and sine-tones.²

Much like the fictional Anthem software has taken the corporate world by storm, sonification has received an increasing amount of public and media attention in recent years (Sterne and Akiyama 2012; Supper 2014). At the same time, sonification is still in many ways a contested practice, not widely accepted as a scientific technique (Supper 2012). This, too, contains echoes of Anthem, whose musical representations are described as a silly gimmick.

It is a third possible parallel between sonification and the fictional Anthem software that I want to explore in this article, however: the relationship between visual and auditory representations of data. In the case of Anthem, the situation is quite clear: the auditory representation is an optional add-on to the software’s visual display. As I show in this article, the way in which contemporary sonification researchers conceptualise the relationship between their practice and visual forms of representation is rather more complex and ambivalent. My analysis is based on a qualitative empirical study of the academic community dedicated to sonification conducted between 2008 and 2012. Specifically, it draws upon 36 semi-structured interviews with sonification researchers, participant observation in different sonification contexts – ranging from conferences and workshops to concerts and public talks, as well as a brief stint of learning how to sonify in a laboratory active in sonification research – and a literature study of published primary literature, including conference proceedings, journal articles and dissertations.

In order to analyse the complex relationship between sonification and visualisation, I begin by exploring how the sonification community explicitly challenges the “visual culture of science”. Subsequently, I show that the relationship between sonification and visualisation is not just defined by opposition: just as often as visualisation is framed as a competitor of sonification, it is cited as its role model. In the following section, I discuss how this apparent contradiction is in fact an expression of a balancing act regarding an essential tension between convention and subversion. Finally, I leave behind the lofty discourse and rhetoric of sonification and turn to an analysis of practices and skills, showing that auditory, visual and haptic skills are intimately intertwined in sonification research.

Lobbying for the ear: challenging the visual culture of science

Practitioners of sonification often introduce their goals as alternatives to what they perceive as a dominant “visual culture” of science. As an alternative or complement to existing techniques of visual representation, they seek to make more prominent use of the sense of hearing in scientific practice. Usually with the help of digital sound synthesis software, phenomena that do not make a sound on their own – such as brainwaves or stellar oscillations – are translated into sound, in the hope that listening to them will reveal patterns in the data that may not easily jump out by staring at a visual rendering of the same phenomenon. However, this approach of listening to scientific data is often dismissed as overly subjective and regarded with scepticism by the scientific community at large. Many sonification researchers have stories, for instance, about how potential collaborators or peer reviewers have rejected the idea of sonification out of hand as being unscientific – the proper way of doing science, in the eyes of these critics, relies upon the sense of sight only.³

When asked about the reasons for this scepticism, sonification researchers often blame the marginalised role of the ear in a scientific world dominated by the eye. While scientists generally receive at least some training in the creation and interpretation of images, they certainly do not get trained to listen scientifically.⁴ As one of my interviewees succinctly put it: “the ear has somehow had a bad lobby for a very long time”.⁵ Many sonification researchers try to counter this perceived “cultural bias towards visualization” (Hermann 2002, 2) by taking up this role as a lobby for the ear. With the help of an emancipatory rhetoric promising to free the ear from its marginalised status within science, they seek to “prove that the ear is able to challenge the epistemological power of the eye” (Dombois 2001, 229).

By referring to these efforts to justify the usefulness and legitimacy of sonification as an act of “lobbying for the ear”, I want to draw attention to the centrality of the ear in this discourse. Arguments in favour of sonification often begin not with the specific advantages of an application, but rather with a general point about what the ear is capable of. For instance, “the ear’s effectiveness in tracking multiple streams of information” (Ballora 2000, 29) is a major selling point for sonification. The ear is said to be “strong in the recognition of time, dynamics of a continuum and tensions between remembrance and expectation” (Dombois 2001, 227), and “able to process many different information streams and (after minor training) it is excellent in detecting even subtle acoustic patterns” (Hermann, Hansen, and Ritter 2001, 1). Another point brought up in its favour is the “impressive sensitivity of the human hearing mechanism and human ability to discern subtle variance in sounds and sound patterns” (Ben-Tal et al. 2002, 1).

The rhetorical reliance upon the perceptual abilities of the ear is perhaps unsurprising, especially considering that auditory perception research is an important component of the interdisciplinary mix that constitutes the sonification research community (Kramer et al. 1997). Nonetheless, it is worth noting that it disregards a large strand of recent scholarship within visual studies, science and technology studies, and the history of science about what makes scientific images so compelling. According to these scholars, the strength of visual representations is “also the result of craft on the part of the makers, rather than an essential quality of these images” (de Rijcke and Beaulieu 2007, 734); it is not a matter of “just seeing” (Vertesi 2014). Indeed, as Michael Lynch has argued, visualization should not be “reduced to perceptual and cognitive mechanisms. Graphs, chart recordings, and digital images are public communicative devices. They have a material organization of their own, and are collectively discussed, reproduced and worked over” (Lynch 2006, 28). Consequently, to understand the appeal of scientific visualisations, it is not enough to consider what the eye is capable of seeing – rather, we have to understand the social, cultural and historical contexts in which these images are constructed and circulated, and the ways in which they function as material and symbolic intermediaries (37). A widely cited example of this line of thinking is Bruno Latour’s (1990) work on images as “inscriptions”, which take their strength not from their visual nature as such, but rather from their ability to break down complex realities into flat, two-dimensional representations that can be dominated and manipulated, compared and circulated.

In order to understand visual culture, therefore, it is not enough to study images as images and consider how the eye can perceive information – rather, we have to understand them also in terms of their material qualities (Poster 2002), the craft involved in producing, storing and sharing them (Favero 2014) and, intriguingly, their sensory qualities, including their tactile and aural characteristics (Edwards 2005; Mitchell 2005).

In pitting the ear against the eye, the sonification researchers tend to overlook that visualisations are not just optical phenomena perceived by the eye, and neither are sonifications just auditory phenomena perceived by the ear. The anti-visual rhetoric of sonification discussed in this section paints a rather simplistic picture of the relationship between sound and vision, between sonification and visualisation. In an article about historical predecessors of sonification, media historian Axel Volmar has recently criticised discourses about the status of sound in science which reduce this question to a comparison between different senses; instead, he argues that “the outputs of scientific representations and transformations as well as the disciplinary and sociopolitical contexts of the research-process should be taken into account” (Volmar 2013, 82).

Although I agree with Volmar about the limitations of a discourse that orchestrates sound and vision as competitors, my goal in this article is not to criticise sonification researchers for their shortcomings in assimilating insights by scholars in visual culture or the history of science. Rather, I want to show that a closer study of how sonification researchers relate to visualisation practices reveals a more complex picture than the somewhat one-sided anti-visual rhetoric described so far. It would indeed be too simplistic to merely pit the ear and eye as competing modalities fighting for their dominance in scientific culture – but it would be similarly reductive to accuse the sonification community of such an oversimplified stance. In the remainder of this article, I therefore want to trace the relationship between sonification and visualisation in the world of sonification researchers in all of its complexity, and to show how this rhetoric tension arises out of the specific context of sonification research.

Following the eye: visualisation as a role model for sonification

As I have shown in the previous section, the rhetoric of the sonification community often pits eyes and ears as competing sensory modalities, and aims to challenge the epistemological dominance of vision. However, this has only been part of the story; in fact, the way in which sonification researchers refer to visualisation is more complex and ambiguous. For all the emancipatory rhetoric of liberating the auditory from its marginalised existence in the shadows of visual culture, the sonification community also looks up to visualisation as a point of reference and role model.

Although sonification researchers often blame the unquestioned reliance upon visual techniques for the dismissal of sonification, they also regularly try to turn it to their advantage. When faced with objections that translating data into sound is too subjective to count as a credible scientific approach, they often point out that visualisation, too, requires subjective and aesthetic decision; if these are accepted for visual displays, they therefore also have to be accepted for sonifications. When questioned about decisions that were made in the making of a sonification after conference talks, for instance, presenters frequently point out that subjective decisions are also inevitable and legitimate components of visualisation practices. The more accepted status of visualisation is thus employed as an ally to argue that sonification can be scientific despite the necessity of making subjective decisions.

Positive references to visualisation are not limited to such ad hoc justifications of subjective elements in specific sonifications, but feature prominently in arguments about the future direction of the field as a whole. Intriguingly, visualisation is enrolled as a role model for very different futures of sonification research. Already in the first comprehensive report about the state of the art of sonification research, prepared in 1997 for the National Science Foundation, the research agenda was modelled after existing work in the field of scientific visualisation:

Scientific visualization was in a similar situation a decade ago with the rapid development of computer graphics technology (…) Lessons learned from the development of the field of visualization over the past 20 years may be very instructive for the development of the field of sonification. (Kramer et al. 1997)

The sonification community is involved in ongoing debates about the direction for future research, such as whether sonification should seek to tie in with artistic practices, or strive to become established as a scientific method (Supper 2012). The sonification researcher Thomas Hermann is a key proponent of the latter position. In his efforts to redefine sonification as a scientific method and to demarcate it from artistic work (“data-inspired music”), references and parallels to visualisation play an important role:

Think of scientific visualization vs. art: what is the difference between a painting and a modern visualization? Both are certainly organized colors on a surface, both may have aesthetic qualities, yet they operate on a completely different level: the painting is viewed for different layers of interpretation than the visualization. The visualization is expected to have a precise connection to the underlying data, else it would be useless for the process of interpreting the data. In viewing the painting, however, the focus is set more on whether the observer is being touched by it or what interpretation the painter wants to inspire. (Hermann 2008, 5)

Intriguingly, however, parallels to visualisation are not only drawn in order to argue for a more scientific orientation of sonification research. In a recent article for the journal AI & Society, sonification researcher Stephen Barrass refers to visualisation with the opposite goal. Rather than aiming at a more scientific conception of sonification, Barrass wants to reconfigure sonification “from an instrument of scientific enquiry to a popular mass medium for a broad audience” and, in doing so, bring about an “aesthetic turn” in sonification research. A similar turn, argues Barrass, has already occurred in visualisation:

The public release of datasets on the internet by government agencies, environmental scientists, political groups and many other organizations has fostered a social practice of data visualization. The audiences have expectations of production values commensurate with their daily experience of professional visual media. At the same time, access to this data has allowed visual designers and artists to apply their skills to what was previously a field dominated by scientists and engineers. (Barrass 2012, 177)

These examples show that visualisation is utilised as a rhetorical resource and role model for arguing in favour of a particular take on sonification research – seemingly regardless of the actual content and direction of this take. For as much as the sonification community engages in an emancipatory rhetoric that seeks to challenge the dominance of vision by pushing the use of sound in scientific practice, it also relies upon references to visualisation in order to carve out a new field of research and establish its legitimacy.

Essential tension: conformity or iconoclasm

In the previous two sections, I have shown that the sonification community frequently engages in an anti-visual discourse, but does not seem to break free from the influence of visual practices. The rhetoric about visualisation within the sonification community is rich, complex and seemingly contradictory. So how can we make sense of these apparent contradictions?

The ambivalence about how sonification should position itself in relation to visualisation is partially related to practical concerns about the usefulness of specific sonification applications. For instance, depending on whether a sonification is designed primarily for visually impaired or for sighted users, it may face different practical requirements. Spreadsheet documents, for instance, are easy to navigate with the eye – but only for people with good eyesight. While visually impaired users can listen to the contents of individual cells with the help of a screen-reader, finding out which cells even contain relevant data is often a slow and arduous process, which can be potentially alleviated by a sonification that tells the user where to direct their attention.⁶ To a visually impaired user, a sonification might therefore be at its best if it conveys information that sighted people usually perceive visually; yet a sighted person is more likely to value a sonification if it solves problems that are not served by visual techniques.⁷ Additionally, research about auditory perception has shown that visually impaired and sighted users may have rather different mental models that affect how they interpret sonifications (Walker and Lane 2001).

Nonetheless, the question of whether sonification should seek to replace or complement existing visual techniques is not a purely practical design question; it also hints at a fundamental ambiguity in how sonification publicly presents itself as a field of research. The idea that auditory displays can be useful tools for people who cannot see is a relatively uncontroversial and intuitive one; it is probably for this reason that it is often cited as a promising area of sonification research, even when describing sonifications that are not specifically designed towards the needs of visually impaired users.⁸ In instances like this, sonification comes close to acting as what Mara Mills (2010) calls an “assistive pretext”, where a disability is used to justify the development of a particular line of research, but the actual needs of disabled users are soon disregarded to make way for more profitable applications or, in the case of sonification, more radical ones. For as much as it can be helpful to “sell” the idea of sonification research with references to visually impaired users, many practitioners of sonification become irritated when their work is reduced to the status of an “assistive technology”, as this suggests that sonification is useful only for people who cannot see.⁹

The question of how to position sonification research in relation to technologies for visually impaired users is linked to broader concerns about the relationship between sound and vision – which itself is linked to the tricky question of how radically sonification should break with the entrenched conventions governing what is to be considered an acceptable way of displaying scientific data.

I want to suggest that we can make sense of this question as an expression of what Kuhn (1977) regarded as the “essential tension” of science. According to Kuhn, science is permanently torn between tradition and innovation, between convention and iconoclasm. Caught in this tension, “the successful scientist must simultaneously display the characteristics of the traditionalist and the iconoclast” (Kuhn 1977, 227). Unlike Kuhn, however, my argument here is not concerned with whether such an “essential tension” is indeed an invariable element of all scientific practice; rather, I want to argue that such a tension between convention and iconoclasm is characteristic specifically for the dynamics of sonification research.

For such a non-essentialistic understanding of the “essential tension”, Cyrus Mody’s recent (2014) adoption of the term is especially helpful. Mody argues that it is not “self-evident whether a particular argument or piece of evidence should be seen as conventional or iconoclastic” (Mody 2014, 223). Whether a new piece of research should count as building upon existing traditions or breaking with them, therefore, is not an inherent quality of the research as such; rather, different interpretations – emphasising either more traditional or more innovative elements – can be given of the same research. The way in which scientists present their work, either by drawing up existing conventions or inventing their own idioms, “guides audiences towards the preferred interpretation” (223). Nanotechnologists, for instance, often present their work as novel and innovative, but at other times emphasise the conventional nature of their work to reassure audiences of its credibility and avoid “needless regulation, public backlash, or disappointed funders” (224). This tension is also reflected in choices of how to visually represent their work, as very creative, unconventional displays are used side-by-side with images that are steeped in more traditional scientific idioms.

Similarly, sonification researchers quite flexibly adjust how they present their research; sometimes as building upon existing, traditional scientific research, and other times emphasising how sonification breaks with these traditions. Sonification is thus sometimes positioned in conformity with scientific conventions such as objectivity and reliability; at other times, it is emphasised to be more cutting-edge and avant-garde than boring old disciplinary science. As I have discussed elsewhere (Supper 2012), these negotiations of the boundaries of sonification research also involve a flexible positioning and careful balancing of sonification in relation to science and art.

What may at a first glance appeared to be an inherent contradiction in the discourse of sonification – the simultaneous attack on the visual dominance of science and reliance upon references to visualisation in order to carve out a field of research for sonification – is in fact the result of such a careful balancing act between tradition and innovation, convention and subversion. Visualisation acts as a stand-in for the traditional nature of science as such, which sonification both attempts to comply with and break away from. As a result, the label of the “iconoclast” for a scientist who tries to break away from conventional scientific research (Kuhn 1977) takes on a striking new meaning; for the sonification researchers trying to break away from the conventions of traditional science, these conventions are indeed tantamount to the reliance on visual forms. Breaking away from traditional science is therefore synonymous with destroying the visual bias of science; it is, quite literally, iconoclasm: the destruction of images.

However deeply embedded as this tension between sound and vision is in the rhetorical discourses of sonification, it does not seem to play much of a role in the craft-work and practices of developing and working with sonifications. In the remainder of this article, I want to shift from the rhetorical discourse of sonification to the practice of “doing sonification”. In this practice, I will show there is no discernible tension between sound and vision; rather, the different sensory modalities are inseparable.

Listening with the whole body: the sonic skills and body-work of sonification

During a workshop in which an interdisciplinary group of scientists worked together on sonifying data from computational physics, one sonification researcher expressed an idea through singing: he suggested that a proton–proton collision could be represented by a “diung diung diung diung diung diung diung diung” sound, sung with rising and descending pitch. Additional sound effects used during his verbal explanations (such as illustrating the idea of a slow frequency vibrator by pronouncing these words extremely slowly and with some added vibrato) further helped to communicate his design ideas.

Among sonification practitioners, this activity is jokingly referred to as “data karaoke”. I first became aware of this practice, in which a human being mimics the sound of a sonification with their own voice, on the first day of a crash course in sonification design, when I struggled with my own self-consciousness about using my voice in this way. Despite the ubiquity of advanced sound synthesis software, I learned that the technologically rather simple technique of “data karaoke” was essential for sonification design, as it serves as a quick, simple and cheap way of expressing and communicating an expectation of what a sonification might sound like.¹⁰ Indeed, some sonifications are deliberately designed to facilitate mimicking, such as EEG sonifications that are based on a model of human vowel sounds to make it easier for humans to “communicate structures and patterns in EEG data by mimicking the patterns with their own voice” (Hermann et al. 2006, 6). The data are thus made to mimic a human being mimicking a data-set.

Data karaoke is an intriguing example of the craft skills and embodied practices involved in sonification work. A helpful starting point for thinking about these skills is the notion of sonic skills, developed by Karin Bijsterveld (2009). Sonic skills are not limited only to listening skills, nor to linguistic repertoire of sound professionals, which Thomas Porcello has highlighted in his related notion of professional audition (Porcello 2004); they also include “the skills needed to employ the tools for listening” (Pinch and Bijsterveld 2012). The term “sonic” as opposed to “auditory” emphasises that sonic skills are not limited to issues of perception; although listening skills play an important role, they are embedded in a broader set of bodily and technical skills related to making, recording, storing and retrieving sound. For sonification, these sonic skills include mastering and shifting between several different modes of listening (Supper and Bijsterveld 2015), as well as tinkering with audio hardware and computer software – and, of course, data karaoke.

If a strength of the concept of sonic skills is its emphasis on how auditory perceptions are embedded in broader material practices, then data karaoke is an instructive example because its shows that they are also embedded in other forms of sensory perceptions, apart from purely auditory ones. The example at the beginning of this section, in which a workshop participant sang his ideas for a new sonification design, illustrates this notion nicely. At the same time that the sonification researcher was expressing these sounds, he also engaged in various other bodily practices, such as slowly stretching out his arm, gesticulating along, walking around the room, and turning in circles to emphasise the physical motion that is being translated into sound.

In doing so, the sonification researcher engages in practices that are simultaneously auditory (through the sounds produced by his voice and heard by the other participants) and visual (through the gestures and movements as they are seen by the other team members). Indeed, in other instances, data karaoke is also accompanied by drawings and graphs, thus further strengthening the relation between auditory and visual elements.

However, data karaoke is not just an auditory and visual phenomenon, but also a haptic one. After all, the voice, gestures and movements of the sonification researcher are not just seen by the other team members; he also feels them in his own body. In that sense, data karaoke can be thought of as an example of what Natasha Myers (2008) has termed “body work”. Myers has used this term to refer to the work that protein crystallographers do when they build and touch models, or even use their own bodies as models, to get a more intimate understanding of protein structures. In a similar manner, sonification researchers who use data karaoke may strengthen their relationship to the data by giving voice to them and feeling them in their own body. This is further supported by the findings of Anna Harris and Melissa van Drie (Harris and Van Drie 2015), who show the importance of mimicry in the teaching of learning of bodily sounds in a medical context. The sonic skills of sonification are thus not a purely auditory, nor even a purely sonic, phenomenon; they involve auditory, visual and haptic skills and, indeed, the whole body of the sonification researchers, as well as various instruments and technologies that are employed in the process. This is quite striking, considering how much of the discourse about the scientific legitimacy of sonification revolves purely around one sensory organ: the ear. It also shows that the “essential tension” that the sonification community tackles in its rhetorical discourses aiming at the establishment of sonification as a legitimate technique, in fact does not cause much tension in practice.

Conclusion

In this article, I have scrutinised the relationship between sonification and visualisation, both in the rhetorical discourses of the sonification community through which they seek to establish the legitimacy of their approach of translating data into sound, and the embodied practices involved in doing sonification work. As I have shown, the practitioners of sonification frequently challenge the epistemological dominance of the sense of vision and instead speak up in favour of the sense of hearing. In this sense, they call into question the visual culture of science. At the same time, they do not break free from this visual culture entirely – in order to establish sonification as its own field of research, they look up to the more established field of data visualisation as a role model.

This apparent contradiction, I have argued, is not a flaw or an inconsistency in the rhetorical discourse of the sonification community. Rather, it is an expression of how the community engages in a careful balancing act to deal with what Thomas Kuhn (1977) has identified as an “essential tension” of science: a tension between tradition and innovation, between convention and iconoclasm. This iconoclasm – a term which Kuhn uses metaphorically to refer to the irreverent attitudes towards past scientific achievements – takes on an almost literal meaning in the case of sonification: to the sonification community, traditional scientific conventions are strongly identified with visual ways of doing things, and breaking free from these traditions equals calling into question the status of vision. The complex and ambiguous relationship between sonification and visualisation is thus an expression of the community’s delicate balancing act, as it wants to revolutionise scientific practice – and in particular, scientific representations – but it still seeks scientific acceptance according to traditional criteria.

In his work on the essential tension in nanotechnology research, Cyrus Mody has recently pointed out that the essential tension doesn’t always “lead to much tension or conflict” (Mody 2014, 244); different representational strategies – some of them more innovative and idiosyncratic, others more traditional and standardised – can co-exist quite peacefully, and indeed support each other. In shifting the focus, in the final section of this article, from the rhetorical discourse to the embodied practices involved in sonification research, it has become clear that this also holds true for sonification research: while the different sensory modalities are often treated as neatly bounded and competing entities in the rhetoric of sonification, in practice it turns out that haptic, visual and auditory skills are intimately intertwined. Sonification research is not just a matter of what the ear can hear; it is also a matter of what the eyes can see, what the body can feel, and how these bodily practices are embedded in a network of material practices.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes on contributor

Alexandra Supper is an assistant professor in the Department of Technology & Society Studies at Maastricht University, where she does research in science and technology studies, with a particular focus on sound and the senses. Among others, her research has been published in the Oxford Handbook of Sound Studies and in the journals Social Studies of Science, Science as Culture and Information & Culture.

Acknowledgements

I would like to thank Karin Bijsterveld, Joeri Bruyninckx, Bernike Pasveer, Jo Wachelder and the participants of the MUSTS Summer Harvest for their comments on earlier versions of this paper.

Notes

1. See and listen, on stock markets: http://www.sonification.com.au/markets/ , on microbial data: http://www.bio.anl.gov/microbialbebop.htm, on particle accelerators: http://lhcsound.hep.ucl.ac.uk/, and on cosmic x-rays: http://www.cfa.harvard.edu/sed/projects/star_songs/ (accessed 8 July 2014).

2. The diversity of sounds resulting from sonification is nicely illustrated by the “Listening to the Mind Listening” concert, performed at the Sydney Opera House in 2004. For this concert, 10 different musical renderings of the same data-set were performed. Despite the fact that all of the pieces used the same data-set – measurements of the brain activity of a person listening to a specific piece of music – and sonified it according to an explicit, reproducible and time-binding mapping between data and sound, the results sound vastly different, with a range of “jazz, orchestral, soundscape, glitch, techno and other genres” (Barrass, Whitelaw, and Bailes 2006, 13). Further information and audio files can be found on http://www.icad.org/websiteV2.0/Conferences/ICAD2004/concert.htm (accessed 9 July 2014).

3. Personal interviews with Alberto de Campo (16 October 2009), Florian Dombois (17 February 2008), Florian Grond (13 October 2009), Thomas Hermann (22 February 2008) and David Worrall (17 November 2010).

4. Personal interviews with Gerold Baier (2 November 2008), Florian Dombois (17 February 2008) and Florian Grond (26 June 2008).

5. Personal interview with Florian Dombois (17 February 2008).

6. Personal interview with Tony Stockman (7 November 2008).

7. For instance, during a sonification workshop in which an interdisciplinary group of researchers developed sonifications of computational physics data, one participant objected to the idea of “acoustically replicating” something which is very strong visually, as this is useful only for visually impaired users; for anyone else, “If you can see, it already works, so why in God’s name would you recreate it” (excerpt from fieldnotes).

8. This is true, for instance, of the dissertations of Thomas Hermann (2002) and Alberto de Campo (2009). In his concluding paragraph, de Campo reflects on the “unfortunate” circumstance that none of the presented sonifications “would be directly usable for visually impaired people”, even though accessibility for the visually impaired is “one of the strongest motivations” for sonification research (176).

9. For instance, during one sonification workshop I observed, several sonification researchers complained about the common misconception that sonification research deals with the design of assistive technologies, and its implications that it is only useful for blind users.

10. However, the use of data karaoke is not limited to the design stage. When presenting finished sonifications at conferences, sonification researchers frequently combine recorded sound files with their own vocal renderings of the same data-set (Supper 2015).