Editorial

This special issue of the IJPADM on Digital Annotation and the Understanding of Bodily Practices follows on from two previous special issues of this journal. Choreographic Documentation (9:1, 2013), edited by Scott deLahunta and Sarah Whatley, and Interdisciplinary approaches to documenting performance (10:1, 2014), edited by Toni Sant. The first takes its point of departure from a handful of initiatives involving prominent choreographers and their companies who were, amongst other things, experimenting with digital technology to document, analyse and disseminate dance. The issue includes several contributions from researchers who were centrally involved in those initiatives, for example, Bertha Bermúdez Pascual’s article about the research projects of the Amsterdam-based dance artists Emio Greco | PC (Pieter C. Scholten) and Christopher Thomson and Katherine Rothman on their work on the educational website for London-based choreographer Richard Alston. The 2013 Choreographic Documentation issue contained five ‘artist pages’ (referred to as ‘documents’ in this issue), set out against the rest of the contribution in both form and content. These artist pages collectively stand for the specificity of attempts to invent or explore ‘ways to appropriately express thinking and concepts discoverable in a particular artwork or in the principles of a certain creative process.’ (4) The editors of that issue closed their editorial by reflecting on the possibility that ‘As more artist-initiated publications are emerging, and existing ones are undertaking new research and future projects are being planned, we see the possibilities for a new discourse to emerge from dance practice that better addresses perceptions “awakened” by it (Louppe 2010: xiii).’ (5)

This perspective on exploring specificity of expression and referring to what makes an artistic approach or work unique and particular, also resists the kinds of standards associated with documentation and its organisation. Therefore, another question could be asked. How would individual efforts to achieve unique forms of documentation fit the purpose of becoming this new discourse emerging from practice? This question was critically raised in Choreographic Documentation by Johannes Birringer who was invited to reflect on the artist pages. He eloquently describes the contributions of Bertha Bermúdez, Myriam Gourfink, Carol Brown, Myriam van Imschoot, Jeanine Durning and Elizabeth Waterhouse to the issue, admiring their ‘exactitude’ and reflecting on the ‘physical intelligence both embodied and intellectual’ their unique approaches evidence. However, his conclusion also provides an occasion to pause and wonder ‘about the immense growth of data, discourses, documents and moving subjects/objects that are being generated, and from a curatorial or archival point of view, I think the widening of publishing creativity will reach limits.’ (Birringer 2013, 12)

This surfeit of data and curatorial theme was picked up by Toni Sant in his 2014 Editorial to the special issue Interdisciplinary approaches to documenting performance in which he questions the tendency on the part of performance scholars and practitioners to consider the significance of documenting and documentation to be secondary to artistic works. He references the often cited 1993 publication Unmarked by performance studies scholar Peggy Phelan in which she argued that ‘Performance's only life is in the present’ (Phelan 1993, 146) and observes that the ‘last 25 years of technological development have made it less likely to keep live performance ontologically pristine. For better or worse, performance arts now operate in a cultural economy that is hard to separate completely from media technology.’ (Sant 2014, 4) Sant’s perspective and the motivation for his special issue is that performance arts ‘are fast catching up with the emerging field of memory studies and the development of digital curation as a formal process of preservation and maintenance in archiving digital assets (texts, photographs, videos, illustrations, sound recordings, etc.) for long-term access.’ (Sant 2014, 3) He goes on to further elaborate his perspective on digital curation in his 2017 edited volume Documenting Performance in which he makes a distinction between generic documentation and ‘systematic documentation, ideally through standard methods of archiving.’ (Sant 2017, 2) For Sant, systematic documentation points toward the need for ‘data management,’ requiring specialist knowledge and skills to be obtained from fields such as ‘library and information science.’ (Sant 2017, 15)

We still sit at the crossroads of these two concerns: Sant’s perspective on digital curation, with its concern about data standards and the need to collaborate with information science, and the idea that the documentation of complex and particular artistic practices resists classification and categorisation. Here it would be possible to speculate that digital annotation of time-based recordings could function as the means of connecting the documentation of particular artistic practice(s) with computational data management and information science. Our questions in the call for this special issue reflected on this possibility, wondering how digital annotations made available to computation in various formalised forms, for example as metadata, might start to play an important role in the growing number of databases and digital archives of bodily practices.

Now we are prepared to go to publication with four articles and nine documents, we can reflect on how our issue on digital annotation establishes this connection. With some exceptions, the contributions do not feature explicit discussion of the kind of systemic documentation and standards Sant was referring to. Most of our contributors are theatre or dance practitioner/ educators and their reflections on the idea of standards is implicit in the discoveries they made in the process of annotating, using their chosen digital annotation tool and making decisions about what to annotate and how. They are also not concerned with matters of digital curation mentioned as a growing challenge by Birringer and as a potential solution by Sant. The questions this issue grapples with most predominantly are related to practical annotation methods used in the context of experiments in dance education, on-line transmission, performance analysis and critical exchanges. When the annotator is not coming from a discipline like linguistics or psychology with methodological training in the use of annotation tools such as ELAN¹, the practice of annotating becomes a matter of methodological² invention that is as analogue as it is digital. In other words, paying attention to, recognising and making decisions about particular features captured in the recording, as a manual practice where seeing is not in any sense automated. This manual practice of annotation draws out the ‘know-how’ of performance artists, bringing to the process expertise in knowing what is of significance to annotate. Across our contributions, this kind of knowledge, gained in the context of forms of bodily practice is brought to bear on the content of recordings, with varying degrees of self-participation, including annotating one’s own performances.

The contributions to this special issue also demonstrate the need for practitioners to be able to work differently with time-based media which is now ubiquitous in capturing the processes and outcomes of live performance. In support of Sant’s observation of the difficulty of keeping ‘live performance ontologically pristine’, theatre scholar Sarah Bay-Cheng writes that the performance ‘archive of the future will no doubt become increasingly dependent on (if not overwhelmingly devoted to) moving images on screens.’ (Bay-Cheng 2007, 39) The question she asks is then ‘how do we analyse mediated theatre as evidence of live performance?’ (Bay-Cheng 2007, 40) Digital annotation leads us to consider how we might better work with video sources and how annotation might contribute to establishing ways and setting certain standards related to the study, sharing and organising of ideas about live performance. Annotation can amplify or augment video recordings in a way that informs the task at hand as well as any future seeing and noticing. This lends a recursive dimension to the manual practice of annotation, as is reflected in several of our documents, the more the material is repeatedly observed and annotated (whether with text or drawings on the video), the more layers of understanding accrue. Annotation may encourage individuals to see a perspective that they might not have discovered as a result of their own viewing habits and preferences.

This layering of information may offer an expanding exegesis of a performance work, accumulating critical commentary or interpretation in ways similar to the historical tradition of marginalia in manuscripts and books. Commentary or interpretation goes to the question of what annotations might reveal and for whom, which points toward the possibility of publication. Two of our contributions discuss the publication of embodied and artistic research with a commitment to academic legitimacy. This means consideration of certain academic standards such as being able to be cited and challenged as well as submitting to peer review as part of the publication process. Included also in our issue is work being done to theorise digital annotation practice in the context of dance with its history of notation. The distinction is made between codification systems and annotation tools, underscoring the significance of the shift from notating dance performances to the documentation of processes.

We must also observe that the COVID-19 pandemic, well underway by the time we set up the call for this issue, has resulted in an accelerated adoption of digital tools in performing arts teaching and practice. The potential for digital annotation supporting remote collaboration and pedagogy is mentioned in some of our contributions, but it implicitly underpins nearly all of them in a way we think worth highlighting. Enabling virtual collaborative work processes is now the norm for a great deal of software, and this includes some of the tools used by our contributors. These tools support not only collaborative work on video documents, but also sharing and accessibility. This again ties into Sant’s observations about the ontologically pristine, as expressed by anthropologist Johannes Fabian who writes about documents that can be shared and accessed online:

Consigning documents to a virtual archive makes them more real, not in any ontological sense but in terms of their ‘practicality,’ that is, as regards their potential to mediate between the events that were noted or recorded and our efforts to represent the knowledge gained as adequately as possible. (Fabian 2008, 5)

There is another thread running into and through our issue, and that is a history of relatively small-scale annotation software development emerging from the initiatives involving prominent choreographers already mentioned. Returning briefly to the Choreographic Documentation issue that focussed on these initiatives, we can refer to an article by Carla Fernandes and Stephan Jürgens titled ‘Video annotation in the TKB³ project: Linguistics meets choreography meets technology’. The TKB project involved a collaboration, starting in 2009, with the Portuguese choreographer Rui Horta to develop the Creation-Tool to ‘assist choreographers during compositional processes’ (Fernandes and Jürgens 2013, 119). The Creation-Tool supported various modes of annotation, including text and drawing on top of the video. Fernandes and her collaborators continued to develop this annotation software through other research projects, changing the name to Dance Pro in the context of the Europeana Space project.⁴ The latest iteration is Motion Notes, now web-based, which features in one of our contributions reporting on the use of this annotation software.⁵

Two other annotation software tools feature in the articles and documents of this issue. Research Video (RV)⁶ a project of the Zurich University of the Arts is an initiative that focussed on the potential of annotated video to be used as a publication medium for academic research. In a significant development toward this and reflecting the comments above about academic legitimacy, RV has formed a partnership with the Research Catalogue which underpins the Journal of Artistic Research.⁷ This enables the prospective peer review publication of annotated videos. RV was inspired by the annotation software project titled Piecemaker (PM) emerging from the Motion Bank⁸ project, now based in Mainz, DE. The first version of PM was created by David Kern, long-time collaborator and performer with Frankfurt-based choreographer William Forsythe. PM was designed as a real-time video annotation tool to support the rehearsal process of The Forsythe Company, and it was used for internal needs to that effect for a period of almost six years (2008–2013). The software was adopted by Motion Bank in 2011 and has been rewritten and now merged with a related publication tool into what is today called Motion Bank Web Systems.⁹ Several contributions to this issue report on the use of these Systems, with links to three published supplements to the main text contribution on-line.¹⁰ Our issue also has examples, including two video essays, of standard editing software such as Premiere Pro being used to annotate video recordings with text, graphics and line drawings. In the case of the video essays, this relationship is reversed with the supplement being the separately published Research Statement.

What is significant about these freely available annotation tools¹¹ is that they have all emerged out of a background of artistic and/ or cultural research. This affords them a certain artistic legitimacy that is in contrast to the example of ELAN with its claim to scientific legitimacy.¹² This refers more to cultural not technical differences. As discussed briefly, the academic legitimacy sought by some of our contributors through new forms of publication of embodied and artistic research does blur this distinction. In addition, some of the initiatives in the Choreographic Documentation issue were grounded in collaborations between artists and scientists. In this context, annotation offers a possibility to connect these two ‘worlds’, to enable exchange and communication between artists, scientists and technologists, and indeed one of our articles addresses this explicitly.

Finally, although the contributions to this issue do not address this explicitly, digitisation increasingly played an important role in the humanities over the last decades. Research data, data standards, knowledge representation and information architecture became an important topic with regard to literature, art and culture, not only but also in view of the emergence of digital archives. Especially in digital humanities, this also entails the development of digital methods that seek to exploit the possibilities of the growing pool of cultural data. While we have been suggesting here that the contributions to our issue are indicative of a growing area of unique methodological invention, grounded in an artistic legitimacy that is simultaneously striving for more academic recognition, it is also possible to speculate about a different set of connections to the digital humanities. The challenge here is nearly a paradox. On the one hand, the annotation software tools we have been writing about and are used by our contributors are designed to be usable by theatre, dance and other practitioners without specialist training in systems of notation, which establish certain shared standards and conventions in analysing and publishing dance. But these tools also do not require the practitioners to have the kind of ‘digital aptitude’ that underpins participation in the digital humanities. That is the ability to work behind the surface of the screen and its analogue juxtaposition of text and moving image to develop research questions at the level of metadata. (Blades and deLahunta 2020) Those contributions to our issue that do demonstrate this interest and capacity involve collaborations between dance and data specialists. The close study and augmentation of video recordings by dance practitioners provides the crucial starting point with a digital methodology built on top of this, with formalisation and computer-generated visualisations as part of a mixed-methods research design. Thinking about the future of similar approaches to collaborative research also refers us back to Toni Sant’s view that skills need to be obtained from fields such as ‘library and information science.’ (Sant 2017, 15)

As we wrote, we still sit at the crossroads of these two previous special issues of this journal. Between the artist-initiated shift beginning in the early 2000s toward digital documentation of specific performance practices with the goal of sharing and publishing embodied knowledge and, in the context of this growing collection of digital performance data, the proposal for more digital curation and data understanding. This issue has afforded us the opportunity to dig deeper into the kind of progress that is being made at this intersection between bodily practices and the digital, to draw attention to an emerging space of methodological invention, fresh theoretical and interdisciplinary insights and a shared story of software development. One aspect related to this story needs to be mentioned here. Without sustained development the annotation tools used by our contributors will become obsolete, as is often the case with software developed in the context of research projects. These are not software products, but are strictly non-commercial.¹³ They are freely available and some are open source. In particular, the open source projects create the possibility to integrate parts of their code into other digital archival projects.¹⁴ But we have to see how this story evolves, as without it some of the progress captured in our issue may disappear. The many influences on the use of digital annotations for the understanding of bodily practices we have included in this issue may be partial, but nevertheless show the complex field in which we situate this practice of annotation. As we witness the digital playing an even more significant role in our lives in the context of the current pandemic, one of our goals is to ensure that bodily practices are not forgotten. We hope this issue, with its many practice-based and focussed contributions, can make a positive contribution to this.

Notes

1 ELAN is software widely used in academia developed since 2000 by the Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen, The Netherlands. https://archive.mpi.nl/tla/elan (accessed 06 Feb. 2021).

2 The use of the term ‘methodological’ here is deliberate and refers to the growing contextual research framework our contributions may arguably be indicative of.

3 TKB refers to the Transmedia Knowledge-Base for Performing Arts. https://tkb.fcsh.unl.pt/. (accessed 06 Feb. 2021).

4 Dance Pro. Europeana Space project. https://www.europeana-space.eu/dancepro/. (accessed 06 Feb. 2021).

5 Motion Notes was developed in the context of Culture Moves. https://portal.culturemoves.eu/. Free registration with a link to tutorials here: https://motion-notes.di.fct.unl.pt/. (accessed 06 Feb. 2021).

6 Research Video is free to use. A link and tutorials are available here: https://researchvideo.zhdk.ch/ (accessed 06 Feb. 2021).

7 Journal for Artistic Research is an international peer-reviewed journal that disseminates artistic research from all disciplines. https://www.jar-online.net/journal-artistic-research. (accessed 06 Feb. 2021).

8 Motion Bank. First phase documentation and scores (2010–2014) http://motionbank.org/. Current developments: https://medium.com/motion-bank. (accessed 06 Feb. 2021).

9 Motion Bank Web Systems are free to use. Tutorials and access provided here: https://tutorials.motionbank.org/. (accessed 06 Feb. 2021).

10 Three supplementary websites: http://scores.motionbank.org/brazillab2019/; http://scores.motionbank.org/somatic-sauce/; http://scores.motionbank.org/jbmf-revisited/. (accessed 06 Feb. 2021).

11 There is an additional project called MemoRekall. The free webapp is provided here. http://www.memorekall.com/home-en.php. (accessed 06 Feb. 2021).

12 ELAN has a support system of funders and sponsors it would be very difficult to emulate in the field of artistic-scholarly research. https://archive.mpi.nl/tla/elan/funding. (accessed 06 Feb. 2021).

13 WeVu is one example of a commercial video annotation tool: https://wevu.video/. Education platforms https://mediasite.com/ and https://www.planetestream.co.uk/ also offer video annotation. (accessed 06 Feb. 2021).

14 Two dance organisations ICKAmsterdam (https://www.ickamsterdam.com/) and Pina Bausch Foundation (https://www.pinabausch.org/) have been working for several years with the Motion Bank Web Systems, adapting them to fit their documentation and archival aims.