The black box aesthetics: performative dynamics between artificial intelligence and human interactive staging

ABSTRACT

The present paper addresses a theoretical model for human-machine interaction in intermedia performance taking as reference the author’s perspective. It is discussed the role of artificial intelligence as an autonomous, pervasive, and real-time evolving entity in dialectical relation with the performer and/or audience. The convergence of physical and virtual elements suggests the emergence of hybrid systems, insofar as digital instances manifest their own agency and humans show mechanical actions linked to information processing. Three typologies of human-machine interaction are inferred: 1) the algorithm elaborates the output through self-referential data, as in Δnfang; 2) the algorithm gathers information from performed actions, as in Convergence; 3) the algorithm processes data from the audience’s reaction, as in DoPPioGioco. As digital and analogue devices are employed, programming environments are hidden from human agents and filtered through the author’s gaze. Comparing the underlying black box rules with stage setting, it is observed to what extent algorithms are mediated and enacted. The article considers the information bias commonly attributed to artificial intelligence as rooted in the aesthetic domain, with significant dramaturgical implications.

KEYWORDS:

• Artificial intelligence
• algorithmic theatre
• digital agency
• intermedia performance
• black box aesthetics

Acknowledgements

The permission to report figures and information about the pieces has been kindly granted by the Fronte Vacuo collective Figure 1, and additional information gathered by the interviews with Marco Donnarumma and Andrea Familiari; Alexander Schubert for Figure 3 and additional information gathered via the email exchange; Antonio Pizzo on behalf of CIRMA for Figure 5.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Notes

1 The relationship between performance and technology emerges as a relevant issue during the 60s. Artists such as John Cage, Alvin Lucier, Nam June Paik, Dick Higgins, started to experiment with an extended conception of plays merging sounds, visuals, movements, and texts often extemporaneously generated and enhancing the importance of the surrounding environment in influencing the content (Higgins and Kahn 2012). In their happenings, bodies and objects on stage were mediated by electronic tools such as electrodes, oscillators, computers, or televisions, which soon became the focal element to break down traditional rules of representation. During the 90s and 2000s, such attitude was implemented through new prosthetic technologies (e.g. in Stelarc, and Marcel·lí Antúnez Roca), the internet (e.g. in Tod Machover, and Shu Lea Cheang), video games (e.g. in Blast Theory, and John Paul Bichard), and software implementation (e.g. Isadora, and Eyecon) (Dixon 2007; Giannachi 2004). The contemporary research, as held by the mentioned artists and further analysed, also mixes the definition of specific programming environments and various media with pluralistic and extemporaneous participation by the audience, performers, and digital entities.

2 Agency is commonly addressed to humans as the ‘satisfying power to take meaningful action and see the results of our decisions and choices’ (Murray 2016, 123). When applied to computing, it requires both the author to exploit the ‘system-modelling abilities to bring forth life … from empty matter’ (Murray 2016, 223–224) and the observer’s belief in machine consciousness (Bates 1994). As further discussed, the selection of specific parameters implies the shaping of live entities perceivable through devices on stage. Artificial agency might be simulated via predefined outcomes or affect the performance in real-time through stochastic processes, still concealing the underlying digital dynamics to a certain extent.

3 The term, as used in machine learning, derives from statistics and refers to a test for proving ‘if a data sample is typical or atypical compared to a population assuming a hypothesis we formulated about the population is true’ (Emmert-Streib and Dehmer 2019, 946). In contrast to the philosophical acceptation, meaning the assumptions of facts not yet realised but which are predicted as possible, it considers only quantitative data based on probabilistic functions, thus implying the mapping of data samples and their observation through the comparison with a benchmark.

4 The performance was premiered at Romaeuropa festival in October 2019. Except for the article mentioned in the paragraph (Caramiaux and Donnarumma 2021), drafted by the authors themselves, there are no scientific writings about the play. For more information, see the Fronte Vacuo website (Fronte Vacuo 2022). Data here reported have been also inferred through the online interview held with Marco Donnarumma and Andrea Familari on May 9, 2022, and the analysis of the entire performance video of the premiere kindly given to me by the authors.

5 Pure Data and TouchDesigner are visual development platforms based on nodes and real-time processing used to program multimedia systems (Derivative 2017; IEM 2022). The former is employed by Donnarumma for sound management; the latter by Familari for light control.

6 These numeric data are the predictions proposed by the algorithm, continuously changed towards reaching the target values; the distances between these values and the target ones; the rewards automatically given to the algorithm to understand if it is approaching or not the target values (positive when getting closer and negative otherwise). Musical patterns involve 14 different phrases composed of 8 notes on different pitches and with different accents; these phrases are selected or not according to the AI numerical outputs. Light pulsations, instead, regard three different elaboration of the AI values each bringing to a specific oscillation.

7 The piece, co-developed with IRCAM, provides two versions: the staged and recorded ones. The former, premiered in October 2020 in Bundeskunsthalle, Bonn, embodies the original concept. The latter, premiered in February 2021, was conceived only for online streaming due to the pandemic restrictions of that time (Schubert 2021a). I will consider the staged performance because of its prominence for the author and due to the focus on the present article. Nevertheless, I will report data about the recorded one because it is the only extended performance available and, as stated in the extended explanation done by Schubert himself (Schubert 2021b), propose the same dramaturgical content. Other information has been inferred through the email exchange with Schubert himself occurred on March 21, 2022, to whom I am kindly thankful.

8 As Schubert himself stated in the abovementioned email exchange, also the elaboration is, currently, partially fixed, even if the author’s goal is to make it entirely extemporaneous in the future.

9 The performance test run took place at the University of Turin in February 2019. Except for the article mentioned in the paragraph, drafted by the authors themselves, there are no other scientific writings about the performance. Data here reported have been also gathered through the online video documentation on Vimeo (Officine Sintetiche 2019).

10 The relationships between the algorithm, performers, and audience mentioned during the article might be further extended to other kinds of interaction. For example, the audience or both audience and performers can be encouraged to react to the algorithm outputs; or the AI may gather data from the audience and performers simultaneously. Most presumably, the underlying dynamics related to the digital process would similarly persist.

Additional information

Notes on contributors

Luca Befera

Luca Befera is a PhD candidate at the Department of Arts and Humanities, University of Turin, where he studies the aesthetics of algorithmic performance and intermedial artworks. He merges the analysis of the compositive process with ethnographic methodologies, delving into the transformative potential of staged actions. Besides his current research, he has applied this approach since the internship held in the Hochschule für Musik und Theater Hamburg in 2020 under the supervision of Alexander Schubert, aimed to indagate some of the author’s pieces in loco. In his former training in musicology at the Department of Cremona, University of Pavia, he examined the influence of digital syntax and devices on contemporary sound-based approaches, especially focusing on post-spectral authors and electronic dance music. During and after his master’s degree, he supported the courses in Music Theory there held. He also earned a diploma in classical piano, currently evolving his performative practices towards contemporary approaches. He has published articles and participated in conferences encompassing various topics, such as the influence of information technology on composing mediation of digital and analogue devices on scenic settings, and interactive and social dynamics within multimedia theatre.