Breaking from realism: exploring the potential of glitch in AI-generated dance

ABSTRACT

What role does deviation from realism play in the potential for generative artificial intelligence (AI) as a creative tool? A deep case-study was performed to explore interactions with AI-generated dance sequences as an inspiration source in dance composition and improvization. We present a simple interface created in collaboration with an experienced dancer for browsing AI-generated dance. By including glitches, the physics-breaking mistakes often encountered in AI-generated artefacts, we examine their affordances and possible use cases through sessions with the dancer. Through a process of reflexive thematic analysis, we identified that generative AI can engage a dancer through surprise, inspiring a transformation from abstract to physical movements. Our work challenges existing notions of the importance of realism in dance generation models, exemplifies the importance of close collaboration with practitioners in evaluating AI-generated artefacts and proposes glitch as a potential use case for dance ideation as it encourages dancers to embody unfamiliar movement qualities and break from ingrained patterns.

KEYWORDS:

• Dance
• generative AI
• artistic research
• thematic analysis

1. Introduction

In previous work, we found that dancers can be drawn to the potential of non-realism as a creative catalyst in human-AI interaction. Generative models of dance can often produce mistakes, creating output which is decidedly non-human. We refer to these mistakes as glitches. Glitches in AI-generated dance can take many different forms, such as getting stuck in repetition or becoming completely unrecognizable as a human form. We pose the question: to what extent could the glitches inspire movement and contribute to the dancer’s creative task? The importance of surprise and novelty to our perception of creativity is in no way a new concept, it is at the center of many crucial works on human (Sawyer 1999) and computer creativity (Boden 1992). Boden pinpoints surprise as a central marker of a creative artifact. She proposes three types of creativity we might observe in both humans and machines, each identified by a different type of surprise. When describing projects wherein human and computer can cooperate in generating otherwise unimaginable images (Boden 2007), she states that these computer-generated images can ‘often cause the third, deepest, form of surprise […] The one appears to be a radical transformation of the other, or even something entirely different.’ Glitch is most commonly understood to be synonymous with a machine error, an unexpected and usually unwanted deviation from the norm. Artists have engaged with glitch as a form of deconstruction and creation for decades, exploring and exposing noise artifacts inherent to technology (Menkman 2011).

As our work aims to produce models that might contribute to the creative processes of dancers, we seek to understand the relationship between artists and generative AI. Including dancers themselves in the development and evaluation of generative AI models for dance poses technical, logistical and communicative challenges. Fruitful collaboration requires that both AI-practitioners and members of dance communities are able to articulate complex experiences and technical intricacies. This may be one of the reasons this approach is often not prioritised. Instead, the evaluation of generative AI models of dance often focuses on accuracy, which leverages quantitative measures to argue that the generated output is realistic. For some applications realism may be a sufficient metric for measuring the quality of AI-generated dance. However, we question whether this is an appropriate method for evaluating AI when the rationale for the generated output is to spark artistic inspiration and novel ideas.

In this work, we examine these questions through a practice based case-study in the form of a series of sessions collaborating with a professional dancer and choreographer. The dancer has more than 15 years experience as a performer, dance teacher and choreographer working mainly with lyrical and modern jazz. Our research process was structured as follows, we implemented a simple online browsing interface and facilitated a series of sessions where the dancer could use the interface to explore AI-generated dance and glitch. The sessions included choreographic and improvization experiments wherein the dancer took an active role in deciding how the AI-generated dance was used. Throughout the time the dancer spent with the interface we collected his reflections through open-ended interviews, diary entries and videocued recall. Our work describes how the unexpected and broken nature of glitch forms a catalyst for a series of events for the dancer, beginning with the initial surprise of the glitch followed by an urge to transform the impossible movements of the AI through embodiment. By evaluating our AI-generated movement sequences through embedding them in an artistic practice we argue that glitch presents a viable use case for AI-generated dance as it can play an important role in improvization and ideation by promoting the breaking of habitual movement patterns.

2. Related work

The relationship between technology and dance has a history that spans several decades. From the earliest experiments with electronic music and projection technology (Schiphorst 2013) to the latest advancements in virtual and augmented reality (Plant et al. 2021) as well as AI-generated dance (Crnkovic-Friis and Crnkovic-Friis 2016). Technological advances have made their way into tools for visualizing and constructing choreography (McGregor 2020), in performance (Fdili Alaoui 2019), interactive installations (Jacob and Magerko 2015; Kumar, Long, and Magerko 2020; Rokeby 1986–1990), and in teaching (Soga, Umino, and Longstaff 2005). With the advent of generative models using deep learning, dance generation systems have become more prevalent (Bisig 2022). Many of these generative models require little or no interference from humans to create dance. The motivation for creating generative models of dance range from creating realistic choreographies for virtual agents in games to creating inspirational sources for the ideation phase of choreographic work as part of a creativity support tool (CST) (Berman and James 2015; Carlson, Schiphorst, and Pasquier 2011; Leach and Delahunta 2017). CSTs aim to enhance the creativity and innovation of users by providing a platform for idea generation, visualization, and collaboration. The ideation phase is generally seen as one of several stages in the creative process, together with problem analysis and idea evaluation (Gabriel et al. 2016), used to challenge an innovator’s existing mind set, allowing them to perceive the problem with fresh eyes (Shneiderman 2000).

A recent survey (Remy et al. 2020), found that 41% of studies on CSTs used short periods (less than or up to an hour) of interactions to evaluate their tool. Most evaluations of generative movement models consist of quantitative metrics such as preference ratings (Li et al. 2020), beat synchronicity (Li et al. 2021) and comparisons to other models. While these methods shed light on the models’ abilities to generate realistic dance movement, including practitioners in the development and evaluation of generative models is an important step to creating generative AI that is interesting to the people for which it is intended (Sturm and Ben-Tal 2017). When it comes to bodily practices, these crucial insights are partly embodied and can be challenging to put into words (Fdili Alaoui et al. 2015). Having dancers explore interactions with AI-generated movement as part of their practice may reveal otherwise elusive insights. This could in turn illuminate which aspects of AI in dance should be prioritized in future development of interactive and generative tools.

This work outlines what to our knowledge is the first exploration of the effect of AI-glitches on human-AI interactions in dance. However, previous studies involving interactions between human dancers and AI have already encountered examples of how deviance from realism may stimulate new ways of moving. In the project AI_am (Berman and James 2015) a dancer interacts in real time with an AI which displays ‘a weird grace, punctuated by a glitchy flow.’ This appears to impact the dialogue between the dancer and the AI in a positive way. As the authors describe it; exposure to these unnatural movements of the AI allows the dancer to ‘broaden the quality of movements’. Similarly, in the work of Carlson et al. (Carlson et al. 2016), the construction of choreography is facilitated through the selection of key-frames which are mutated using a generic algorithm. The resulting poses were often unusual and potentially impossible for a dancer to mimic directly. Leveraging the, at times, absurd outputs produced by generative AI, has also recently been explored in other contexts, such as using AI generated text as creative writing prompts (Singh et al. 2022). Composer Kim Cascone coined the term ‘the aesthetics of failure’ to describe the rapidly proliferating glitch music genre, propagated by often self-taught composers working ‘beneath the previously impenetrable veil of digital media’ (Cascone 2000), p. 12. The defining feature of this genre is the exploitation and exploration of the failings of technology in order to make new soundscapes, timbres and rhythms. Aspects of the glitch genre have now been adopted in mainstream pop-music.

Naturally, deviance from realism is not desirable in all cases of movement generation. When generating movement sequences for animation, for example, generative methods with high realism have already shown great promise in decreasing the workload of animators (Harvey et al. 2020; Starke et al. 2020). The realism and accuracy of a generative AI can also be important in interactions where it is beneficial for the user to gain a feeling of control (Fdili Alaoui 2019). Realism is seen as an important first step in the pursuit of creative AI (Ritchie 2007). This is reflected in many of the methods used in generative models of dance. Many of the most successful generative models used in dance generation today, including our own work, learn to predict the next pose based on the models’ training data (Li et al. 2020; Li et al. 2021). This encourages mimicry. However, in using AI-generated movement as a creative catalyst, realism may be less important.

3. Methods

As an initial exploration of the concept of glitch in improvization and dance creation, we invited the dancer to spend a few hours improvizing while taking inspiration from AI-generated movement sequences (see Figure 1). Following this initial exploration we decided to delve deeper into how generative AI-dance, and glitches in particular, can be embedded with dance practice. We did so by inviting the dancer to take part in a more in-depth process, using AI-generated dance clips of varying degrees of glitch in the construction of a choreography. Including our initial exploratory session, this work consisted of five stages: Initial experiments working with glitch and AI-generated dance, rating and interface building, choreographic development, post-choreography introspection and finally, implementation into teaching. An overview of our sessions and data collection can be found in Table 1. Each session is described in further detail in Section 4.

Figure 1. The dancer watches and reacts to the movement sequences generated by the model. Some sequences show a clear humanoid shape while others are more abstract.

Table 1. This work was performed across five sessions and documented through discussion transcripts, diary entries and video-cued recall. Our initial exploration of glitch as a creative catalyst prompted the implementation of our browsing interface and the following sessions using the interface in choreographic development and teaching.

Session ID	Description	Documentation
0	Dancer spends 2 h in improvization with access to 10 AI-generated dance animations displaying various amounts of glitch.	- Discussion transcripts - Photo - Video
1	Dancer views and rates 15 AI generated movement sequences based on criteria of his choosing. The ratings are then used to order the clips to be displayed through our online interface.	- Video-cued recall - Discussion transcripts - Photo - Video
2	Using the interface the dancer spends time creating a choreography.	- Diary entries
3	The completed choreography is recorded on video and played back with the dancer for video-cued recall and discussion.	- Video -cued recall - Discussion transcripts - Video
4	The interface is used in a 2-hour improvization class with six students.	- Discussion transcripts

3.1. The AI-generated dance

In previous work, we trained several generative deep neural networks on our open-access data set of 3D motion capture data (Wallace et al. 2019). The data set consists of 164 one-minute recordings of improvised dance performed by 9 dancers. Our implementations did not include any built-in movement constraints, instead, the models needed to learn these constraints from the underlying training data. Also, as no corrections applied to the generated output, the models occasionally produced impossible movements. Limbs can extend and rotate past human limits at velocities that would break physical laws. The clips compiled for this study range from abstract shapes where the human form is not identifiable, to simple movement sequences such as swaying and small arm movements. Some clips show a mix between the two where the model has produced sequences that are human-like in some parts and distorted in others. Figure 2 shows examples of three different clips presented to the dancer.

The majority of the sequences were generated by a mixture density recurrent neural network (MDRNN).

The MDRNN is a sequence prediction model which combines a recurrent neural network consisting of LSTM cells (Hochreiter and Schmidhuber 12 1997) and a mixture density network (MDN) (Bishop 1994). The remaining examples were produced by a Transformer model (Vaswani et al. 2017) trained on the same data. The transformer was implemented using the hyperparameters and architecture presented by Li et al. (2021). The models were sampled at different temperatures as well as from various stages in training in order to achieve different types of glitch. When the model is sampled at higher temperatures the glitches become more chaotic as the learned probability distributions are re-weighted creating a higher likelihood of generating unlikely values (Wallace et al. 2021a). For additional details on the data set and model implementations, we refer readers to our previous work (Wallace et al. 2021b).

Figure 2. Poses from three AI-generated dance clips shown to the dancer in our study. The clips range from stable, human-like movements (a) to semi-glitched, where the sequence contains moments where the body distorts (b) to fully glitched (c) where it is no longer possible to differentiate between limbs.

3.2. The browsing interface

We animated the pre-generated dance sequences using p5.js. The movement sequences were displayed using point-line visualizations and were given an alphabetical ID marker based on their order in the application (see Figure 4(b)). The application was made available through a website allowing users to navigate through the different animations using the arrow keys. The interface is available at https://golden-longma-2c587c.netlify.app/.

Figure 4. (a) The dancer explores the AI-generated clips. (b) A screen shot of the interface deployed to the web.

3.3. Data collection and analysis

This research was conducted in accordance with all norms and regulations for ethics at the research institute and has been approved by the relevant national ethics agency.¹ The nature of the project was disclosed to the participant and informed consent was received at the outset of this research.

Sessions with the dancer were recorded on video and later transcribed and translated. The dancer’s initial interactions with the interface as well as their final choreography was recorded and later played back during video-cued recall sessions with the dancer and first author present to gain the dancer’s reflections on their embodied experience. While the dancer worked on the choreography he recorded video and took notes of his process, which were later shared with the authors. We used a reflexive thematic analysis approach (Braun and Clarke 2019) to conceptualize themes based on the various data sources collected from the sessions. The material was translated from Norwegian to English. Following this, the files were coded using an iterative process. Some initial impressions were formed at this stage and discussed among the authors. These impressions formed the basis for following iterations of coding and code-clustering. By working reflexively with the data and codes, we conceptualized three themes. The themes are detailed in our findings section.

4. Embodying the glitch

The following sections describe the implementation of each session with the dancer in further detail.

4.1. Initial experiment working with glitch and AI-generated dance

As part of our initial exploration of the use of non-realistic AI-generated dance in a dancer’s creative practice, we presented the dancer with a collection of movement sequences for a dance composition task. The dancer was asked to imagine that he needs to compose a dance and that his inspiration for the piece can come from 1 or more of 10 short (10-30 s) AI-generated movement sequences which range from realistic to highly abstract. The animations were projected on to the wall of the dance studio (see Figure 1). The first author was present for the improvization session and controlled the playback of the animations from a laptop. Each clip was looped until the dancer decided to move to a new clip. This improvization session was followed up by a discussion between the dancer and first author. The aim of this session was to facilitate reflections around how the glitched clips affected the dancer’s improvizations compared to working alone.

During discussions following the interaction session the dancer explained that the clips where the human shape is clearly visible lay stricter guidelines for where his movements begin and where they go. In this sense, the more humanoid shapes portraying simple movements such as extending an arm are easier to work with as they form a good foundation for his improvization. However, when imagining using a generative model to browse through movement sequences and see if something sparks his imagination, he clarified that he would not necessarily want all sequences to be recognizable as a human form. Building on his ideas from this session we prepared a browsing interface where AI-generated movement clips of varying degree of glitch are accessible and invite the dancer to take part in setting the interface up, selecting clips and using it in his practice to create a choreography.

4.2. Rating and interface building

We began the following session by having the dancer rate 15 AI generated movement sequences based on criteria of his choosing. This gave us some initial insights into what measures the dancer used when evaluating which AI-generated movements were most interesting. The dancer decided to rate each clip based on 4 criteria: glitch, amount of movement, flow and preference. He explained that his choices of criteria were based on his initial impressions of the differences between clips, as well as his personal movement style. Figure 3 shows the ratings for each of the 15 clips.

Figure 3. Line graph presenting the ratings from 1 (least) to 5 (most) across four features chosen by the dancer: quantity of movement (QoM), preference, glitch, and flow. The dancer’s preference for each clip is most closely correlated (RMSE = 0.70) to the QoM.

The dancer felt the most meaningful order would be amount of movement as this allowed him to construct a dynamic of changing intensity in his choreography. Based on the dancer’s ratings, the clips were placed in order from lowest amount of movement to highest. The animations would thereby increase in amount of movement as he navigated forwards using the right arrow key. This resulted in the ordering shown in Figure 3, with clip IDs corresponding to the dancer’s rating of the amount of movement in each clip. One clip was disregarded and removed from the application during rating as the dancer explained that it lacked enough movement to contribute anything.

Following the ratings, the dancer tried out the browsing interface with the clips placed in the order he decided on. This initial browsing and improvization with the AI-generated clips was recorded on video and subsequently watched back together with the first author in order to get the dancer’s account of his initial impressions from using the interface and working with the AI-generated movement sequences. Having defined the ratings and interface, subsequent sessions involved using this interface to create new choreography.

4.3. Choreographic development

To test the utility of our interface for assisting in a choreographic task, the dancer was asked to create a dance using the AI-generated movements as inspiration. As the interface was deployed online the dancer could access it at any time and was asked to use it for as long as he wished. While working with the clips the dancer began by selecting a piece of music to accompany his choreography. This is similar to his usual process: ‘I always start with the music. Or I start with a feeling and then I choose a song.’ While the music plays in the background he browsed through the clips, taking note of a selection that he felt spoke to him. In his second session working with the interface he decided on a structure for the clips he had selected. The dancer explained that he wanted his composition to have a bell-curved progression in the amount of movement: starting small, getting progressively more active, and then slowing down again.

He spent some time taking notes for each of the clips he selected, describing their movement patterns. He requested that the order of the clips be altered to reflect his new selection and structure. Integrating this suggestion, the final collection of clips and their order was set to B, G, F, I, K, N, H, C. Following this new structure the dancer began improvizing with the different clips. Over time, patterns emerged within his improvization. Certain movement sequences would repeat themselves throughout his improvizations and stick in his memory. He referred to his awareness of these patterns as the construction of tendencies. As a concluding part of the choreographic work he worked to make these tendencies something permanent: ‘This happened through continuing the improvisation until the tendency had cemented itself in my body and had become more concrete.’

4.4. Post-choreography introspection

After the dancer reported that he had completed his choreography, we were invited to his dance studio to record the performance and discuss his experience using the interface. Using video-cued recall of the performance, the dancer detailed his process, reflecting on how the different clips affected the final choreography. We also discussed the dancer’s experience so far and considered possible ways forwards. A picture-in-picture recording of the completed choreography and the clips selected by the dancer is available online.²

When discussing how he might imagine using AI-generated dance in his practice, the dancer explained that for him – as an experienced dancer and choreographer – using the AI-generated dance as a tool for inspiration is somewhat redundant: ‘For me, when making something, unless I am really stuck choreographically, I don’t really know how useful it is for me.’ The dancer further explained that there are two use-cases he would like to explore, one would be to use the animations on stage in a chance choreography (Cohen 1961). The second use-case described by the dancer is using the animations in his teaching, particularly as prompts in an improvization class. While both use cases form interesting avenues of research, we chose to prioritize using the interface in an improvization class.

4.5. Implementation into teaching

Dance students from a local dance school were invited to take part in a 2.5 h improvization class with the AI-dance interface. Six students took part, each with around 10 years experience as dance students in classical, modern, jazz and contemporary styles. At the dancer’s request, an additional set of AIgenerated clips were added to the interface and the ordering was randomized prior to the improvization class. The students were asked to take part in three rounds of improvization. In the first, the students were given the instruction to let themselves be inspired by what they saw on the screen. They were encouraged to move freely while the AI-generated clips were played through one by one, projected on a large screen on stage and controlled by the teacher (see Figure 6). After a brief discussion the dancer initiated a second round of improvization. Here, the students were paired up. One faced the projection on the screen and was given the same instruction as in the first session. Their partner stood with their back to the screen and so could not see the AI-generated clips but instead took their cues from their partner’s movements. The students were not given any further instructions but were asked to not touch each other. In the last session the students continued to be paired up, but were now allowed to move freely with each other and both were free to observe the AI-generated movements. Figure 5 shows a visualization of the field-of-view of the students during each prompt. Following the improvization class, a final discussion session with the dancer was recorded to explore how the AI-generated dance was used, how the students and dancer experienced the use of the interface in this setting, and how it might be improved in future work.

Figure 5. The dancer provided three different prompts in the improvization class. During prompt 1 the students face the screen, during the second prompt students face each other so only one in each pair can see the screen. In the final prompt the students can move freely. This figure shows the field of view of the dancers for each prompt. During prompt 1 and 3 the students all see the AI, while during prompt 2 only one student in each pair can see the AI.

Figure 6. Six dance students use the interface as a visual inspiration source during the improvization class.

5. Findings and reflections

Glitch has been explored not only as noise or merely a mistake, but as a tool for understanding (Peña and James 2016) and something that allows us to celebrate ‘failure as a generative force’ (Russell 2021). Our findings support these notions of glitch as an instigator for exploration through disruption. From our data we conceptualize three themes; surprise as a challenge and a benefit, transforming the impossible and breaking movement patterns. We further define a temporal ordering, as can be seen in Figure 7, where surprise encourages transformation which allows for the breaking of movement patterns. Our three themes can in this way be viewed as a process containing a series of events unfolding over time. In the following sections we present these themes.

Figure 7. We find a natural ordering of our conceptual themes, where surprise encourages transformation which allows for the breaking of movement patterns. This process is initiated when improvizing using the AI-glitches as a creative catalyst, allowing the dancer to challenge himself, through translation from impossible to possible, to break ingrained movement patterns.

5.1. Surprise as a challenge and a benefit

Throughout the study, notions of surprise or disruption frequently came up when discussing glitch, both in reference to positive experiences and as a challenge. The unpredictable nature of the glitches would at times disturb the flow of his improvization, causing him to stop and think. Reflecting on these moments during video-cued recall, the dancer revealed that he did not experience these pauses as particularly disruptive to his overall process. The clips initiate associations in his mind, when he feels he has fully internalized a clip he describes a feeling of disinterest that motivates him to switch to the next clip. When an example contains moments of glitch, this process of understanding and internalizing the clip seems to take longer and elicit more thought and engagement. He described during his first interaction with the interface that ‘[the examples with more glitch] give more opportunities to do fun things’. If one of the clips surprised him or disrupted his flow he describes the event as refreshing or re-situating him in the present moment. The second prompt of the improvization class, where one of the students in each pairing could see the AI and one could not, was perhaps the most challenging as the students would ‘see their partners movement language suddenly change without knowing why’. Simultaneously, this was ‘one of the things they found most exciting’. The improvizations in this session prompted ‘some interesting twists and turns’, which struck him as particularly compelling. When the dancer’s experienced the surprise of a novel movement sequence, they were challenged to rethink their movement patterns, particularly when the movement sequence contained strange or impossible movement.

5.2. Transforming the impossible

The moment of disruption would often initiate a deeper engagement in translation from the movements performed by the AI to the dancer’s own movement language. We repeatedly observed the dancer’s use of mirroring and shadowing (Blackwell, Bown, and Young 2012) of the movements displayed by the AI. During the video-cued recall immediately following his first session using the interface, the dancer explained that his strategy was to ‘look at each individual avatar, extract the movement pattern from each avatar, and try to set it in to [his] own body.’ This transformation process was naturally the most clear when the dancer was interacting with the examples that are the most realistic. He explained that this process of transformation was more ‘literal’ with the clips that contained less glitch. In these cases he felt less inclined to interpret the movements, stating that: ‘you can just do what it does.’ These moments of mirroring and shadowing were less obvious to the authors when observing the improvization with the more abstract examples. In video-cued recall of these improvizations however, the dancer clearly found links between his movements and his interpretation, stating for example that ‘[The AI-generated dance] has some poses where I feel like its falling. So I tried to play around with that’.

Many of the poses generated by the AI are impossible to mimic. The dancer describes that in these cases he shifted from mirroring to focusing on the overall shape of the movement. How does it rotate?

Are the movements soft or sharp? Does it come across as stretched out or compressed? The dancer explains that the process requires him to perform a transformation from what he sees to how he can move: ‘[Regarding glitches] you can’t mimic. You must interpret.’ The dancer expressed that the glitches allowed him to feel like he was being more innovative and ‘personal’ in his own movements compared to when the AI was less abstract. While the dancer felt challenged in positive ways by this translation process, attempting to internalize the non-realistic movements was at times challenging for his students. When reflecting on the use of the interface in the improvization class, he explained that while the majority of his students found the more glitched clips to be motivating, two students out of the six expressed that they at first ‘struggled to take something from these movements’. When reflecting on the improvization class, the dancer describes this process as ‘[The students] are imbuing the movements with ‘soul’.’ Through the process of transformation by embodying impossible movements, the dancers’ movement languages are challenged and unfamiliar patterns of movement are discovered.

5.3. Breaking movement patterns

While the clips that were clearly discernible as human were easy for the dancer to translate into his own movement patterns, the more abstract shapes triggered different associations. The glitches appeared to be most beneficial when they were transient, springing from, or reverting back to, a recognizable human form after some time. In these cases, the dancer found both a grounding in the familiar and an inspirational goal that was inherently unreachable. The human-like examples grounded his movements in the concrete. When the figure distorts, shrinks and grows, he could no longer mimic the shape and thus his focus shifted to other aspects, allowing new ideas to form. He noted how, to him, the more chaotic clips prompted a specific way of moving, stating that ‘one of the things that [the examples with more glitch] generate in me, is speed’. The glitches also prompted him to move in ways he would not usually move. In his first improvization with the clips he noted that: ‘I also go down into a bridge pose at some point – it didn’t look so good but it’s what it looks like the avatar is doing. I really liked working with this clip. It gave me a lot to work with’. This movement was not something he would usually do, but he interpreted the movements of the AI as a bridge pose, and so he attempted to incorporate this into his improvization. While he did not necessarily like how this looked, he enjoyed this breach of his usual dance style inspired by the AI’s movements. At several points the dancer noted that the glitches inspired a playfulness in him: ‘[The glitch] has a little wiggle that I enjoyed playing with, almost like a drunkenness.’ Playfulness was particularly present when the glitched movements were difficult to interpret: ‘the head seems to move through openings [in the body] which is fun to play with’.

When reflecting on the improvization class, the dancer mentioned the AI glitches being particularly useful for the students that have a harder time breaking from their usual movement style: ‘Everything we do in improvisation classes is meant to encourage students to break out of their habitual movements and create something new.’ ‘[the AI-generated dances] generate an impulse to create something. And if I was new to choreography, and not as established as I am today, I think it would be more useful for me to get these kinds of prompts because it would help me create an expression.’ When breaking from an ingrained movement pattern the dancers expand their range of movement, gaining insight into their own body mechanics and allowing them to explore their preferences as well as their dislikes.

6. Discussion

Our three themes can be viewed as a process containing a series of events unfolding over time. This process is beneficial in dance creation and improvization when the aim is to break from habitual movements and find inspiration and self-awareness. We find that glitch and non-realism facilitate this process by acting as a catalyst for movement and aiding in re-imagining movement through a form of defamiliarization. In the following sections we contextualize our findings and their implications for the role of generative AI in dance practice. We then turn to explore limitations and present potential avenues for future work.

6.1. Glitch as a use-case for generative AI in dance

Glitches are by definition unexpected. They are mistakes, deviations from an otherwise predictable sequence. This can create a challenge, a mistake to correct. The act of re-adjusting our expectations is in many ways itself an act of creativity. Kimmel, Hristova, and Kussmaul (2018) point to the similarity between contact improvization and jazz improvization, emphasizing how mistakes urge the participants to rethink and re-calibrate: ‘when one’s ideas fail in interaction and an instant repair or re-contextualization is needed, creativity is stimulated’ (Kimmel, Hristova, and Kussmaul 2018, 13). When our expectations are not met, it encourages us to expand our inner model of the world. This is expressed in Boden’s definition of the different types of creativity and how each can be tied to a different kind of surprise (Boden 2010). In Sangild’s descriptions of the appeal of glitch music, we find similar reasoning: ‘Digital technology allows for cloning, for perfect copying of digital information. When bugs and glitches occur, such perfection is disturbed. Imperfections are often annoying, some would say downright ‘bad,’ but they can also be used aesthetically to create new sounds and textures, causing music to evolve further.’ (Sangild 2013, 207)

Making mistakes is often associated with being human. Thereby, glitches and malfunctions in machinery can have a humanizing effect. At one point, the dancer said that a certain movement seemed to him to be ‘something the glitch wants to do’, as though the glitch itself was a personality, intervening in the AI’s attempt to move. An empathy with the AI model seemed to emerge from this, which inspired a sense of freedom in the dancer. Similar notions have been found in studies of human-AI interaction in music (Thelle and Fiebrink 2022), where musicians expressed a feeling of freedom from judgement when improvizing with an AI counterpart. Exploiting an apparent limitation or boundary to promote creativity is a strategy found in many creative fields. For example, Brian Eno and Peter Schmidt (Eno 2001) created Oblique Strategies with this approach in mind. Taking inspiration from an unpredictable and strange agent, stimulates creativity. In this study we see the dancer attempt to imbue the artefacts with meaning and movement. This translation between modalities can be likened to tasks such as converting text to images (Ramesh et al. 2021), or creating drum patterns using a Chopin étude (Sol 2022).

Dance is constrained by our bio-mechanics since our movements are restricted by what the human body can do. While composers and writers can randomize characters or musical notes as easily as a generative AI, dancers cannot grow or shrink their limbs, and their speed and position are limited. In order to transform the glitches produced by the AI into something the human body can do, an expansion of our interpretation is required as the dancer attempts to embody the glitch. In this way, the glitches are similar to the use of inanimate objects in performance and improvization practices (Galeano and Matuszak 2014). The use of prompts or objects in improvization, is common in many styles of dance. Most often these take the form of verbal instructions. Either asking for concrete dance steps or body parts to be used, or by introducing some mental imagery such as ‘you are a sprouting seed’. When asked to elaborate on how the AIgenerated dance clips are different from verbal instructions he explains: ‘If you use a prop you give no guidelines regarding the movement quality, or expression, or mood, or how much of the body [the improvisor] should use. But if you use the avatar, you will get a more concrete guideline on how much of the body to use, mood, flow or not flow. You will give them a framework that is more concrete than a prop, but less inhibiting than an instruction.’ Thereby, AI-glitches have the potential to exist in a sort of sweet-spot, remaining somewhat familiar, yet challenging the observers expectation.

As with the notion of defamiliarization within design (Wilde, Vallgårda, and Tomico 2017), attempting to embody the AI-generated movements as they flow in and out of a recognizable human form, is a way of making familiar movements strange (Loke and Robertson 2013). Defamiliarization is employed in art and design to break our habitual patterns of perception by altering the familiar, allowing us to gain new perspectives. When interacting with strange virtual agents (Berman and James 2015), or bodies with unfamiliar morphologies (Gemeinboeck and Saunders 2017) we can experience a broadening of our own movement qualities. Building interactive AI models of dance that encourage our observed sequence of events culminating in breaking familiar patterns of movement, can introduce valuable tools for improvization and choreographic ideation. Exploring the non-realism of glitch in AI-generated dance can inspire dancers to try new movement combinations, explore new physical possibilities and challenge their preconceived notions of what is possible within their body. By leveraging the body of work which presents the importance of surprise and unpredictability in what we perceive as creative, we argue that future development of generative AI for dance should center these aspects.

6.2. Limitations and future work

This research describes a practice-based study to examine how a dancer with a background in lyrical modern dance responds to glitches in our generative AI system. The findings build on existing ideas of how surprise, free association and lateral thinking encourages novel ideas. As the use of visual media, and particularly AI-generated movements, is in itself a novel experience for the dancer and his students, we also acknowledge that this novelty can be expected to wear off as participants get used to using the interface. The initial novelty of seeing the impossible movements of the AI triggers a longer process of interpretation and internalization of the glitch. However, in our work we experienced that the dancer ultimately familiarizes himself with the relatively small number of examples provided. If the glitch becomes predictable, we lose the initial spark that initiates the positive cycle of surprise, transformation and breaking patterns.

Limiting the study to focus on a single dancer allows us to delve deep into their specific experience, yet simultaneously makes it difficult to draw more general conclusions. Other dancers with different backgrounds and experiences could approach the experiment differently and contribute valuable, and perhaps, opposing views. The examples which were discarded by the dancer, for example, were those where the AI-generated movements were small. Reflecting on this, he explains that he considers himself a ‘quite maximalist choreographer’ and that dancers with different stylistic approaches could see the examples containing less movement as very interesting starting points. By including additional dancers with varying backgrounds in development and evaluation, we could gain a more nuanced understanding of the effects uncovered in this study.

The dancers also provided feedback on the usability and functionality of the interface. To ensure that each clip is visually distinct the dancer suggested changing the color of each clip. Considering the importance of novelty, the dancer further suggested generating new movements on-the-fly such that the element of surprise is preserved even when spending several hours with the application. In future work, we aim to improve on the interface and continue collaboration with the dancer to perform iterative evaluation of the interface development.

7. Conclusion

Our study suggests that the AI-generated movements were able to draw the dancer out of his usual movement patterns. These findings support previous research in creative domains which upholds the importance of surprise and unfamiliarity in creative catalysts, and supplies a specific example of how this manifests in AI-generated dance. The more human-like examples allowed the dancer to find a grounding in concrete movement concepts by providing a clear starting point for improvization. Concurrently, the higher level of realism introduced a mental constraint that often led to disinterest. Through including occasional glitch in the AI-generated dance a transformation is initiated from the abstract shapes or impossible movements into movements that the dancers could perform. The dancer experienced a disruption of his expectations, which prompted a deeper engagement with his own movements in order to interpret the glitches and transform them. This process encouraged him to break from his usual movement patterns. The dancer offered a succinct description of this challenge: ‘What is difficult in improvisation is to create movements that you have not been taught, and instead create new paths. [..] you often begin with the steps you have learned in your classes, the nice pointe technique you have practiced for hours, the pirouettes, opening up in the chest. In improvisation, that is not necessarily what we want. What we want is for you to be yourself, for your inner movement language to come through, to create new movements, and that is quite difficult.’ We argue that exploring glitch facilitates a sequence of surprise, transformation and the breaking of patterns, creating a practical use case for AI-generated dance in improvization and ideation. This challenges the notion that realism should be the ultimate goal for generative AI in an artistic practice. We suggest that there is potential in occasionally embracing, and particularly embodying, the glitch.

Acknowledgments

The authors would like to thank dancer and choreographer Bendik Sundby for contributing his time, reflections and performance.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was partially supported by the Research Council of Norway through its Centres of Excellence scheme, project number [262762].

Notes on contributors

Benedikte Wallace

Benedikte Wallace is a researcher at the University of Oslo, Department of Informatics and RITMO where her work explores human-AI interaction through deep learning, motion capture and creative computing.

Kristian Nymoen

Kristian Nymoen leads the data governance program at Skagerak Kraft and has an interdisciplinary academic background spanning from computer science to music cognition. His research has focused on digital signal processing of motion data (sensors, video, motion capture) and audio data.

Jim Torresen

Jim Torresen holds a Professor position at Department of Informatics and RITMO at the University of Oslo. His research interests include machine learning, bio-inspired computing, artificial intelligence, adaptive systems, robotics, reconfigurable hardware and applying this to complex real-world applications.

Charles Patrick Martin

Charles Patrick Martin is a computer scientist specialising in music technology, creative AI and human-computer interaction at The Australian National University, Canberra. Charles develops musical apps such as MicroJam, and PhaseRings, researches creative AI, and performs music with Ensemble Metatone and Andromeda is Coming.

Notes

1 University name and ethics agency information omitted to ensure anonymity

2 https://youtu.be/XGQnKkrOEsU