Being there – XR performance at PQ23

ABSTRACT

This article presents a personal, critical reflection on the XR work presented at PQ23 that explores virtual aesthetics, emergent cognitive paradigms and their impact on the art-function of performance. The discussion considers the impact and potential of extended reality (XR) forms for managing this ‘art-function’ at the point of live encounter. I will consider critical frameworks that explore the psycho/physical impact of VR on the user (Popat, Thomas and Glowacki) and phenomenological approaches that aim to disrupt the political limitations of normative presentations of ‘possibility’ (Escobar, Grosz, Ahmed) in order to develop a framework within which work presented at PQ23 might be examined.

Being there

The PQ23 theme ‘RARE’ included an explicit invitation to consider a juxtaposition of the live and the virtual, and the impact of live cultural events on our humanity (|Fantová 2022). Recent years have seen significant blurring of definitions of ‘liveness’ and ‘presence’ in contexts relating to every aspect of daily experience, and PQ23 presented a range of projects across various programmes that sought to exploit the practical and artistic possibilities offered by virtual and mixed modes of ‘being there’. Indeed, much discourse in this area focusses precisely on the ways in which the explicit doubling of experience within immersive technologies can operate to generate a genuine and unique sense of presence which, when framed scenographically, can be particularly affective. Themes suggested by Popat (2016) and Thomas and Glowacki (2018) on the experiential nature of virtual environments draw on James Gibson's (1986) models of visual perception (particularly those relating to concepts of affordance) and the role of proprioception in user experience.

At PQ23, extended reality (XR) work in exhibitions and in panel discussions at the Art and Digital Talks (presented as part of the 36Q + H40 collaboration) demonstrated a growing sense that there has been a profound change in the ways in which we relate to ourselves and to our environment. Shifts in cognitive paradigms have become increasingly evident over the last 15 years, and while emergent technologies are undoubtably at the heart of this phenomenon it cannot simply be explained by increased familiarity with those technologies. Panels and audiences at the Art Talks were in agreement that not only is the world radically different to the one we found ourselves in 2007 (the watershed/Rubicon launch of the first iPhone) but that in a post-Bowie era, even 2016 now seems disproportionately distant. This radical repositioning of human experience in the late 2010s is evident in the political/anthropological works of commentators such as Dona Haraway (2015) and James Lovelock (2019), which seek to shift the articulation of epoch from geological to human timescales, fragmenting the Anthropocene into a number of identifiable sub-epochs characterised by human, social, political and ecological step changes that operate both serially and synchronously.

For Haraway, the critical factors in the identification (and naming) of epoch should now be ‘scale, rate/speed, synchronicity, and complexity’ (Haraway 2015, 159), and while the scale and speed of recent change are self-evident, I think that the key to understanding recent cognitive shifts and their impact on performance lies in notions of synchronicity and complexity.

PQ23 XR

The deployment of virtual, augmented and mixed reality (VR, AR¹ and MR respectively, but collectively ‘extended’ reality or XR) technologies was most evident at PQ23 in the Exhibition of Countries and Regions (Estonia, Georgia, Mongolia, Thailand), Performance Space (This space is for you/cet espace est pour toi) and 36Q/Arts Talks. The intentions of artists deploying virtual environments² varied significantly between projects, but broadly speaking fell into three categories: the capture and re/presentation of performance and archive materials, projects of facilitation, and dramaturgical interventions which explicitly constitute a relational ‘formation’ in Bourriaud's (2002) sense of the concept. While it was in the last of these that I found the most affecting work, a consideration of this work in total raises questions relating to the position and action of XR technologies in contemporary scenographic (and dramaturgical) practice – questions which for me prompt consideration of a series of theoretical frameworks that explore the conceptual functioning of these technologies in performance.

Re/presentation of performance and archive materials

The Exhibition of Countries and Regions included three exhibits that deployed VR technologies as part of their offer, each taking a very different approach to framing and presentation. Each work offered models of capture and reorganisation of performance propositions in which choices relating to locus (real/virtual) and ambiguity (explicit/implicit) of framing became critical aspects of the positioning of the audience.

Much critical discourse in this area focusses on virtual environments (VEs) as exemplars of scenographic worlding, in which notions of affordance are prominent. Thomas and Glowacki extend these arguments and propose that ‘participation’ in VEs presupposes that the participant can in some way affect their environment. They argue that VEs can be split into two categories – ‘those in which the user or participant is connected i.e. can reach out and touch the [world] and those in which they are disconnected’ (Thomas and Glowacki 2018, 147) – and that the latter does not qualify as virtual reality, as agency is not afforded to the user. However, definitions of affordance that rely on environmental interaction are problematic in a consideration of performance, where physical interaction is not necessarily encouraged (or desirable). In these contexts, the invitation to engage in acts of composition, through movement or aspect, might be considered an appropriate affordance. Indeed, Gibson's (1986) model of ecological perception (from which notions of affordance are drawn) includes a significant emphasis on the role of motion as an active element of visual perception. For Gibson it is important that the optical array is in constant flux. The head moves, we walk around,³ objects of interest are inspected, and it is only through observing changes in the ambient optical array that we can construct visual information into an understanding of the objects that comprise our physical (and by extension virtual) environment.⁴ In these terms, the act of looking (rather than simply seeing) is both active and generative.

The Mongolian national entry, Horizon – Sight Level (Sundui 2023), offered a presentation of culturally specific open-air nomadic performance space in an attempt to engage the spectator in an experiential encounter focussing on presence. Located within a literally shifting landscape, evoking the undulating terrain inhabited by nomadic theatre, the experience offered some framing in the real world during a short ‘onboarding’ with the technology, but the principal purpose of the real-world location was to provide a safe and comfortable place to sit. The experience itself then was primarily located VR and principally framed through editing. The recorded format meant that while audience members were able to look around, they were confined to a fixed point of perspective, limiting compositional opportunities. The audience was implicitly guided to a consideration of material that was not captured as archive/record, but rather authored in ways designed to facilitate an emotional response to the constant evolution of site and staging. The slightly unsettling experience of jump cuts in the fully immersive video heightened the sense of time and journey but also significantly decreased any perception of agency (or affordance), while expansive views of landscape and caravan that were well suited to ‘ambient’ viewing conveyed a strong sense of the scale relationship between the nomads and the vast land that they inhabit.

Eternity, from Estonia (Pōllu 2023), deployed multiple perspectives of archive/documentary material to explore notions of evanescence relating to Tartu New Theatre's open-air performance of Seraphima + Bogden. Framing here was superficially explicit but gained implicit/metaphorical significance the longer one spent with (and the deeper one ventured into) the exhibit. Management of this framing was located exclusively in the reality of the exhibition. The audience was invited to engage with different materials as archive of the performance event, starting with drone footage of the performance site on the exterior of the exhibit. An internal space contained a sound, video and physical installation depicting the slow recolonisation of the performance setting by its scrubland location (framed in the text as the main protagonist of the work), season by season. An immersive VR presentation of the same space in the moment of performance formed a coda to this increasingly metaphorical engagement with the work, a virtual world within a virtual world, framed as a re-presentation of place and performance – the whole existing both as record and companion. As with Horizon – Sight Level, this performance recording was in the form of a 360-degree fixed-perspective video, and while compositional opportunities were similarly limited, the inclusion of audience within the frame of the filmed performance event seemed to ground the spectator experience in ways that preserved a greater sense of agency and presence. This sense of presence was significantly enhanced at the moment of exit. Tomas and Glowacki have noted a phenomenon that they term ‘residue environment’, characterised by a sense of continued presence after leaving a VE that is experienced as an imprint on physical reality. They describe this hybrid experience as an ‘inter-subjective imagined realm’ (2018, 165). The experience of leaving the VR-mediated immersive video to find oneself still within a physical representation of that environment was both unsettling and intensely engaging. At the point of exit, the residue environment was intensified by the real, and the resulting inter-subjectivity produced an uncanny confusion in which it was, for a moment, impossible to separate real and virtual. Of the three exhibits, Eternity demonstrated the action of formation in the clearest terms, and the evolution of the space and the positioning of the viewer experience in both human and diurnal timeframes represented a strong invitation to dialogue with the work.

Rare Memories of the Future (Georgia, Gunia-Kuznetsova 2023) included almost no physical presence (a comment on the realities of lockdown) juxtaposed with an impressive immersive VR construct through which the audience was guided in real time. The VR experience was a spatial archive retrospective of the work of scenographer Miran Shvelditze (1947–2022). Archive material was located within the virtual construct of a theatre in which front of house, auditorium and backstage spaces contained re-presented artefacts from the scenographer's work as though in a curated exhibition. Participants were encouraged to sit, but I certainly felt compelled to stand (an option that offered more compositional opportunities) and motion was fixed within the VE. Moving through the world of the theatre, I returned to the central auditorium space, where I also encountered a video recording of the scenographer's work in performance, presented on the virtual stage and viewed from a variety of perspectives. While less sophisticated in the management of the spectator's transition from real to virtual space, the deployment of the spatial archive was an important one. Notions of affordance and participation were limited here to acts of visual composition, but the availability of the archive material to both ambient and ambulatory vision is significant in this context. Gibson argues that we are only able to establish a sense of the constant overriding characteristics of an object (which he terms its ‘formless invariant’) when in motion (Gibson 1973), and that this perception of object as a whole is entirely different to that delivered by static imagery, and represents an active constituent of knowledge. In this way, the deployment of spatial and kinetic techniques brought context and a greater sense of the materiality of the work of the scenographer than would have been available in other forms of digital archive.

Chris Salter has noted that

despite the fact that we mainly perceive capture as a purely extractive mode, new relationships forged between … human senses and sensing technologies also catalyze new imaginaries which produce new visions of entanglement and feedback among our senses and bodies, technologies, and the wider environments in which we live. (Salter 2022, 9–10)

All three of these projects shaped the presentation of captured materials in order to author a spectator experience that functions as artwork in its own right. I have already noted that notions of authorship are implicit in the presentation of virtual environments, but Popat (2016) and Thomas and Glowacki (2018) suggest that an explicit focus on spectator experience is also an inevitable consequence of the deployment of VR platforms. They suggest different ways in which mechanisms of ‘entanglement and feedback’ might energise virtual environments, offering the user modes of being more fully present. The position of real and virtual phenomena in a kind of contrapuntal doubling are central this debate. Popat notes that in VR, internal processes of proprioception mean that the (real) body of the spectator is felt but not seen while the (virtual) environment is seen but not felt, ‘folding physical and virtual together rather than experiencing a binary division’ (Popat 2016, 367). For Thomas and Glowacki, this is more than experiential, and they suggest that ‘VR offers an environment in which the user is required (through sensory rewiring) to be much more actively engaged in exactly where they are and what they are doing’ (Thomas and Glowacki 2018, 163), an aspect of the technology which was exploited in the second set of projects.

Projects of facilitation

At the other end of the production process, PQ23 also platformed projects and propositions with the potential to similarly re-author the experience of practitioners.

The iDesign platform previously presented in this journal (Del Favero, Thurow, and Wallen 2021) was presented in the context of its current development in association with Opera Australia. The platform enables creative and technical teams to work remotely or together in a virtual theatre environment⁵ that engages with interactive modalities throughout the design pipeline. The practical and financial advantages of such a space are obvious, but project lead Dennis Del Favero's concept of the space is more cognitive than practical, offering a model of ‘experience as a conduit for knowledge’ (Del Favero, Thurow, and Wallen 2021, 93). I have spoken elsewhere about the unique insights offered by digital modes of reconstruction as ‘re-enactment’ (Fergusson Baugh 2018), but in this virtual environment, production teams can engage in processes of ‘pre-enactment’ that Del Favero claims can facilitate an intensification of knowledge, a claim that is supported by Thomas and Glowacki's (2018) position on the perceptual demands of VR.

The ongoing development of this platform has seen significant improvements in both lighting and real-time rendering systems, but the funded collaboration with Opera Australia has also supported the addition of frame-based elements (including scoring functions that facilitate exploration and planning discussions around moments of performance) and actor optimisation that includes the addition of dancers to the platform in order to develop a ‘dialogical, cyber physical aesthetic’ (Thurow 2023). The team are also working on the development of functionality driven by machine learning to automate processes of checking sightlines.

In the Thai national entry, Theatre to Go (Thapparat 2023), AR was offered as both a technology of facilitation and a scenographic proposition. As a response to the urgencies of lockdown, it principally facilitated theatre in isolation, pursuing an agenda of democratisation and access (and won the award for ‘visionary scenographic strategy’ for this contribution) but also captured existing performances and developed new dramaturgies specific to the platform. For Tanatchaporn Kiitikoing (Khon Kaen University), the project offered ‘another kind of experience [and a] new kind of perception’ (Thapparat et al. 2023).

Theatre to Go explored the definition of theatre (a place for seeing), refocussing from notions of place to acts of seeing by relocating the theatre within the home of the spectator through the use of AR technologies. While this was developed in part as a response to lockdown, the project also represents an attempt to create a more stable funding environment for Thai artists who are not only subject to their own national funding shortfalls but also frequently denied access to international funding opportunities for which English language is a pre-requisite. The project adopted a literal and physical ‘on demand’ delivery model in which individuals could order delivery of a ‘take out’ containing a model box and stage props (and interval refreshments) which, when assembled, formed the anchor point for an AR performance viewed through the spectator's mobile phone or tablet. The team appropriated the communicative function of the model box between designer and director in order to form tangible performative moments with their distant, isolated and largely screen-reliant audience. For director Teerawat Mulvilai, the intention of the project was ‘not to duplicate the live performance but rather to design another kind of experience’ (Thapparat et al. 2023).

While radically different in delivery, there were notable similarities in intention between this project and Denmark's Home Box Theatre (Forchhammer and Lindberg 2023), which also sought to imagine a future where performing arts might take place not in public theatres but in private homes. Home Box Theatre acknowledged the creative contributions of XR but it also identified challenges to notions of liveness and community (in Fantová et al. 2023, 51), proposing an alternative, object-based (but technically mediated) solution.

The elephant in the room

The second of the Art and Digital Talks dealt specifically with the deployment of augmented and mixed realities in live performance. In the general discussion presenters and audience briefly touched on questions of definition, and in a room full of practitioners nearly all conceded that they had at some point needed to defend the status of their work as ‘theatre’. Acts of formal definition are not always helpful, but attempts to articulate the function of art are an essential element of the creative process, both as a means to establish principles that underpin evaluative judgements and as a guide for artistic development. In truth, there are as many definitions of art-function as there are critical theorists (from Aristotle to Zich), but my observation of XR performance at PQ23 (and sound performance at PQ19) suggests to me that the management of simultaneity (or synchronicity) is the key to understanding the action of technology-driven performance.

Art-function exists only in the encounter. It would be hard to find any commentator who suggests otherwise. Even before post-structuralism explicitly articulated the centrality of the reader, notions of reflection were implicit in many art theories. Shakespeare held a mirror up to nature, Strindberg presented a slice of life; both implicitly require a viewer who experiences resonance between life and fiction. Critically, the viewer must be able to understand themselves as the subject of the artwork. Their engagement cannot be tangential because understanding this resonance requires an ability to confront the ways in which the artwork is about them. The key, then, to shaping the art-function of performance (indeed, one might argue, any artwork) lies in establishing conditions under which the viewer resolves the confrontation of their own lived reality with the authored reality that is presented to them. This is a productive process and cannot be completed by a spectator who is not receptive to the task.

I am deliberately avoiding notions of active and passive spectatorship here as they do not address the nature of the activity the spectator must engage in and can be misleading. David Shearing's deployment of notions of mindfulness (in McKinney and Palmer 2020, but first presented at PQ15 Talks) is useful in this context. For Shearing, it is a reflective awareness of body, mind and environment that facilitates embodied engagement and agency (in McKinney and Palmer 2020, 147). In this relational encounter, reflection becomes a particular mode of participation (139–140), but one might argue that reflection is in fact an essential characteristic of the art-function in all contexts. No artwork can force the spectator into this state, but there are formations that offer relative advantage or disadvantage in completing the art-function of performance.

By exploiting tensions between the visual and the kinetic (VR) or pass-through and authored sound and video (AR), XR technologies offer artists modes of framing that explicitly place spectators in a constant sense of experiential ambiguity, that may be heightened or mitigated through the development of appropriate dramaturgies. In his foreword to Chris Salters’ work on Entanglement (Salter 2010), the director Peter Sellars argues that this ambiguity is not unique to the content of virtual representations in performance, but is characteristic of the broader deployment of technology as interface, delivering ‘the potential to fragment and diversify the master narrative. Offering simultaneous, multiple perspectives, freshly negotiated independent narratives and the direct experience of ambiguity’ (Sellars in Salter 2010, x). This ambiguity represents an implicit requirement for external reflection that locates the audience within the formation rather than as an unmandated observer.

Concerns expressed by the Danish team about the role and function of XR in live performance are inevitably an important part of any debate on theatre, but while I spent many happy hours in Home Box Theatre, in Theatre to Go I felt the invitation to participate in that debate in more tangible ways. This is perhaps an indication that Thomas and Glowacki's observations on the role of ‘sensory rewiring’ in the demands on the user extend beyond VR and can also be used as a tool for understanding the action of AR. In VR, they have noted that ‘the physical environment remains present but as a sort of background to a potentially more dominant virtual visual experience’ (2018, 151). Lived reality persists, but the authored (virtual) reality risks taking on greater significance. AR technologies offer the artist opportunities to closely manage the extent to which lived reality remains present.

In Shearing's model, we can argue that the key to facilitating mindful/engaged reflection lies in maintaining a meaningful balance between the realities (lived and authored) experienced simultaneously by the spectator, and when framed in this way, we can see that the art-function of performance tends to break down in formations that privilege one reality over the other. We may not always use that language, but notions of passive spectatorship, the pejorative use of the word ‘entertainment’, and dissatisfaction with star vehicles and mega-musicals all address the same fears for the potential for drama (stories) to defeat the art-function of performance. Strindberg launched the Naturalist project with an excoriating attack on the action of audience immersion/submission in what was effectively late melodrama. Brecht's subsequent rejection of Naturalism was presented in almost exactly the same terms. This should serve as an important reminder that the management of balance in this confrontation of realities in performance is both subjective and dynamic. It is the responsibility of the artist to monitor the productivity of the art-function and respond accordingly (this is, of course, why definitions founded in function are more useful than those founded in form).

Conversely, formations that privilege the spectator's lived reality can be equally disruptive. While they do certainly engage an ‘active’ spectator, certain forms of immersive/interactive performance, or dramaturgies that exist at a nexus with gaming practice, can also render the spectator unproductive. If not carefully managed it is possible for immersion (deployed to offer the spectator a fully subjective experience) to actively objectify the audience within the scope of the performance, offering the possibility of simply engaging with the event only as a lived reality and, once again, causing the art-function to break down. Strategies for the mitigation of this possibility have been offered in Shearing's model of mindfulness, and Matt Adams (Blast Theory) places an emphasis on the importance of framing in interactive work – he asserts that the critical work for the artist lies in the development of a ‘meaningful point of landing for the interaction’ (Adams 2021), and that this may ultimately represent the bulk of the audience experience.

In truth, the processes by which this balance breaks down are greatly accelerated in dramaturgies that are founded on the use of mediatised technologies, as these issues are compounded by the existence of a fundamental paradox (acknowledged only in part by Home Box Theatre) … live art must in some way reflect the human condition; it is not possible to do this without acknowledging the pervasiveness of mediatised technologies; the deployment of mediatised technologies to some extent defeats liveness. Brecht gave Naturalism 30 years, but the digital artist may find a resolution to the paradox today that might not still function tomorrow. One only needs to look at the many instances of arresting, original art formations that are first normalised and then commodified through processes linked to ‘Got Talent’ shows, Eurovision and commercial advertising to see that digital artists/performers need to be engaged in a constant cycle of evaluation and revision to avoid the collapse of meaningful engagement with their work. It's a bit like plate spinning.

It is of course in this context that we can see dramaturgical advantages offered by the deployment of AR and MR technologies in performance.

MR/AR dramaturgical formations

Elon Musk's Grand Museum of Consciousness, a Piece for Mixed Reality,⁶ and REMAP's⁷ production A Most Favoured Nation both developed dramaturgies that sought to exploit opportunities offered by augmented and mixed reality technologies to present digitally authored narrative elements in a live performance environment.

In Elon Musk's Grand Museum of Consciousness, dramaturgs Carlos Costa, Jorge Palinhos and Miguel Mira aimed to explore the apparent redundancy of using immersive VR in live performance.⁸ Taking conceptual inspiration from work undertaken by the Dutch probation service on the use of VR in communicating victim impact (CEP 2017),⁹ the production was set in the context of a fictional uploaded version of the consciousness of a comatose Musk. Building on the notion that direct experience strengthens understanding, the team explored the proposition that the virtual experience presented would inflect the audience's understanding of and sympathy towards Musk's Objectivist philosophy. In order to achieve this, the team deployed a three-phase dramaturgy comprising a narrative ‘onboarding’ into the technology, a VR experience depicting scenes from Musk's life (deployed as a fixed point of view, 360-degree video in which the audience had only compositional agency and presence), and an interactive IRL¹⁰ section in which the audience were invited to re-enact those scenes. For Palinhos the importance of the experiment lay in evaluating the relative impact of the virtual experience of witnessing and the embodied experience of re-enacting ideas that were alien to most of the audience. Feedback indicated that it was the virtual representation that had the greatest impact on understanding (though sympathy remained limited) but that, critically, this impact seemed to have an inverse relationship with the individual's previous experience with VR.

Popat has noted the significance of the location of the VR screen at the limits of our physical experience. For her, this constitutes a reconfiguration of sensory experience that directly impacts our sense of embodiment. She frames the screen as a Husserlian nullpoint, a ‘here’ from which all ‘theres’ are measured, and notes that while the implied dualism of subject/object might not be helpful in a general understanding of embodiment, it is a useful tool for exploring the relationship between human and avatar (2016, 363). The notion of ‘nullpoint’ is also echoed in Michael Polanyi's exploration of proximal and distal phenomena in relation to tacit knowledge (Polanyi 1967). Polanyi's exploration of tool use describes the phenomenon of ‘indwelling’ as a process by which growing familiarity with tools experienced at the edge of sensory perception gradually shifts them from the distal (explicit) sphere to the proximal (tacit) sphere, until they become extensions of our physical/proprioceptive experience. This explains the process by which control interfaces become invisible to experienced gamers, but it is also interesting to consider in this context. Popat notes ways in which presence in VEs is achieved through a process in which an alien virtual ‘not me’ slowly gives way to a more familiar ‘not not me’ (2016, 364). Palinhos’ experience here suggests that in this case, the explicit (rather than tacit) unease of the ‘not me’ experienced by the novice VR users has played a significant role in the impact of this performance.

A Most Favoured Nation was developed between 2019 and 2022 and set in the world of the Amazon Prime series Man in the High Castle (adapted from Philip K. Dick's novel) and deployed real-time augmentation, simultaneously experienced by the audience. In this production, the handheld, AR device was rendered as a ‘window’ into the non-present narrative. The impact of this conceit will be familiar to anyone who engaged with Sharon Rashef's Remember Me (Site Specific Performance PQ19) or Jan Mocek's Play Underground (presented alongside PQ15), both of which engineered audience point of view to match recorded audio or video replayed through a mobile device. The critical difference here is that deployment of AR in A Most Favoured Nation was both genuinely interactive, requiring the audience to frame their own stage view in extended theatrical space, and diegetic. For the development team, this was an essential element of the dramaturgy.

For creator Jeff Burke, it is important that theatre adapts and appropriates the things that challenge us and our humanity, and that while the deployment of AR is in this context a dramaturgical intervention, it is also a way of interrogating our engagement with technology in a consumer context. For Director Mira Winick, the technology presented metaphysical opportunities for audiences to engage in ‘the act of seeing and unseeing … existing and not existing … to be seen and not seen’ (Burke 2023).

These two projects both found ways of making the balance/confrontation of lived and authored realities an explicit focus of the dramaturgies they developed. Whether they approached this sequentially (Elon Musk's Grand Museum of Consciousness) or simultaneously (A Most Favoured Nation), this confrontation was managed by a direct invitation to the audience to engage in acts of composition (through either re-framing or re-enactment) that activated Sellars’ notion of ambiguity (Sellars in Salter 2010, x), precisely by casting them with an external view but locating them within the formation.

Palinhos’ (2023) observation that the effectiveness of this formation apparently diminished with familiarity illustrates the fragile nature of this balance and the demands for conceptual agility placed on the digital artist.

The question of immersion

Salter has noted that the human desire to have our existence validated through mechanical capture and representation is centuries old (Salter 2022, 9) and thus significantly predates the reality of current technology, but in its current manifestation, the impact of our digital existence has profound implications for theatre and performance.

In his editorial ‘Virtual Reality Has a Storytelling Problem and Theater Will Save It’, James Ferguson argued that VR struggles with the presentation of linear narrative because of the compositional freedom afforded to the user, and that the kind of scenographic direction found in theatre might offer solutions (Ferguson 2016). Popat (2016) suggests that the relationship between the two forms can be productively reciprocal. She acknowledges that VR is essentially an experiential rather than a narrative platform, but argues that this emphasis on experience and presence in VR can enable participants to relocate themselves as embodied beings in ways that offer theatre makers opportunities for enhancing a present focus on live and narrative performance elements. In truth, though, theatre also has a growing storytelling problem from which it must save itself. We are increasingly becoming enculturated to expect that dramatic fiction should function as escapism (a notion that explicitly privileges authored realities). On stage, this is clear in the rise of ‘franchise’ theatre that re-stages big-screen hits, but theatre aside, our changing attitude to dramatised fiction is most evident in our consumption of media, and the practice of binge watching is symptomatic of this.¹¹ Binge watching evidences a desire in viewers for complete immersion in the world on offer, but there is compelling evidence to suggest that the immersion sought is one of oblivion, in which the viewer seeks to lose themselves rather that find themselves. A survey of resent research (Panda and Pandey 2017) indicated that while a perception of social belonging was a factor in binge behaviour, the avoidance of negative life experience (escapism) was a greater driver. The immersion may be consuming but it can only be transitory and is likely to be unproductive. Notions of avoidance in our consumption of narrative are further complicated by growing cultural tendencies towards simultaneous consumption of different media in ways that overlay a constant virtual mobile presence over real-world experience.

These phenomena represent an increasing challenge to the presentation of narrative performance, partly because one might argue that the confrontation of negative life experience is an essential part of the art-function of theatre, but mostly because without close attention to the management of the immersion, it tends to place the audience in an unreceptive/unproductive state from the outset. Popat's (2016) proposal that VR technologies might offer theatre modes of ‘productive reciprocity’ in the development of narrative offers further insight when considered in respect of augmented and mixed realities.

Digital utopias?

Mira Winick's observation that VR/AR technologies represent opportunities for ‘seeing and unseeing’ was one that was echoed by many XR practitioners at PQ23. Jeff Burke, Jorge Palinhos and Tung-Yen Chou all independently asserted the aim not to develop narratives that XR could do well but to explore themes that could only be addressed in XR. Audiences at the Art Talks agreed that this was important and, when challenged, proposed that XR offered unique opportunities to resist normative expectations, and that XR performance was most productively focussed on issues relating to the historically marginalised and dispossessed, citing issues of gender identity, mental health and indigenous culture as examples. One might argue that an increased sensitivity to historical marginalisation and its attendant privilege is another aspect of the conative shift of the last decade.¹² In terms of the mechanisms by which meaning is constructed, it is particularly significant that the work presented at PQ23 that most directly addressed this kind of thematic material used only VR models of presentation.

The Canadian VR/passthrough experience in the Performance Space exhibition, This space is for you/cet espace est pour toi (Janssen 2023), offered a series of critical and theoretical propositions in the presentation of their work. Framing VR as a space of regeneration, Janssen evoked Anna Tsing's notion of ‘refugia’ (in Haraway 2015), Sarah Ahmed's notions of the queer orientation of space (Ahmed 2006), and concepts of pluriversal worlding (Escobar 2020) and un-worlding (Halberstam 2022) in order to focus the spectator/participant on the ‘possibilities that MR technologies have to interrupt socially normative relationships between bodies and environments’ (in Fantová et al. 2023, 295).

It is possible to explore the resonance of immersive VR with non-normative (or, indeed, anti-normative) modes in the context of its evolution from broadly open-source platforms such as VRML¹³ and Second Life – platforms frequently driven by communities of cultural outliers and natural disruptors – but the critical positions explored here offer alternative, and potentially more revealing, interpretations.

A site-responsive piece set in and featuring the top floor of the National Gallery, This space is for you/cet espace est pour toi placed spectator/participants in a virtual, volumetrically captured model of the performance venue that they traversed accompanied by a virtual ‘dancer’, humanoid in form but not otherwise identifiable. The space was populated with other interventions (a sphere, flocking birds) and occasionally disrupted by pass-through video of the audience's actual view of the same space. The spectator/participant could orient themselves and – within the limits of the small performance area – move within the virtual world, but the overall motion of the piece was dictated by the simulation, which guided them around the hall before ‘flying’ across the open atrium of the building to return to the starting point.

Thomas and Glowacki cite Artaud's deployment of the term ‘virtual’ in the context of a theatre in which notions of the ‘virtual’ act as a bridge between ‘that which does and does not exist’ (Artaud and Corti 2010), but Artaud's ‘virtual’ does not align with modern usage; it does not reference notions of simulation or substitution but rather notions of virtue. Artaud's ‘virtual’ evokes an implicit ideality that may only exist as possibility, and that is set in opposition to a material reality that is by extension flawed in nature.

Escobar (2020) and Elizabeth Grosz (2001) both attempt to problemetise notions of possibility from a standpoint of political marginalisation. Both note that tangible possibility only exists in what can be imagined and is therefore both grounded in, and limited by, a broadly hegemonic experience of material reality. For Escobar this means that the individual must learn to understand not only other possibles but also other (‘pluriversal’) possible possibles that exist beyond our recognised personal and political horizons (2020, ix). For Grosz the consequence is that notions of ‘possibility’ themselves are essentially heteronormative, and she adopts the term ‘virtual’ in a sense that is much closer to Artaud's to explore ‘possibilities’ that are liberated from the limitations of experience.

Ahmed's (2006) work explores notions of orientation in our phenomenological engagement with the world. She notes that notions of perception (and, by extension, possibility) tend to be driven by our orientation towards them. To have one's perception challenged is to become disoriented. She cites Frantz Fanon in noting that as a result disorientation is more characteristic of the marginalised body, and that ‘some bodies more than others have their involvement in the world called into crisis’ (2006, 133) by hegemonic, normative notions of possibility. This phenomenological reading perhaps proposes an alternative explanation of the value of XR in the resistance of normative expectation … that it is a function of the disorientating tendency of VR rather than an expression of the values of historic developers.

This space is for you/cet espace est pour toi then explores the possibility that technologies of capture without agenda driven by identity are uninflected (or phenomenologically ‘dis-oriented’), and as such they can facilitate queer readings of spatial relationships. As with Elon Musk, the work draws on the suggestion that experience acts as an intensifier of knowledge. The process of ‘onboarding’ into a virtual environment invites the user to consciously engage with a different form of experience in which they must first divest themselves of familiar modes of seeing and being in order to reinvest in a different kind of embodiment (a process of dis/embodiment, perhaps) that offers opportunities to reject the normative tendencies of the everyday ‘possible’. Popat suggests that these opportunities may be grounded in the absence of tactile perception of the virtual environment. She notes that an absence of physical consequence, particularly in respect of other avatars, tends to suspend social prohibition, partially disrupting the social construction of self in relation to others (2016, 373). For Thomas and Glowacki, there is a more fundamental perceptual change in which VR offers expanded modalities of seeing which decentralise vision and reconfigure seeing as a visual/proprioceptive act. In this way, they claim that ‘seeing no longer belongs solely to the eyes and therefore emerges as a less objectifying and more a felt and empathetic sense’ (2018, 146).

In This space is for you/cet espace est pour toi, the presentation of the twice dis/embodied body (the dancer and the participant each dis/embodied in a different sense) in this doubled space (virtual and pass-through) resists any fixed reading and asserts the plurality of our experience through the figurative and literal orientation of the participant. Ahmed's essential notion of disorientation is strongly related to Sellars’ notion of ambiguity in its capacity to fracture and reposition ‘uncertain boundaries’ (Sellars in Salter 2010, x). The work as a whole evokes the suggestion that virtual spaces might represent represent political ‘reugia’ from which the reorientation towards pluriversal possibility might resist (and ultimately reconstitute) humanity's more destructive tendencies.

Director Tung-Yen Chou's work In the Mist (presented in the first of the Art Talks) also featured a ‘queering’ of virtual space. This piece offered a 360-degree filmed, first-person visual experience of men's sauna encounters that focussed on capturing the sense of ‘un-presence’ that Chou experienced on his first visit. Exploring notions of intimacy and loneliness, the audience experience was managed (again) by an act of doubling in which the VR interface was located within the setting on which it was filmed. Chou explained that for him immersive VR work must be ‘visceral’, and it must bring a sense of live performance to the recorded medium; he likened it to a space between dream and memory. In this context, the work did indeed take on a distinctly dream-like quality, and the content (which was undeniably explicit) became a backdrop to an exploration of moments of connection. Moving between moments of anonymity and intimacy, the work makes considerable use of eye contact which (in this recorded context) takes on additional significance. The eye contact that is made with the performers (who remain, like the spectator, disconnected from the action) becomes part of the performance contract. The eye contact is unsettling, but the content of the work also makes looking away voyeuristic and uncomfortable. The encounter with the eyes of the disconnected performer then becomes the more comfortable and most emotional aspect of a performance in which themes of loneliness and connections are beautifully realised by a framing that reveals its overt theatricality as the set is deconstructed in the VR experience, leaving the spectator alone in a space that both does (physically) and does not (virtually) exist. For Chou, VR presents the spectator with a tailor-made theatre of their own, and he asks ‘what if everything is possible?’ (Chou 2023).

Conclusions

While broadly positive, I was on occasion reminded that the possibilities offered by XR are not without cost. Home Box Theatre's framing of XR's challenge to key concepts in the performance contained a warning about the disruptive possibilities of creating a ‘separation between the experience and the imagination’ (in Fantová et al. 2023, 51) – though Jeff Burke was keen to remind us that the fact that AR technology has not yet reached its stable state makes definitive judgements impossible. Questions of access continue to ensure that while digital modes of being (or refugia) are resistant to the marginalising tendencies of normative expectation, they are not immune, and representation in those potential refugia tends to favour the Global North. It was also evident in discussions relating to the presentation of marginalised voices that the language used was often one of absence (ghosts, shadows, reflections, etc.) that potentially indicated an implicit (or even invisible) tendency for virtuality to decentre these themes in the real world.

Overall, though, the work on show at PQ23 offered a range of approaches, paradigms and exemplars for scenographic/dramaturgical consideration. What was very clear was the fact that in order to function as art, scenographies/dramaturgies founded in XR are particularly subject to the ways in which audience experience is framed. Work of this kind cannot be presented in a vacuum because the context and the spectator are essential elements of the formation.

Much of the work at PQ23 drew (implicitly or explicitly) on assertions that embodied experience offers different and perhaps more effective modes of understanding, and that in virtualised environments, this kind of productive dis/embodiment can be propagated by explicit invitations to engage in acts of composition that locate the spectator within the formation. In this way, XR performance can occupy a similar space and function to that held by the Avant Garde of fine art in the 1960s, where the value of art was explicitly focused on the moment of encounter rather than any suggestion of intrinsic value (which ultimately acts to commodify the artwork, facilitating the collapse of meaningful engagement). The demands placed on the spectator mean that XR can (ironically) deliver an ‘aura’ (Banjamin and Underwood 2008) of authenticity to media that we might otherwise consider fixed and closed through the invitation to acts of constant re-composition and through an internal focus prompted by ‘sensory re-wiring’.

Processes of phenomenological reorientation lie at the heart of this. Indeed, for Shearing (in McKinney and Palmer 2020, 147), this is the case with all performance, as the disruption (or unlearning) of normative modes of cognition and participation is an essential element of immersive scenographic practice. In the case of XR (and particularly VR), these processes of phenomenological reorientation are an intrinsic part of an experience that naturally tends towards notions of possibility that are both pluriversal and anti-normative.

In his discussion of his development as an artist, Tung-Yen Chou cited Notes on Blindness (2016) as a landmark work. This work was developed from the audio diaries of Australian academic Professor John Hull and guides the participant through Hull's perceptual shift from sight to hearing. Hull's enduring message at the end of the work is that when removed from our immediate experience, phenomena ‘lose existential meaning … and ultimately don't matter’ (John Hull in Lartigue, Togman, and Desjardins 2016). The many presentations available at PQ23 demonstrated the ways in which XR technologies can be used to develop experiential understanding of alternative propositions that can ensure that audiences can engage with pluriversal possibilities in meaningful and productive ways.

Additional information

Notes on contributors

Cat Fergusson Baugh

Cat Fergusson Baugh teaches applications of digital technologies in theatre and performance at the University of Hull. She has worked in the area of digital reconstruction of extant and historic theatrical spaces since 1995 and has had work exhibited at the Victoria and Albert Museum and the Columbus Museum of Art. She has moderated sound, technology and XR panels at PQ19 and PQ23.

Notes

1 In a landscape that increasingly blurs the two terms, my use of 'VR' and 'AR' in this article primarily relates to mode of delivery. Where I have used 'VR', this refers to the use of head-mounted displays (HMDs) as user interface, with or without augmentation using passthrough capabilities. 'AR', here, exclusively refers to the deployment of augmentation of pass-through video on mobile devices.

2 As Thomas and Glowacki (2018) note, the term 'reality' in this context is neither accurate nor helpful, and while it may (by custom and practice) be used in the description of technologies, the term 'virtual environment' (VE) is more productive in the discussion of content. It also establishes the centrality of authorship within the experience.

3 Gibson (1986) identifies these conditions as discrete forms of vision which he terms ‘ambient’ and ‘ambulatory’.

4 This is a view supported by Sanchez-Vives and Slater in their work ‘From Presence to Consciousness’ (2005).

5 iDesign has been planned for deployment in a number of environments, from mobile device through immersive VR to large-scale projection and cave presentation.

6 A co-production between Visões Úteis and Teatro Académico de Gil Vicente (and supported by the University of Coimbra).

7 UCLA's Centre for Research in Engineering, Media and Performance.

8 Palinhos claims that VR is in itself a form of technically supported theatre (Palinhos 2023).

9 In a pilot programme, VR has been used to leverage direct (virtual) experience of the aftermath of domestic violence to enhance offender understanding of the impact of their crimes.

10 In real life.

11 Albeit a shift that has to be set against a backdrop of late capitalism and authoritarian populism that has tended to frame these issues as 'snowflake' concerns.

12 Commercially available on most VR platforms.

13 Virtual Reality Modelling Language.