The Mental Abuse Matters Virtual Reality Project: creative practice in cinematic VR and immersive sound

ABSTRACT

This article discusses my practice work The Mental Abuse Matters VR Project; a live action narrative drama and Virtual Reality experience where the user is an embodied victim, experiencing an episode of emotional abuse in a domestic setting. Immersive sound is used to replicate emotions in the body such as shame, dread, anxiety, humiliation and panic in the user and to represent inner thoughts, memories and states of dissociation, with the goal of moving beyond language towards a visceral narrative. This is a prototype intervention that aims to be used as an empathy training tool for frontline Health and Social Care and Justice sector staff to improve trauma care, and potentially a therapeutic VR experience for victims and perpetrators of mental abuse. The article aims to discuss the development, creative practice and planned dissemination of this pilot VR experience, evaluating it in the context of traditional study methods and existing health and social care interventions in VR.

KEYWORDS:

• Cinematic virtual reality
• mental abuse
• trauma
• embodiment
• visceral narrative drama

Introduction

In this article I discuss the Mental Abuse Matters VR Project (MAMVRP) (Baxter 2021) and the core aims, objectives, methodology and practice process of this work. The core aims of the piece are to explore and interrogate representation of emotionally abusive behaviour in the perpetrator, and the physiological and psychological responses to this behaviour in the victim, within a heterosexual intimate relationship using live action Cinematic Virtual Reality (CVR), positional sound and real actors in a narrative drama; and to gauge audience reception of it as a potential health and social care intervention (Baumann et al. 2019) or training tool. I argue that the methodology of auto ethnography (Ellis, Adams, and Bochner 2011), as used in this case, allows for deep exploration of themes around emotional abuse and is effective in crafting a story that aims to elicit empathy in the viewer – and discuss the process in relation to other examples of auto ethnography. In the VR film, the victim in an emotionally abusive relationship is a woman. The reason for choosing a female in a heterosexual relationship was due to the auto ethnographic nature of this research. The experience is 9.5 min in duration and is a narrative drama where the user is embodied (Kilteni, Groten, and Slater 2012) as the victim in an emotionally abusive intimate relationship. It takes place over the course of one evening in the lives of this couple and tells the story not only through plot and dialogue, but through immersive binaural sound which aims to replicate felt emotions in the body when certain triggers arise. These triggers involve non-verbal signifiers such as body language and facial expression and internal states of memory, dissociation and intrusive thought, in addition to dialogue between characters. The article also discusses the development and production process involved in creating this 360-degree world, the background of its intended use as an intervention in health and social care and the beginning stages of an experimental study with undergraduate social work students at Queen’s University Belfast. Research questions in the creation of this work centre around the translation of inner experience into an immersive environment, both visually and aurally, and the potential application of immersive narratives in social care.

Background

The MAMVRP uses embodiment with the aim of eliciting empathy for the experience of someone who is living with gaslighting, control, humiliation and passive-aggressive behaviour from their partner (Engel 2002). These forms of behaviour, while corrosive to mental and physical wellbeing and sense of self in the victim, are not obvious to the outside world. Indeed, often the person experiencing them does not understand what is happening or the physiological effects that result from emotional dysregulation and fight, flight or freeze responses in the nervous system – states that recur over time in an emotionally abusive relationship where a couple is living together, for example. Embodiment is defined as the sense of having a body, and the body can be considered to be both the subject and object of medical science and practice (Matamala-Gomez et al. 2019). When considering the creation of an embodied Virtual Reality (VR) piece that seeks to represent the internal experience of emotional abuse, a key question emerged: How can a complex inner state so turbulent and nebulous that it often remains a mystery to the person experiencing it be translated into a visual medium like CVR? There remain knowledge gaps in this area – including few practical interventions specific to mental abuse-related trauma, the limited, though increasing, use of live action VR for interventions across health and social care and the dearth of film/live action using real actors in the immersive intervention space. Although live action is being used more frequently, it is still relatively rare in VR compared to Computer-Generated (CGI) worlds. This practice work explores how creative technology can push the boundaries of both film and social work, and the points at which these two disciplines intersect; and whether technologies that might seem impersonal or disassociated from the human condition, a means of ‘escape’, could in fact help us to understand it better, relieve suffering and improve empathy.

The MAMVRP evolved as a practice-based research project from an animated film that I created as part of a portfolio titled Mental Abuse Matters (Baxter 2021). MAMVRP is iterative creative practice in dialogue with a growing body of work that uses creative technology such as VR film or animation for therapeutic and/or mental health training and therapeutics experiences, and academic discussions in the field of creative immersive technology. The interventions I will discuss concentrate on the use of VR for training in empathy and rehabilitation in the domestic and sexual violence space, in both live action video and CGI. While my previous film was based on anonymous testimony of a friend, the MAMVRP brought auto ethnography into my practice, as it was based on my own story.

From my initial 2D animation practice, which used a painterly animation style, came the desire to explore a visceral narrative of the body beyond language and visuals, through the medium of 1st person embodied (Kilteni, Groten, and Slater 2012) CVR where the user of the CVR headset steps into the shoes of a character in a drama narrative based around an emotionally abusive heterosexual relationship. This narrative has similar aspects to the animated narrative in terms of behaviours and emotions, but is a different story, centring around one evening in the life of a couple whose friends come for dinner. The film is 9.5 min long and covers one evening in the lives of this couple. It begins with the woman, Lisa, alone in her bedroom getting ready for a dinner party with her partner and friends which will take place in her house, beginning with an internal dialogue of things her partner has said to her as she chooses a dress and puts on makeup. The partner Gabriel then arrives home, and there is a brief exchange between them before the friends arrive. Listening to the hubbub from downstairs, the woman reluctantly descends the stairs and joins her partner and two friends at the dinner table. Throughout the evening, the other three characters converse with each other and occasionally with her. As time passes at the table the user experiences emotions such as alienation, shame and panic – triggered by things that are said, memories triggered or facial expressions from those around her. The film ends with the friends saying goodbye and a brief conflict interaction between the two partners when they have left. The film exits into a calm field of yellow flowers.

As with the creative choices around the animation which used a painterly style rather than CGI in an attempt to represent disordered and nebulous internal states, Live Action VR was chosen rather than CGI VR worlds, because the use of real actors allows for subtle behaviours such as passive aggression, gaslighting, humiliation and veiled threat to come across in a realistic way through body language, facial expression and tone from a real actor. This is, arguably, still difficult to achieve in the same way through CGI (Ding, Zhou, and Fung 2018). The idea that a filmic medium could place a user inside the mind and body of a character seemed an intriguing and potentially powerful way to convey emotion and aid empathy and understanding in the viewer, exploring internal emotional states, inner voice and subtle behaviours of a perpetrator as well as escalating states of panic and shame in a victim.

VR as a health and training intervention in domestic and sexual violence

Many existing VR research projects informed my practice. In the sphere of domestic and sexual violence, the former being where the narrative of the MAMVRP is located, there are some important precedents for using VR technology with victims and perpetrators of physical violence and sexual assault, using animated worlds. The first is led by Psychologist Dr Claudie Loranger in the Cyberpsychology Lab at the University of Quebec. Dr Loranger and her team of psychologists has created what they call the first VR environment designed exclusively to help survivors of sexual assault. The VR experience includes virtual environments – a bar, which leads outside to a bus stop – to help ease patients into the process. These environments are designed to be progressed through over several sessions, so that users can become used to the process and memories can be triggered in a piecemeal, manageable way. The final step is a VR version of a sexual assault, which cuts to black and is audio only. The approach is based on the exposure therapy methodology, where those who’ve experienced trauma or people with phobias learn to cope by reliving the scarring incident or exposing themselves to what they fear with 3rd person avatars. While the MAMVRP work is being trialled as an empathy training intervention within social work in the first instance, and not as exposure therapy, I am interested in its potential as an exposure intervention in the future and this work was an inspiration in the development of the piece. I was interested in taking the scenario one step further, by allowing the user to be embodied and ‘feel’ the sensations of the victim. Future research might involve investigating the efficacy of the piece in an exposure therapy context.

Another important project in the embodied VR space is led by Mavi Sanchez Vives at the University of Barcelona, with perpetrators of physical domestic violence. In the study, researchers from the University of Barcelona showed that, often, violent people have a lack of emotional recognition, but that a virtual experience measurably improves their empathy. ‘These illusions have an impact on the participant by altering perceptions, attitudes and behaviour’ (Best 2018). In this project, researchers analysed the impact of immersive virtual reality on 20 abusers and 19 control participants. The VR experience involved perpetrators being embodied in the female experiencing the violence, in other words inducing full body ownership illusions, so they could understand the fear and trauma such behaviour instils. This is similar in intention to my own work in mental abuse, but again this piece uses an animated rather than live action environment. Working with perpetrators to gauge empathy in a rehabilitative space is also a future ambition with the MAMVRP.

As part of a pilot study in the University of Aberystwyth (Aberystwyth University Archive 2022), Dr Helen Miles and Andra Jones from the Department of Computer Science, who specialise in virtual environments and virtual reality, collaborated with Rebecca Zerk, Sarah Wydall and Elize Freeman from the Centre for Age, Gender and Social Justice to create a VR training tool to allow users to see things from the perspective of an older woman making a disclosure of domestic violence and abuse. Participants take part in an immersive experience depicting the initial police response at the home of the victim, and can choose different options for next steps around safeguarding as part of the experience – there are 8 possible different pathways and possible outcomes. The aim was to gauge whether it increased empathy and sensitivity to victim-survivors.

In the area of embodiment and empathy training, London-based organisation Bodyswaps create soft skills training modules for the corporate and health sector, for example around anger management, empathic interaction with dementia patients and racism. SAGE Publishing is partnering with Bodyswaps, an award-winning virtual reality (VR) platform, to launch a cutting-edge resource for health and social care training. The Mental Health Practitioner VR experience supports students in developing soft skills through an immersive virtual environment, representing a first step into a new generation of learning. I had the opportunity to view some of these modules first hand, as Bodyswaps ran a trial into the education sector, supported by Meta. Bodyswaps use avatars directly addressing the camera and speaking to each other to outline what the content is about, and then the training is delivered via CGI animated scenarios using avatars and multiple choice questionnaires as you move through the training. On first-hand experience, I found the avatars unsatisfactory and distracting in terms of facial expression (which was extremely limited) and body language though the scripting was of a high standard and the branching narratives were well laid out. As trials into education are at an early stage, there is a dearth of academic literature reviewing Bodyswaps at present.

Methodology

Having created an animated film that told someone else’s story, I decided to take a leap of faith and create a film that told my own. This was not an easy decision, for complex personal reasons. As an early career researcher with an industry background, it took time to recognise that auto ethnography is a recognised methodology, and it became the primary one employed in the creation of this piece. It took many redrafts, however, for me to find the courage and confidence to express it in writing. In many ways, I am still finding my voice as an auto ethnographic researcher and understanding that this is an authentic and validated mode of research within the academic literature. As Anne Byrne (2000, 247–248) observes:

there are some research projects, born, as Liz Stanley (2016) would say, out of ‘necessity’ which carry within them transformative possibilities for the researcher … (when) the myth of the scientific observer must be done away with. Instead we have an ‘experiencing because knowing subject’ whose ontologically based reasoning provides the claims to knowledge’ (Stanley 1996, 47) … very often stories, or personal narratives, more so than positivistic research methods, are the only way of closing the ‘memory gap’ (Grunfield 1995; Ringelheim 1997) between traumatic experiences and the words available to us to tell them.

The writings of feminist auto ethnographers Liz Stanley, Anne Byrne as above and Amani Hamdan (2012) have been particularly influential. Speaking narratives from the inside out has not always been a sanctioned research methodology in an academic context but is becoming more so with the increasing amplification of female voices, particularly.

In this ethnographic context, the key motivator for creating a visceral narrative in the VR film was the experience of trying to articulate my own story of emotional abuse; particularly in institutional settings such as hospital and family court. In these situations, the gap between lived experience and language had never felt wider. In fact, in conversations with social workers, midwives and hospital psychologists it often felt unbreachable. This was a disorientating and demoralising experience as I had always been articulate in my adult life and it led me to wonder how such scenarios are for less articulate women – those traumatised by experiences, frozen with fear, or with language barriers. A conversation with a family court social worker that ultimately led to my experiences being invalidated in the context of emotional abuse was the inspiration for exploring empathy training in social work. This social worker had, as her sole diagnostic tool, a piece of paper depicting The Duluth Model Power and Control Wheel, created in 1984 (Pence and Paymer 1993) and I was powerfully struck at the time, and subsequently, by the abstract and inadequate nature of this as a resource being used by frontline workers 32 years later. While undoubtedly an important milestone in the understanding of domestic abuse, and widely used in an institutional context, the Power and Control Wheel is, arguably, a limited resource when considering other potential training methods encompassing lived experience testimony, or empathy, for example (Figure 1).

Figure 1. Control Wheel.

Drawing of a wheel cut into sections with text describing abusive behaviours.

While I had clearly articulated in conversation with the social worker behaviours that would be classed within the wheel as using intimidation, using emotional abuse, minimising denying and blaming, using children and using isolation, and she had agreed that this was the case, these elements were dismissed in the final report. When reflecting on this scenario I used the felt experience of attempting to articulate the effect that a look, a gesture, an apparently throwaway comment, or a repeated ‘funny’ story could have on a person whose identity and confidence has been systematically eroded by emotionally abusive behaviour to inform the script writing process, which is described in more detail below.

Another powerful aspect of auto ethnographic research was connecting my own experiences in therapy to a larger body of work. Identifying bodily responses in real time and locating where trauma in the body has been ‘held’ are therapeutic techniques used in Somatic Therapy and Gestalt Therapy (How Gestalt Therapy Works to Heal Past Trauma) as a way of processing traumatic experiences. Gestalt body work was the most effective means of tapping into and releasing trauma in my own life, working with a therapist who identified language as a barrier to my progress in a therapeutic context. Describing my articulacy as a ‘defence mechanism’ and method of separating myself from deeply held feelings and memories, she used truncated words while she asked me to hold different parts of my face and body which released a torrent of emotion that had never occurred in therapy before. This visceral landscape, the stories stored under our skin, is extremely interesting to me as a practitioner using personal experience to create practice work and is linked to Vessel van der Kolk’s work in The Body Keeps the Score (Van der Volk 2014) a key influence in attempting to articulate the language of the body. Van der Kolk presents traumatic stress and traumatic memories as key to the understanding and development of neuroscience. Similarly Gabor Mate and Peter Levine (https://scienceandnonduality.com/videos/trauma-and-the-somatic-connection accessed March 2023) discussing trauma and the body and Lamia (2011) on Shame and its manifestations were helpful in the formulation of auto ethnographic methodology.

The MAMVRP also builds on the dominant theoretical framework for the use of film in the mental trauma intervention space, which is the Trauma Film Paradigm (TFP). As argued by Anne A Cuperus et al. (2017) the TFP, introduced by Lazarus (1964), is a well-established method to study the effects of analogue psychological trauma under controlled laboratory settings. The TFP uses film depicting traumatic events to elicit trauma-related emotional memories, producing measurable responses analogous to viewing a traumatic event in real life (James et al. 2016). However, Dibbets and Schulte-Ostermann (2015) suggested that watching films is passive and lacks active behavioural engagement, and explored the idea that VR may be more effective because it can induce a feeling of embodiment, presence and agency lacking in a two-dimensional film, which may lead to more realistic (Slater 2009) and more emotional (Riva et al. 2007) responses. The key innovation of embodied VR is that it allows users to be the protagonist in a narrative (Sanchez-Vives and Slater 2005). Therefore, when developing creative practice around this VR project an important consideration was the use of presence and embodiment in a 360 world, elements that are absent in the original 2D film interventions used in traditional TFP scenarios. I was interested to expand on it by exploring the creative technology of CVR and look at how immersion and embodiment might affect the data gathered around viewer reception of traumatic narratives. A further layer of practice-based research is added by the inclusion of binaural sound and positioning sound in different parts of the body in an attempt to represent emotion. While the first animated piece in the Mental Abuse Matters portfolio attempted to capture cognitive dissonance, dissociation, shame, anxiety and withdrawal through the use of abstract shapes and soundtrack in 2D, the VR piece attempts to create a 360 world in which a headset user using headphones experiences these in a more direct and immersive way through the soundscape, point of view and visual distortion to represent dissociation, for example. Sound is used in a layered, positional form. There are three levels of dialogue within the piece. One is within the environment of the house, between characters; one is the inner voice of the victim character that the headset user embodies and the 3^rd is memory in the head of the victim character, triggered by specific behaviours, words or facial expressions of the perpetrator. Does the fact of presence (Slater 2009) make a difference to reception of and response to a narrative around emotional abuse?

The development and production process for Mental Abuse Matters

Research questions during concept and script development for the MAMVRP were centred around representation of a turbulent inner state through a VR world, and how soundscape, dialogue and visuals might be employed to enhance the experience of that state in the user. For example, can sound be positioned to replicate physiology and used, along with storytelling signifiers, to elicit empathy through direct experience; and can this empathy be measured? How might embodied VR interventions for training in health and social care enable retention of information and understanding of the victim’s experience?

The first steps in creating a narrative drama were beginning the characterisation and script writing process, visualising the 360-degree immersive space, and deciding the degree of embodiment for the victim character. Central to this was the aforementioned concept of the visceral narrative, that might allow a headset user to experience the internal states of the character. These include self-talk, memory, shame, dread, panic, and dissociation for example. To realise this, I felt that on-location dialogue should be minimal and that a soundscape representing emotions, rather than music tracks or a composed music score, should be used throughout. The use of a score or music soundtrack might take the user out of the interiority of the experience when the aim was to build up the tension both in the body and mind of the user and in the surrounding location of the house, towards a crescendo at the end. This was designed to be achieved through sound effects such as heartbeat and breathing, an intermittent ambient soundscape pulsating in and out, and tone, body language and looks directed towards the victim at certain times throughout the narrative, as outlined by the Sound Designer Hanne Slattne, below.

Specific points throughout the story act as triggers for internal states of shame or dissociation and their representation in the soundscape or visuals. Binaural sound was designed to recreate emotions in the body of the user in response to specific triggers from the perpetrator such as a facial expression, tone, phrase, body language or telling a particular story; impressionistic memories that flood the internal world of the user; and visual and aural distortion to recreate the experience of a dissociative state, (Codrington 2017) which has been linked to the triggering of a shame attack (Lamia 2011) in those who have experienced trauma. For example, the opening sequence contains an internal commentary in the woman’s head, in the voice of her partner, with snippets of comments made about her appearance or demeanour over time. These snippets she has come to associate in her mind with certain items of clothing or makeup, as she gets dressed. When descending the stairs to join her friends and partner at the dinner table, the user experiences a vertiginous feeling through visual perspective, along with the dread themed soundscape at the chest/stomach level. Isolation is highlighted by the juxtaposition of her dissonant internal landscape and the warm, friendly hubbub and laughter downstairs as she psyches herself up to join them. Once seated, a throwaway comment about her cooking increases heartbeat and an affable account of a ‘funny’ (but actually humiliating and oft repeated) story by her partner triggers a memory in the victim, which is presented through audio, and a visual distortion to represent her dissociation – something which for me often occurred alongside the shame attack described by Lamia (Lamia 2011) while in company. Another aspect I was keen to explore is the subtle (though perhaps unconscious) collusion of the friends in the humiliation and ostracisation of the victim character, Lisa. As they become increasingly drunk throughout dinner, much hilarity ensues around apparently fond memories they share where Lisa is the butt of the joke. When scripting the piece, I was aware of the desire to tread a fine line with the friends – they may sense that there is something not right, and try to corral Lisa into participation, while simultaneously being swept along by the charisma and conversational dominance of her partner Gabriel. The aim here is to convey the intangible nature of emotionally abusive dynamics – both experienced and witnessed.

As a writer, navigation of the VR world was at the forefront of my mind; how the needs of this story differ from flat screen dramas. I wanted the focus to be on actors’ facial expressions, body language and the immersive soundscape, rather than distracting the user with complex plot or dialogue; the immediacy of the embodied experience was the most important element for me in terms of creating empathy for the victim when inside the VR headset. So here, the concept of the Visceral Narrative is key; not solely in the sense of eliciting a psychological and/or attendant physiological response in the viewer (Welles Schock 2014) but literally getting beyond language and into the recreation of physiological responses in practice work. So often, language is a boundary between, for example, a health or social care practitioner and a trauma victim. It is either insufficient to represent the experience or at times impossible, if the victim is not able to verbalise because of trauma freezing or language barriers. When inhabiting the victim character as a writer I moved between her internal world and dialogue, including memories and self-talk, and the external world of her interaction with her partner and friends. The victim character is at a crisis point in her mental health, experiencing symptoms of extreme anxiety, panic and dissociation (Haferkamp et al. 2015) and the relationship with her partner is at breaking point.

In writing a traditional narrative film using first person perspective, characters’ backstories might be represented in a variety of ways, including flashbacks or voice over. These were not an option for the MAMVRP, because the user is ‘dropped in’ to the life of these characters and experiences a moment in time directly, without exposition; as the user embodies the main character, it is not possible to tell her story in the 3rd person. The challenge, therefore, was to recreate a backstory through visceral signposting.

The challenge then became how to establish accumulated emotional abuse over time, without including a 3rd person backstory to tell us about this couple. I wanted to avoid an ‘introduction’ sequence as this would necessitate seeing the victim in the 3rd person, an established trope in games that might affect immediacy and engagement on the part of the user. Instead, feeling emotions in the body tied to the perpetrator – his presence, voice, memories of things he had said and distressing events – could potentially be an innovative way of establishing the direct link between behaviour and mental destabilisation over time. Inner states could also be conveyed through memory dialogue and inner voice. Another question that came up when devising the scenario before writing began was the presence of the victim in the experience; should we ever see her? How much should we hear her, and what should she say? In collaboration with Dramaturg and Sound Designer Hanna Slattne, we attempted to block out the physical and audio space, from the user’s point of view. Through various testing experiences it became apparent that both the physical and aural presence of the victim character should be limited, to enable as much immersion in and merging with the character as possible. Hanna created a bespoke method of recording sound near the stomach and chest, to produce vibratory effects in these areas of the body, and she created the illustrations below to visualise our plans (Figures 2–4).

Figure 2. A scene by scene breakdown of where emotions appear in the body.

8 pencil drawn body figures with colours painted in different sections of the body.

Figure 3. Emotion positions in the body.

2 pencil drawn body figures with arrows pointing to different labelled body parts.

Figure 4. Humiliation in the body.

1 pencil drawn body figure with red paint around the head, neck, chest and stomach with the word Humiliation at the top.

Hanna says:

The ideas of the interoceptive sound design for MAM was to create the visceral and physiological feelings linked to strong and complex emotions. This interoceptive sound design was to be layered and integrated with the diegetic sound design of the filmed location. As there was external dialogue as well as some internal thought, I needed to create a sense of spatialisation for the interoceptive sounds as if situated below the head and in body, away from the sonic space normally reserved from the internal voice of the protagonist, in the middle of the head (mono).

As I was layering my sound design with the recordings of the actors’ voices, the ambisonic location ambience and other sound design, I realised that I needed to reduce complexity of the initial ambition and focus on producing two registers that recur, and through repetition, become familiar and easier to read. I created the dread/fear strand of sounds using low frequency sounds and a set of higher frequency sounds denoting embarrassment/humiliation. These are layered with interoceptive breath and heartbeats as well as being clearly distinguished from the other sound elements. The dread theme is designed to give a sense of a long-term built-up presence of stress that is always with Lisa, the protagonist, when she is inside the house, and that is heightened as soon as she hears the key in the door when Gabriel, her partner, comes home. Modulations in the sound respond to Gabriel’s movements and the intensity increases as he approaches. The Embarrassment/shame and humiliation theme is used mostly when guests are present and also changes in intensity responding to the actions of Gabriel.

Considering that visuals, dialogue, and the actors carry the main present tense storytelling, the challenge for me was to represent Lisa’s subjectively felt sense of self, laden with a substantial backstory. The idea was for an immersant, with their own experience of mental abuse, to recognise this thick presence of coercion responding to the slightest of triggers. (Slattne 2022)

Many elements in VR narrative drama are different from flat screen film making (Mateer 2017). As filmmaker Doug Liman comments in his interview with Robertson ‘VR should be more emotionally involving, but that doesn’t happen automatically by just taking a VR camera and sticking it onto what would be a traditionally blocked scene for 2D’ (Robertson 2016).

Firstly, the position of the embodied victim user/character within the VR world must be considered (Landau, Hasler, and Friedman 2020). The scenario is a first-person point of view experience of an established intimate relationship, where there is a history of emotional abuse over years. However, we wanted to limit this VR experience to a maximum of 10 min and keep dialogue sparse, for reasons outlined above. Although the behaviour is subtle, a live action first person experience is intense; it may potentially be more triggering than avatars because of the immediacy of photorealistic imagery (although this is currently unknown because live action VR has not so far been used frequently in a therapeutic or training context); too much dialogue may undermine the physicality of the experience; and there is research to suggest that lengthy VR experiences can potentially be damaging to eye health (Do VR Headsets Damage Vision). As an actor, the victim character Lisa was required to be both there and not there. Her voice occasionally needed to be captured on location and her voice over was used throughout, however her fellow actors interacted with the camera rather than the character on set, a novel and disconcerting experience for the actress Mary Lindsay who plays the victim and was seated behind the camera for sound capture purposes. Traditional narrative TV drama has in the last 10 years developed a penchant for ‘breaking the fourth wall’ (McGuiness 2022) and this was a necessary part of the actors’ methodology in that they interacted with a camera, not a person, and naturalistic reactions were more difficult to achieve. Compounding this was a storyline that required the actors to subtly dismiss, humiliate and undermine the victim character throughout a dinner party conversation. Good practice was, therefore, to check regularly on VR headsets how the experience was coming across for the user; did it feel like sitting at a table with a group of people in terms of distance and their interaction with the camera, and were the layers and complexities of this 4-person interaction coming across effectively? Similarly, was the passage of time clear as we went in and out of her dissociative states and memories, while seated at the dinner table? Both visuals and sound become distorted at points in the experience where specific words, facial expressions or body language trigger the victim. The idea is to demonstrate what repeated patterns of behaviour, which may appear innocuous at first glance, can do to a person’s physiology, nervous system and mental state, building to a crescendo where Lisa explodes and Gabriel steps back, contemptuous and triumphant, with the familiar refrain that she is being hysterical (Figures 5 and 6).

Figure 5. Dinner scene.

Photo of two men and a woman at a dinner table looking directly at the camera.

Figure 6. Conflict scene.

Photo of a man standing in a kitchen looking directly at the camera with a contemptuous look on his face.

It was, therefore, an iterative process to capture the right intuitive mix for the friends, Alex and Bronagh (played by Colin Ash and Roisin Gallagher) of looking at the perpetrator Gabriel (played by Paddy Buchanan), looking at the victim Lisa and conveying both an oblivious, increasingly drunken complicity with him and an understandable concern for her while also timing the private looks of warning or contempt that pass directly from Gabriel to Lisa as intimate partners. Occasionally scenes would be delivered with interaction that felt too frequent, too sparse or exaggerated in some way and scenes would require a re-shoot to accommodate a more intuitive experience for the user.

VR production workflow

The directing process and working with actors in VR live action was an interesting departure from traditional filmmaking. The panoptic cinematic VR camera has 6 lenses and captures in 360 degrees, so scenes can be shot from beginning to end rather than creating a shot list of different set ups and angles to put the jigsaw of the narrative together, as with 2D filming. The main concern is where to place the 6-lens camera to ensure that actors are close enough while avoiding visual distortion, and that the stitching together of these lenses works in post-production to create a seamless experience.

The logistical process was relatively smooth and seamless compared to flat screen filming, with a less labour-intensive workflow because the need to take a selection of shots is removed. However, blocking and interaction with the camera are complex in the 360 environment and extra care must be taken, as it’s more difficult for mistakes to be corrected in post-production. Scenes looking and sounding real, rather than computer generated as in games, was an important factor in creating the universe of the MAMVRP because user response to facial expression, body language, tone and sound position is a means of understanding its effectiveness and the degree of embodiment. The scenes were stereoscopic CVR is:

a 360-video filmed using a panoramic video camera system and played back as an equirectangular video file which allows the user to look around the scene as it unfolds. Depending on the camera system and stitching process the scenes can be either monoscopic (flat) or stereoscopic (3D). Here you have the advantage of scenes looking completely real and not computer generated as with game engines. Scenes are also usually captured with spatial sound microphones making them sound just as real. (CVR Field Guide 2021)

When placing the panoptic camera in position, it’s essential to be conscious of where to place objects and people in relation to the camera so that they are not in front of stitch lines, and to avoid the visual distortion that can occur from close proximity. As the Occulus Cinematic VR Field Guide states: ‘Stitch lines in 360 video are the areas of overlap between the lenses that have been stitched together, and appear as disconnected lines that are clearly meant to be continuous’ (CVR Field Guide 2021). Maintaining the integrity of the immersive environment is essential to creating an experience that puts the user in the shoes of the victim in this scenario; any break in continuity would take the user out of the head and body of the character and affect their journey through the narrative (Figure 7).

Figure 7. Camera position on location.

3 squares showing field of view with 94 degrees, 154 degrees and 204 degrees written on them.

Next steps and future research

Once completed, the 9.5-minute VR piece was shown to an Advisory Group. This included selected MLAs with a mental health or justice remit at the Northern Irish Stormont Executive, and practitioners in the Health and Social Care, Justice and harities sectors who work with victims or perpetrators of abuse. The group included therapists, family court barristers, members of the police force with a domestic violence remit, GPs, abuse charity workers and probation service trainers and psychologists. Recruitment to the Advisory Group was based on the relevance of expertise to the subject matter and format of the VR experience, and whether they might potentially use a VR piece in a training or therapeutic capacity or advise in some way on that use, for example in terms of safeguarding pathways or governance and regulation. As well as receiving recorded feedback during formal conversations with the Advisory Group, I created a questionnaire for them. I also received informal feedback from the viewing public at a Justice conference. There have been 17 Questionnaire responses so far. Responses include:

It got me thinking that emotional abuse could be overlooked by someone who isn't familiar with the traits of an emotional abuser, and how tricky it would be for the victim to send a cry for help.

it was very realistic and I had to take some time afterwards to digest the experience. But I feel it would be beneficial for training purposes and raising awareness of emotional abuse and the forms it comes in.

The zoning out of the victim was a really powerful aspect which depicted how the victim feels and tries to block out but when you are back in how she can’t escape.

The real-life actors made the experience very effective and surreal. Therefore, it was easier to feel the emotions that the victim would feel. I feel I would be more alert to behaviours like this if I was to see it first hand in my life or a friend's life. Therefore, it might have the same effect in training situations.

There was unanimous agreement that the VR piece put the user into the shoes of the victim in a powerful and visceral way and would make an effective empathy training tool in the health, social care and justice sectors. This is valuable feedback for the further development of this piece as both a training tool and, potentially, a therapeutic intervention in the realm of exposure or rehabilitation therapy as discussed in relation to the work of Sanchez-Vives and Loranger.

Expanding on the themes of trauma representation and viewer reception that emerged in the animated piece, I was interested in developing the intervention aspect of practice as research work with the VR experience – exploring the possibility of using 1st person embodied CVR for empathy training. As mentioned above live action is, arguably, necessary for an effective simulation of emotional abuse because subtle facial expressions and body language are key; however, it also raises complex ethical questions about the potential effects on the user of photo realistic imagery, for example in triggering trauma responses, which I am keen to interrogate and test through practice work.

The next step for the MAMVRP is to run a trial with a non-clinical sample of Social Work undergraduate students at Queen’s University Belfast, to test the prototype for training and gauge user reactions. This is now in its early stages. The trial will interrogate the effectiveness of the VR film as a potential training tool in comparison with the more traditional method used for assessing and understanding emotional abuse, which is a written report. Factors such as empathy, understanding and retention will be measured by questionnaires before and after the experience and presence and embodiment will be measured in post experience questionnaires. Small focus group interviews of between 4 and 6 people will collect qualitative data on the narrative itself and how the use of sound and visuals contributed to a feeling of being inside the victim’s shoes. An important element of preparation was the safeguarding of users given the potential for trauma triggers brought on by the phenomenon of Presence in immersive experiences and the subject matter. Rubin states: ‘Presence is what happens when your brain is so fooled by a virtual experience that it triggers your body to respond as if that experience were real’. That might mean ‘a fight or flight response kicks in’ (2018, 10). This was a key part of the ethics approval for this research trial, which is overseen by an experienced trauma counsellor and Reader in Social Work who is on hand to manage any adverse reactions from the student cohort. Previous discussions with medical practitioners such as Dr Heather Mills, a retired Consultant Medical Psychotherapist in the Southern Medical Trust in Belfast, suggest that the mechanism of triggering may not be obvious. For example, victims of mental abuse can suffer various psychological effects including Complex PTSD. While it might be assumed that those who have previously experienced abuse may be triggered, Dr Mills suggested that in fact those who have located and identified their trauma are less likely to experience a triggering episode as a result of exposure to this narrative. This was borne out by the feedback from Advisory Group members who had been exposed to emotional abuse, all of whom reported a feeling of validation and clarity from identifying patterns that they too had experienced, rather than a re-traumatising effect. Dr Mills suggests that those who have deep underlying and unrecognised trauma, conversely, may be caught off guard while being immersed in such a scenario. In addition to this, a small minority of VR headset users experience motion sickness or vertigo in immersive landscapes (Chang, Kim, and Yoo 2020). Possible solutions to these factors include a short screening clip at the beginning of the experience, to help users orientate and flag any physiological or psychological reactions.

Preliminary findings from Advisory Group feedback on the MAMVRP suggest that that this is an effective intervention in terms of experiencing the inner landscape and bodily responses of an emotional abuse victim through embodiment and immersive sound, and enhancing empathy and understanding, addressing research questions centred around representation of a turbulent inner state through a VR world, and how soundscape might be employed to enhance the experience of that state in the user.

The challenge of depicting mental abuse in a visually creative way has led me as a filmmaker to explore new forms of expression, such as animation and immersive technology, seeking to create tools that are cathartic and practically helpful for victims, perpetrators and care professionals. VR has moved beyond the world of gaming and is now used as an increasingly effective intervention in many different fields. Arguably, this kind of practice is a valuable and exciting use of VR and can lead to new perspectives on emotion regulation and relationships, for both practitioner and viewer, and further develop an already fruitful relationship between film making, social work, psychology and healthcare.

Acknowledgements

This work was conceived, written, produced and directed by Lucy Baxter. Belfast-based company Retinize filmed and edited the piece, and Hanna Slattne was dramaturg and immersive sound designer. the Sonic Arts Research Centre at Queen’s University Belfast and Trustees at the charity Mental Abuse Matters were advisors. Thanks to supervisors and colleagues Dr Paul Best, Professor Cahal McLaughlin, Dr John D’Arcy and Professor Pedro Rebelo.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Future Screens NI [grant number FSNI OC105].

Notes on contributors

Lucy Baxter

Lucy Baxter is a filmmaker and film practice lecturer at Queen’s University Belfast. She has over 20 years’ experience making content for broadcasters, digital platforms, education and cinema exhibition and her book Running a Creative Company in the Digital Age was published in 2017. Her research is across creative enterprise, creative producing in documentary and fiction, Irish cross border co-production and the applications of digital content and virtual reality film in health and social care, particularly emotional trauma. Lucy has several years’ experience producing for international film festivals and was co-founder and head of industry at a new all Ireland international film festival, Docs Ireland, in collaboration with the Belfast Film Festival. This has included creating the first documentary focused international marketplace in Ireland.