Abstract
People with psychosis can experience social functioning impairments. Virtual reality (VR) has been used to assess and treat these difficulties. This systematic review (Prospero CRD42015026288) provides an evaluation of these VR applications. PsycINFO, MEDLINE, Embase, Web of Science, Cochrane Library, and Scopus were searched until May 2020. The Effective Public Health Practice Project (EPHPP) Quality Assessment Tool was used to assess studies. Database searching identified 3810 titles. Fifty-eight studies (published 2005–2020; N = 2,853), comprising twenty-six head-mounted display studies (20 assessment, 6 treatment) and thirty-two immersive 2D screen studies (23 assessment, 9 treatment), were included. There were forty-eight observational studies and ten randomised controlled trials, with 1570 participants (of which, 185 were at ultra-high risk of psychosis) in VR test groups. Nearly half the studies were published since 2016. Assessments targeted cognitive and behavioural indicators of social functioning, e.g. paranoia, eye gaze, or interpersonal distance. Treatments promoted cognitive-behavioural social skills or job interview training. Studies indicate feasibility, acceptability, and effectiveness of VR for social functioning impairments in psychosis. Limitations of studies include the narrow scope of social functioning, small sample sizes, and limited randomised controlled trials and standardised interventions. Findings suggest VR has potential to be integrated with existing psychological approaches.
Introduction
Social functioning has been defined as an individual's interactions with their environment and their ability to fulfil their role in work, social activities, and relationships (Bosc, Citation2000). Impairments in this domain can translate into long-standing problems in people with psychosis (Addington et al., Citation2008), associated with poor outcomes and reduced activities of daily living (Viertiö et al., Citation2012). Psychosis is strongly associated with prolonged experiences of social isolation (Reininghaus et al., Citation2008), diminished social networks (Garety et al., Citation2001), and impaired social cognition (Bertrand et al., Citation2007; Couture et al., Citation2006). These can have a significant, detrimental effect on mood (Hirschfeld et al., Citation2000), and can impact negatively on positive symptoms (Hartley et al., Citation2013), especially given the relationship between psychotic symptoms and mood disorders (Cheniaux et al., Citation2008; Lake & Hurwitz, Citation2006). Cognitive behavioural interventions that target social networks, social support, and social performance appear to show some positive impact on recovery and psychological wellbeing in people with psychosis compared to control groups (Gayer-Anderson & Morgan, Citation2013; Lutgens et al., Citation2017; Schrank et al., Citation2014), although reviews highlight that positive impact on social functioning can be limited and that, in general, social functioning is hard to improve in service users, especially for young people who are at clinical high risk of developing psychosis (Devoe et al., Citation2019).
Virtual reality (VR) is emerging as an effective therapeutic tool to support a range of mental health conditions (Freeman et al., Citation2017; Valmaggia et al., Citation2016), particularly for people who struggle in social situations (Gregg & Tarrier, Citation2007; Riva, Citation2005). An advantage of VR in therapeutic settings, particularly for cognitive behavioural therapy (CBT), is that it can provide standardised, ecologically valid virtual environments that can be manipulated in real-time, while eliciting cognitive, emotional, and behavioural responses similar to those triggered by real-world situations (Slater & Sanchez-Vives, Citation2016). VR technologies appear to have promising applications for assessment and treatment for people with more complex and severe mental health conditions, such as psychosis (Veling et al., Citation2014). Studies have focussed on assessment of paranoid ideation, cognitive and psychosocial rehabilitation, social skills training, and virtual reality-assisted therapies (Fernández-Sotos et al., Citation2020; Rus-Calafell et al., Citation2018; Tan et al., Citation2018) and appear cost-effective when integrated with CBT (Pot-Kolder et al., Citation2020).
The term virtual reality has been used to describe a wide array of interactive computer technologies and interventions, some involving user engagement with immersive, 3D environments through head-mounted displays (HMDs), and others involving user interaction with immersive environments through 2D computer screens. In general, HMD technologies tend to be regarded as more immersive, ecologically valid, and are increasingly regarded as the paradigm of VR; however, 2D screen interventions that involve immersion within a ‘synthetic reality’ or a ‘virtual world’ are conventionally regarded as VR (Seidel & Chatelier, Citation2013).
Previous systematic reviews have not fully analysed these innovations in VR in relation to this vital area of social functioning in psychosis and so the present study (PROSPERO ID: CRD42015026288) aimed to conduct a detailed review and quality assessment of VR-based psychological assessments and treatments for social functioning impairments in psychosis.
Methods
A systematic review was conducted on VR studies that evaluate assessment or treatment of social functioning in psychosis. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Moher et al., Citation2009) and Centre for Reviews and Dissemination guidance for systematic reviews (Khan et al., Citation2001) were followed throughout. Findings were synthesised in this review using a narrative approach.
Inclusion and exclusion criteria
Studies were included if they were experimental, published in peer-reviewed journals, written in English, used VR apparatus, involved interaction in a VR social environment, and assessed or treated at least one indicator of social functioning in people with a stated diagnosis of a psychotic disorder, and if N ≥ 5 for the clinical sample. People at ultra-high risk of psychosis were included in this review due to the high risk of developing psychosis within the first three years of clinical presentation, although it is important to note that not all people at ultra-high risk will transition to psychosis (Fusar-Poli et al., Citation2012). VR studies that primarily assessed or treated social anxiety or social phobia were excluded to reflect the conceptual distinction between social functioning and social anxiety (Voges & Addington, Citation2005). The review adopted the convention of defining VR broadly as either using HMDs or immersive 2D screens, but reviewed these kinds of VR separately to reflect established qualitative differences in user experience between these technologies (Rus-Calafell et al., Citation2018).
Search criteria and procedure
Search terms were agreed by researchers (SR, LV, PG). Truncations and wild cards were used to identify mutations of terms. Studies were identified following a database search of PsycINFO, MEDLINE, Embase, Web of Science, Cochrane Library, and Scopus using the following search terms and Boolean operators: “virtual real*” OR “VR” OR “virtual enviro*” OR “virtual character*” OR “VCs” OR “avatar*” AND “social function*” OR “social dysfunction*” OR “social skill*” OR “social avoid*” OR “social cognit*” OR “social adapt*” OR “social behav*” OR “social inter*” OR “social stress*” OR “social learn*” OR “social percept*” OR “interpersonal” OR “inter-personal” AND “psycho*” OR “schiz*” OR “bipol*” OR “at risk mental state” OR “high risk” OR “early intervention” OR “paranoi*” OR “delus*” OR “halluci*”. All databases were searched for keyword, title, and abstract information. For applicable databases, key subject headings were exploded, and database searches were limited by journal articles and English language. Reference management software Endnote was used to extract data and screen records. Unpublished dissertations, conference proceedings and abstracts without locatable full texts were excluded. Reference lists of key papers were screened. Two reviewers (SR, SP) independently conducted all searching and screening in consultation with other researchers (LV, PG).
Quality assessment
The Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for Quantitative Studies method and tool was used to assess quality of studies (Ephpp, Citation2010). EPHPP assesses six methodological dimensions: selection bias, study design, confounders, blinding to the assigned condition or task, data collection methods, and withdrawals and dropouts. Global ratings are calculated as ‘strong’ if there are no ‘weak’ subscale ratings, ‘moderate’ if there is one ‘weak’ subscale rating, or ‘weak’ if there are two or more ‘weak’ subscale ratings. EPHPP has good content and construct validity, and inter-rater reliability (Thomas et al., Citation2004). Two reviewers (SR, SP) independently conducted the quality assessment in consultation with other researchers (LV, MR-C). Discrepancies were resolved through discussion and agreement was reached in all cases.
Results
Information extraction
Database searching on 14th May 2020 identified 3,810 titles. Nine further studies were identified by searching reference lists of key papers. 2,387 titles were identified for screening after de-duplicating and removing books and book sections in Endnote. After screening, fifty-eight studies published between 2005 and 2020 (N = 2,853) were included in this review. A total of 1,570 participants with psychosis were included in the VR test groups in these studies (561 HMD, 1,009 2D screen). Of these participants, 185 were reported to be at ultra-high risk of psychosis rather than having developed psychosis (all in HMD studies). There were thirty-four case control studies, fourteen cohort studies, and ten randomised controlled trials. There were twenty-six studies using HMD hardware, henceforth referred to as ‘HMD studies’ (N = 1,105; 20 assessment, 6 treatment), which used virtual social environments such as homes, cafes, bakeries, shops, bars, libraries, buses, trains, offices, or sitting at a table with avatars. There were thirty-two immersive 2D screen studies (N = 1,748; 23 assessment, 9 treatment), which used virtual social environments such as supermarkets, buses, job interviews, neighbourhoods, apartments, or one-to-one interactions with virtual agents. See for PRISMA diagram.
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) diagram of virtual reality studies that assess or treat social functioning impairments in psychosis.
Approximately half of all studies were published in the last four years. Studies were from Europe (N = 28), Asia (N = 18), United States of America (N = 10), and Australia (N = 2). Research groups in Korea, the Netherlands, United Kingdom, and United States of America appear particularly active in this research. Most clinical samples had a diagnosis of schizophrenia. A small minority of studies included participants with a diagnosis of bipolar disorder where psychotic symptoms were present (Kim, Jung, et al., Citation2009; Kim, Ku, et al., Citation2009) or included people with psychosis within a mixed diagnosis sample, including people with a non-psychosis diagnosis (Humm et al., Citation2014; Smith, Ginger, Wright, Wright, Boteler Humm, et al., Citation2014; Smith, Fleming, Wright, Jordan, et al., Citation2015; Smith, Fleming, Wright, Roberts, et al., Citation2015).
Whilst both HMD and 2D screen studies were immersive, there were some key differences in the types of apparatus used. HMD studies used headwear with LCD displays or 3D glasses, and often a head-position tracker. Immersion was likely to be greater in these studies because the display typically occupies the participants’ entire field of vision and the participant embodies an avatar. Tasks in HMD studies often included exploration of the environment, functional tasks such as shopping, as well as active listening and responding to avatars. By contrast, 2D screen studies typically used one or more LCD screens with a computer or a projected display. There is possibility of a less immersive experience as only a proportion of the participants’ field of vision was occupied by the display. Immersion was typically achieved in a similar way to HMD through active engagement or interaction with a game-like scenario, interview role-plays, or active listening and selection of responses via a mouse or keyboard. The difference between the two therefore was found in the way in which the participant responded to the virtual environment; through physically speaking or moving in the virtual space with HMD or watching the scene and selecting a response when using a 2D screen programme.
In order to evaluate proposed indicators of social functioning, a vast number of measures were used across all the studies. Some studies assessed ‘functional capacity’ in a broad sense (Ventura et al., Citation2020), whilst others assessed more discrete indicators of social functioning. The most commonly assessed social functioning domains across the studies were emotion recognition, interpersonal distance, eye gaze, social interactions, social cognition/perception, facial cue/facial emotion recognition, and paranoia. Some studies applied assessments that measure specific social functioning indicators such as the BLERT (emotion recognition task), EPT (emotional perspective taking), LSAS (Liebowitz Social Anxiety Scale), RCS (relationship change scale), which are all key elements within the concept of social functioning. Only two studies used The Social Functioning Scale (SFS) (Adery et al., Citation2018; Rus-Calafell et al., Citation2014). However, this scale relies on subjective evaluation while excluding important aspects of social functioning. Similarly, the Social and Occupational Functioning Assessment Scale (SOFAS), used in two studies (Pot-Kolder, Geraets et al., Citation2018; Pot-Kolder et al., Citation2017), is a broad assessment of global functioning based on subjective report, rather than discrete indicators that can be objectively measured. See and for full details of study characteristics.
Table 1. Characteristics of head-mounted display virtual reality studies on assessment and treatment of social functioning impairments in psychosis.
Table 2. Characteristics of 2D screen virtual reality studies on assessment and treatment of social functioning impairments in psychosis.
Quality assessment
Overall, forty-seven studies (81%) received a ‘strong’ EPHPP global rating, ten studies were ‘moderate' and one was ‘weak'. Of the twenty-six HMD studies, twenty-four were rated ‘strong’, two were ‘moderate’, and none were ‘weak’. Study designs comprised sixteen case-control studies, five cohort studies, and five randomised controlled trials. Of the thirty-two 2D screen studies, twenty-three were rated ‘strong’, eight were ‘moderate’, and one was ‘weak’. Study designs comprised eighteen case-control studies, nine cohort studies, and and five randomised controlled trials. Seven of these ten randomised controlled trials were technically defined by the EPHPP as controlled clinical trials due to randomisation not being described. Analysis of methodological dimensions indicated lack of blinding to assigned conditions or tasks, lack of dropout reporting, and confounding variables, such as age, gender, education, ethnicity, and marital status for both HMD and 2D screen studies. See and for full details of quality assessment.
Table 3. Effective Public Health Practice Project global ratings and methodological dimensions of head-mounted display virtual reality studies to assess and treat social functioning impairments in psychosis.
Table 4. Effective Public Health Practice Project global ratings and methodological dimensions of 2D screen virtual reality studies to assess and treat social functioning impairments in psychosis.
Safety and acceptability
Overall, studies indicate that people with psychosis find it safe and acceptable to use VR to assess and treat social functioning impairments; they engage in the social environments, experience acceptable levels of immersion or sense of presence, perceive avatars as real, and the virtual social environments elicit negative thoughts and feelings that are similar to those that they experience in real-world situations (Bell & Weinstein, Citation2011; Fornells-Ambrojo et al., Citation2008; Ku et al., Citation2007; Ku et al., Citation2005; Oker et al., Citation2015; Rus-Calafell et al., Citation2013; Thompson et al., Citation2020; Valmaggia et al., Citation2007).
Head-mounted display studies
Assessment
Twenty HMD studies (N = 838) published between 2007 and 2020 used mainly one, but occasionally two, sessions of VR to assess cognitive, behavioural, and physical indicators of social functioning in people with psychosis. 428 people with psychosis were in the test groups for these VR HMD assessments. Sample sizes ranged from 21 to 170. Most studies used a brief VR task and three of the twenty studies included a follow-up session. All studies were rated ‘strong’. Eighteen of the twenty studies incorporated a control group. In general, tasks invited participants to explore the environment, interact with avatars, identify emotional expressions, interpret social cues, or recall aspects of environments. Environments included bars, cafes, trains, and homes. VR-based assessments were found to be feasible and did not increase anxiety or cause negative experiences (Valmaggia et al., Citation2007). Studies found that clinical samples displayed various indicators of social functioning impairments in these social environments, such as reduced eye gaze and greater interpersonal distance as measured by eye or position tracking technology within the HMD (Choi et al., Citation2010; Geraets et al., Citation2018; Han et al., Citation2014; Kim, Ku, et al., Citation2009; Park et al., Citation2009), greater interpersonal sensitivity (McDonnell et al., Citation2018), increased physical stress responses (Counotte et al., Citation2016), less accuracy in facial recall (Dietrichkeit et al., Citation2020), less accuracy in identifying emotions and social cues (Souto et al., Citation2013; Souto et al., Citation2020), higher belief inflexibility bias and more attention to threat and more external attributions (Pot-Kolder et al., Citation2017), and notably, in six studies, increased paranoid ideation, (Freeman et al., Citation2010; Hesse et al., Citation2017; Valmaggia et al., Citation2015; Valmaggia et al., Citation2007; Veling et al., Citation2016; Veling et al., Citation2014). One study used VR as an assessment of social functioning over six sessions in the context of testing an antipsychotic treatment and found that antipsychotic medication significantly improved social functioning as assessed by VR (Park, Ku, et al., Citation2009).
Treatment
Six HMD studies (N = 267) published between 2011 and 2020 have used between one and sixteen sessions to treat aspects of social functioning impairments in psychosis, in particular paranoia, anxiety, and distress, with two of the six studies including a follow-up session after the main intervention. 133 people with psychosis were in the test groups for these VR HMD treatments. Sample sizes ranged from 12 to 91. All studies reported positive results and appear to have been conducted broadly within a CBT framework. Four studies were rated ‘strong’ and two studies were rated ‘moderate’. All studies incorporated a control group. There were three randomised controlled trials, all of which received a ‘strong’ rating (Freeman et al., Citation2016; Park et al., Citation2011; Pot-Kolder, Geraets et al., Citation2018). Environments across the studies included a variety of indoor and outdoor settings, such as shops, cafes, and streets, as well as transport settings, such as buses and trains. Two multi-session treatment interventions indicated that VR can be a useful tool to supplement social skills training. A ten-session randomised controlled trial (N = 91) of VR role-plays over five weeks improved verbal communication in people with schizophrenia compared to controls with schizophrenia in a non-VR-group (Park et al., Citation2011); and a sixteen-session, pilot study of social skills training, with four-month follow-up (N = 12), showed improvements in social skills in a small clinical sample (Rus-Calafell et al., Citation2014). Studies have also targeted cognitive appraisals such as paranoia in social environments, with one randomised controlled trial indicating that belief testing can reduce paranoia and delusional conviction about avatars (Freeman et al., Citation2016). The largest study, a randomised controlled trial of sixteen sessions of therapist-guided VR-based CBT over eight-to-twelve weeks (N = 116) for people with social functioning impairments in the context of a psychotic disorder employed thought challenging, dropping safety behaviours, and testing harm expectancies in VR, indicating that VR-based CBT significantly decreased anxiety, paranoia, and safety behaviours in people with a psychotic disorder compared to a non-VR control group of people with a psychotic disorder (Pot-Kolder, Geraets et al., Citation2018).
Immersive 2D screen studies
Assessment
Twenty-three immersive 2D screen studies (N = 1,451) were published between 2005 and 2020. All included a single session and brief VR task to assess social functioning in psychosis, in particular eye gaze, emotion recognition, theory of mind, and interpersonal distance. A total of 827 people with psychosis were included in the test groups of these VR based assessments. Sample sizes ranged from 11 to 333. Seventeen studies were rated ‘strong’, five were rated ‘moderate’, and one was rated ‘weak’. Nineteen of the twenty-three studies incorporated a control group. The VR tasks included recognising emotions of avatars, engaging in social interactions or conversations with avatars, interpreting social cues during games or interactions, and completing activities of daily living, e.g. shopping, getting a bus, navigating the environment. VR was found to be acceptable and participants perceived avatars as real (Keefe et al., Citation2016; Ku et al., Citation2005; Oker et al., Citation2015). One study had a very large sample (N = 333) and is likely to have more robust findings in terms of indicating the acceptability of the VR (Keefe et al., Citation2016). Studies found that clinical samples had impairments related to eye gaze (Bekele et al., Citation2017; Caruana et al., Citation2019), impaired theory of mind (Canty et al., Citation2017), performed more slowly and made more errors during recognition and interaction tasks (Ventura et al., Citation2020), had higher intimacy for distant avatars (Park et al., Citation2014), and especially had difficulties in emotion recognition and related facial and social cues (Berrada-Baby et al., Citation2016; Dyck et al., Citation2010; Gutierrez-Maldonado et al., Citation2012; Kim, Jung, et al., Citation2009; Kim et al., Citation2005; Kim et al., Citation2007; Marcos-Pablos et al., Citation2016; Song et al., Citation2015; Thirioux et al., Citation2014). At a neuroimaging level, studies found functional deficits in the mirror neuron system and dysfunction of the dorsolateral prefrontal cortex-superior temporal sulcus network (Kim, Jung, et al., Citation2009; Park, Kim, et al., Citation2009; Shin et al., Citation2015). Other studies found that a VR task may enhance predictions of real-life performance (Greenwood et al., Citation2016); that people with schizophrenia were slowest in decision-making in VR (Han et al., Citation2012); and that performance of a clinical sample was poorer on socially relevant tasks for those with greater severity of reduced emotional experience (Harvey et al., Citation2019).
Treatment
Nine 2D screen studies (N = 279) published between 2007 and 2018 used between one and twelve sessions to treat social functioning. 182 people with psychosis were in the test groups for these VR treatments. Four of the nine studies incorporated a follow-up session after the main intervention. Six studies were rated ‘strong’ and three were rated ‘moderate’. Four of the nine studies incorporated a control group. There were four randomised controlled trials, of which three received a ‘strong’ rating (Smith, Ginger, Wright, Wright, Bell, et al., Citation2014; Smith, Fleming, Wright, Roberts, et al., Citation2015; Tsang & Man, Citation2013). All studies reported positive results. Participants found the VR acceptable, realistic, and easy to use (Bell & Weinstein, Citation2011; Smith, Ginger, Wright, Wright, Bell, et al., Citation2014). Treatments aimed to train social skills, performance during job interviews, and vocational skills. Studies found a reduction in positive and negative symptoms after ten sessions of training in social interactions (Adery et al., Citation2018), improvements in memory and attention after twelve sessions of cognitive training in social environments (Amado et al., Citation2016), improvements in social performance and functioning after eight sessions of vocational rehabilitation training (Sohn et al., Citation2016); and improvements in cognitive functioning, executive functions, problem solving, categorisation, memory, attention, and self-efficacy after ten sessions of vocational skills training (Tsang & Man, Citation2013). Two studies focussed specifically on performance during job interviews and found that VR-trained participants scored higher in observer-rated role-play interviews after five sessions (Humm et al., Citation2014), and ten sessions of VR training (Smith et al., 2015), which appeared to be maintained in terms of job offers at follow-up (Smith et al., 2015).
Discussion
The aim of this systematic review was to evaluate VR as a tool for the assessment and treatment of social functioning impairments in psychosis. The review followed the convention of defining VR as either using HMDs or immersive 2D screen technology. In general, both assessment and treatment studies typically invited participants to explore social environments or to engage in conversations with avatars who provided programmed verbal and behavioural responses. Overall, studies indicated that VR is feasible and acceptable to assess and treat social functioning in psychosis, and its effectiveness is tentatively supported. Methodological limitations include limited randomised controlled trials, standardised environments, standardised measures of social functioning, longitudinal analysis, a limited number of HMD treatment studies, and a large number of studies with very small sample sizes. Interest in the possibilities of using VR to improve social functioning in psychosis is clearly growing, given the escalating number of recently published studies. This has the potential to have a large impact as more virtual environments are programmed. However, the limitations of the current studies indicate the need to conduct further research in this relatively new area.
VR assessment studies have typically aimed to showcase potential advantages of VR rather than produce original findings about social functioning impairments in psychosis. The cognitive and behavioural indicators of social functioning represented in these studies, such as paranoia, emotion recognition, interpersonal distance, and eye gaze, are well-known and established impairments associated with psychosis (Barkl et al., Citation2014; Edwards et al., Citation2002; Osborne et al., Citation2020; Thompson et al., Citation2011). VR studies have sought to demonstrate advantages of novel VR-based approaches over traditional approaches, such as emotion recognition tasks that provide a controlled environment against which to test impairments, or eye and position tracking integrated within the HMD which allows researchers to investigate behavioural indicators of social functioning that can otherwise be difficult to measure. These studies suggest that controlled VR environments with eye and position tracking might be an optimal way to measure these indicators. However, lack of direct comparison with non-VR-based interventions means that there can only be limited support for this view.
The most sophisticated treatment studies have used role-plays with avatars in VR as a form of social skills training, or used VR environments to supplement CBT techniques, such as belief-testing or dropping of safety behaviours. However, the relative paucity of treatment studies overall, and especially within the HMD category given its present-day prominence, limits our understanding of their potential. Limited longitudinal data in treatment studies means it is often impossible to know whether gains attributed to VR-based interventions were maintained in social situations outside of VR. Only one HMD treatment study (Pot-Kolder, Geraets et al., Citation2018) compared VR-CBT to non-VR CBT showing that momentary anxiety and paranoia significantly decreased, and safety behaviours reduced over time in VR. Nevertheless, the sheer array of virtual environments used in these studies is a clear indicator of the potential of VR to expand the options in therapeutic settings in ways that can be convenient to both clinicians and service users, for example to carry out CBT techniques such as behavioural experiments or exposure therapy in a variety of settings without leaving the clinic. This is a fast-moving field and further studies currently underway promise to further demonstrate the way VR can integrate with CBT-based approaches (Freeman et al., Citation2019; Lambe et al., Citation2020; Nijman et al., Citation2019; Realpe et al., Citation2020). As this field develops, studies must seek to understand what aspects of social functioning could be targeted by VR-CBT, how these interventions can best employ VR-based experiences, and identify key interpersonal and social processes that can be tailored and personalised to individuals receiving that treatment (Riches et al., Citation2020).
This review highlights the heterogeneity of approaches that we regard as VR-based interventions for social functioning impairments. This heterogeneity limits the validity and generalisability of the studies. Overall the studies included in this review used a wide variety of VR apparatus, environments, software programs, and measures. The 2D screens that were used across studies appeared to be of a wide variety of sizes, with some very large, although in general fewer apparatus details were provided for 2D screen studies. There are important reasons to include both HMD and 2D immersive screen studies in this review as both are regarded as VR by convention, but they evidently provide different user experiences (Rus-Calafell et al., Citation2018). This issue of heterogeneity is detrimentally affected by mixed quality of reporting by researchers on details such as VR apparatus, environment, participants’ task in VR, and participants’ navigation through the virtual environment, with some studies providing very little detail about these aspects of their study design and procedure. Many treatment studies provided little substantive content of VR sessions and the level of involvement by therapists. Since most studies create new environments there is a significant challenge to establish validity and reliability, especially in treatment studies. These factors limit the extent to which we can generalise results while also highlighting a more fundamental issue. The broad and evolving definition of the term ‘virtual reality’, driven in part by developments in hardware technology and programming, pose a challenge for comparability and replicability and suggest a need for clearer definitions and conceptual frameworks for VR (Girvan, Citation2018). This challenge may be compounded with developments in augmented and mixed realities; and as social lives move more online. Many of these issues are not psychosis-specific but should be regarded as important considerations in the design of future VR studies that aim to support social functioning.
One clear limitation of the research discussed in this review is the narrow scope of social functioning that has been studied and the lack of a specific measures that evaluate such a general construct. This review employs a broad and pragmatic definition of social functioning based on previous research (Bosc, Citation2000). However, the range of what we might regard as social functioning is large in scope; and, despite the ubiquity of the term, there is much debate in the literature about how to define the concept (Yager & Ehmann, Citation2006), including in the context of psychosis (Bjornestad et al., Citation2019; Burns & Patrick, Citation2007). The studies identified by this review are conceptually limited insofar as they investigate only discrete indicators of social functioning and specific social situations with limited social interaction, e.g. eye gaze, interpersonal distance, job interviews, or general social skills with limited interactions with programmed avatars. While they yield interesting findings, the extent to which one can draw general conclusions about social functioning is limited. Rather, the tentative conclusions that can be drawn pertain to the discrete indicators, rather than to social functioning in the broader sense. The concept of social functioning includes, as a matter of convention, a much broader set of components, encompassing but not restricted to employment, accommodation, social activities, family, friendships, and intimate relationships. These are life experiences that can currently only be accessed in a real-world community. It remains unclear how the ecological validity and evolving technology of VR can be harnessed to expand to more sophisticated interventions that target family, relationships, and more complex social situations; as well as how VR technology can be used to accommodate the reciprocal nature of social interactions.
Strengths and limitations of the systematic review
This is the first systematic review to investigate the use of VR to assess and treat social functioning impairments in people with psychosis. Strengths of the current review include the design of the search strategy; the large number of databases accessed; and employment of the distinction between HMD studies and immersive 2D screen studies, which clarifies differences in interventions and aids interpretation of the findings. Potential bias and error in literature searching, screening, and quality rating process was minimised by using two researchers to carry out all stages independently. There may be potential for error in the calculation of sample sizes due to limited or unclear reporting on dropouts and use of identical participants across studies. The exclusion of grey literature was intended to ensure the review focussed on only the highest quality peer-reviewed studies, but one consequence might be that findings could be subject to publication bias. The exclusion of qualitative studies was due to limitations of standardised tools such as EPHPP to assess their quality in comparison with quantitative studies. Their inclusion may have provided rich data on participants’ social experience in VR, although very few qualitative studies have been carried out in this area. Nevertheless, the EPHPP was chosen over alterative tools, e.g. the Cochrane Risk of Bias, due to its capability of providing consistent quality ratings for a range of study designs. However, there may be limitations of its sensitivity on specific domains. With the inclusion criteria employed in this review, there may be limitations related to diagnosis of participants. Samples in some studies were a mix of diagnoses, including psychosis, and so could have been classified separately. People at ultra-high risk of psychosis were included due to their high risk of developing psychosis and people with Bipolar 1 were included where there was evidence of psychotic symptoms. This provides for more comprehensive review; however, these participants may not be representative of people with psychosis and so these studies could be classified separately.
Conclusions and future research
With the development of software and programming, future studies may wish to broaden the range of social environments and interactions in order to achieve greater ecologically validity. Central to this aim will be the development of more interactive, multi-avatar social environments, in which real-time therapeutic treatments for social functioning impairments can be tested and carried out, with key mechanisms identified. As technology evolves, it will be important for researchers to broaden the focus of social functioning to include dynamics within personal relationships. VR studies have already shown that people with psychosis can create believable avatars of people known to them (Craig et al., Citation2018). The next stage in this research may be the development of technology that allows clinicians and programmers to work with people with lived experience to co-create more sophisticated avatars and complex social environments.
VR-based psychological assessment and treatment appear to have potential to improve support people with psychosis to manage difficulties with social functioning. This may have benefits for psychological wellbeing and mood, although studies are limited by the narrow scope of social functioning and by lack of standardisation. Nevertheless, this is an exciting and promising field of research that has potential to advance psychological interventions for people with psychosis.
Supplemental Material
Download Zip (46.3 KB)Disclosure statement
The authors have carried out virtual reality research and have published some of the studies included in this review. Dr Riches has received a grant from the Health Innovation Network for virtual reality research. Professor Garety and Dr Rus-Calafell have received a grant from the Wellcome Trust for Avatar Therapy. Dr Valmaggia is the Head of the VR Lab at the Institute of Psychiatry, Psychology and Neuroscience and she has received grants from the following funding bodies for her VR work: Medical Research Council, Economic and Social Research Council, Psychiatry Research Trust, National Institute for Health Research, NARSAD, and Wellcome Trust.
Additional information
Funding
References
- Addington, J., Penn, D., Woods, S. W., Addington, D., & Perkins, D. O. (2008). Social functioning in individuals at clinical high risk for psychosis. Schizophrenia Research, 99(1–3), 119–124. https://doi.org/10.1016/j.schres.2007.10.001
- Adery, L. H., Ichinose, M., Torregrossa, L., Nichols, H., Bian, D., Wade, J., Granholm, E., Sarkar, N., & Park, S. (2018). Effects of a virtual reality social training intervention on loneliness and social cognition in patients with schizophrenia. Schizophrenia Bulletin, 44(suppl_1), S421–S421. https://doi.org/10.1093/schbul/sby018.1030
- Amado, I., Brénugat-Herné, L., Orriols, E., Desombre, C., Dos Santos, M., Prost, Z., Krebs, M.-O., & Piolino, P. (2016). A serious game to improve cognitive functions in schizophrenia: A pilot study. Frontiers in Psychiatry, 7(APR), 64. https://doi.org/10.3389/fpsyt.2016.00064
- Barkl, S. J., Lah, S., Harris, A. W., & Williams, L. M. (2014). Facial emotion identification in early-onset and first-episode psychosis: A systematic review with meta-analysis. Schizophrenia Research, 159(1), 62–69. https://doi.org/10.1016/j.schres.2014.07.049
- Bekele, E., Bian, D., Peterman, J., Park, S., & Sarkar, N. (2017). Design of a Virtual Reality System for Affect Analysis in Facial Expressions (VR-SAAFE); application to schizophrenia. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(6), 739–749. https://doi.org/10.1109/TNSRE.2016.2591556
- Bell, M. D., & Weinstein, A. (2011). Simulated job interview skill training for people with psychiatric disability: Feasibility and tolerability of virtual reality training. Schizophrenia Bulletin, 37(suppl 2), S91–S97. https://doi.org/10.1093/schbul/sbr061
- Berrada-Baby, Z., Oker, A., Courgeon, M., Urbach, M., Bazin, N., Amorim, M.-A., Martin, J.-C., Passerieux, C., Roux, P., & Brunet-Gouet, E. (2016). Patients with schizophrenia are less prone to interpret virtual others' empathetic questioning as helpful. Psychiatry Research, 242, 67–74. https://doi.org/10.1016/j.psychres.2016.05.022
- Bertrand, M.-C., Sutton, H., Achim, A. M., Malla, A. K., & Lepage, M. (2007). Social cognitive impairments in first episode psychosis. Schizophrenia Research, 95(1–3), 124–133. https://doi.org/10.1016/j.schres.2007.05.033
- Bjornestad, J., Davidson, L., ten Velden Hegelstad, W., Stain, H., & Berg, H. (2019). Re-thinking social functioning in psychosis. Schizophrenia Research, 211, 93–94. https://doi.org/10.1016/j.schres.2019.07.054
- Bosc, M. (2000). Assessment of social functioning in depression. Comprehensive Psychiatry, 41(1), 63–69. https://doi.org/10.1016/S0010-440X(00)90133-0
- Burns, T., & Patrick, D. (2007). Social functioning as an outcome measure in schizophrenia studies. Acta Psychiatrica Scandinavica, 116(6), 403–418. https://doi.org/10.1111/j.1600-0447.2007.01108.x
- Canty, A. L., Neumann, D. L., & Shum, D. H. K. (2017). Using virtual reality to assess theory of mind subprocesses and error types in early and chronic schizophrenia. Schizophrenia Research. Cognition, 10, 15–19. https://doi.org/10.1016/j.scog.2017.09.001
- Caruana, N., Seymour, K., Brock, J., & Langdon, R. (2019). Responding to joint attention bids in schizophrenia: An interactive eye-tracking study. Quarterly Journal of Experimental Psychology, 72(8), 2068–2083. https://doi.org/10.1177/1747021819829718
- Cheniaux, E., Landeira-Fernandez, J., Telles, L. L., Lessa, J. L. M., Dias, A., Duncan, T., & Versiani, M. (2008). Does schizoaffective disorder really exist? A systematic review of the studies that compared schizoaffective disorder with schizophrenia or mood disorders. Journal of Affective Disorders, 106(3), 209–217. https://doi.org/10.1016/j.jad.2007.07.009
- Choi, S. H., Ku, J., Han, K., Kim, E., Kim, S. I., Park, J., & Kim, J. J. (2010). Deficits in eye gaze during negative social interactions in patients with schizophrenia. Journal of Nervous & Mental Disease, 198(11), 829–835. https://doi.org/10.1097/NMD.0b013e3181f97c0d
- Counotte, J., Pot-Kolder, R., Van Roon, A. M., Hoskam, O., Van Der Gaag, M., & Veling, W. (2016). High psychosis liability is associated with altered autonomic balance during exposure to virtual reality social stressors. Early Intervention in Psychiatry, 10(Supplement 1), 20. https://doi.org/10.1111/eip.12395
- Couture, S. M., Penn, D. L., & Roberts, D. L. (2006). The functional significance of social cognition in schizophrenia: A review. Schizophrenia Bulletin, 32(Supplement 1), S44–S63. https://doi.org/10.1093/schbul/sbl029
- Craig, T. K., Rus-Calafell, M., Ward, T., Leff, J. P., Huckvale, M., Howarth, E., Emsley, R., & Garety, P. A. (2018). AVATAR therapy for auditory verbal hallucinations in people with psychosis: A single-blind, randomised controlled trial. The Lancet Psychiatry, 5(1), 31–40. https://doi.org/10.1016/S2215-0366(17)30427-3
- Devoe, D. J., Farris, M. S., Townes, P., & Addington, J. (2019). Interventions and social functioning in youth at risk of psychosis: A systematic review and meta-analysis. Early Intervention in Psychiatry, 13(2), 169–180. https://doi.org/10.1111/eip.12689
- Dietrichkeit, M., Grzella, K., Nagel, M., & Moritz, S. (2020). Using virtual reality to explore differences in memory biases and cognitive insight in people with psychosis and healthy controls. Psychiatry Research, 285, 112787. https://doi.org/10.1016/j.psychres.2020.112787
- Dyck, M., Winbeck, M., Leiberg, S., Chen, Y., & Mathiak, K. (2010). Virtual faces as a tool to study emotion recognition deficits in schizophrenia. Psychiatry Research, 179(3), 247–252. https://doi.org/10.1016/j.psychres.2009.11.004
- Edwards, J., Jackson, H. J., & Pattison, P. E. (2002). Emotion recognition via facial expression and affective prosody in schizophrenia: A methodological review. Clinical Psychology Review, 22(6), 789–832. https://doi.org/10.1016/s0272-7358(02)00130-7
- Ephpp, P. (2010). Quality assessment tool for quantitative studies. Hamilton: The Effective Public Health Practice Project. http://www.ephpp.ca/PDF/Quality%20Assessment
- Fernández-Sotos, P., Fernández-Caballero, A., & Rodriguez-Jimenez, R. (2020). Virtual reality for psychosocial remediation in schizophrenia: A systematic review. The European Journal of Psychiatry, 34(1), 1–10. https://doi.org/10.1016/j.ejpsy.2019.12.003
- Fornells-Ambrojo, M., Barker, C., Swapp, D., Slater, M., Antley, A., & Freeman, D. (2008). Virtual reality and persecutory delusions: Safety and feasibility. Schizophrenia Research, 104(1–3), 228–236. https://doi.org/10.1016/j.schres.2008.05.013
- Freeman, D., Bradley, J., Antley, A., Bourke, E., DeWeever, N., Evans, N., Černis, E., Sheaves, B., Waite, F., Dunn, G., Slater, M., Clark, D. M., et al. (2016). Virtual reality in the treatment of persecutory delusions: Randomised controlled experimental study testing how to reduce delusional conviction. The British Journal of Psychiatry, 209(1), 62–67. https://doi.org/10.1192/bjp.bp.115.176438
- Freeman, D., Lister, R., Waite, F., Yu, L. M., Slater, M., Dunn, G., & Clark, D. (2019). Automated psychological therapy using virtual reality (VR) for patients with persecutory delusions: Study protocol for a single-blind parallel-group randomised controlled trial (THRIVE). Trials, 20(1), 87. https://doi.org/10.1186/s13063-019-3198-6
- Freeman, D., Pugh, K., Vorontsova, N., Antley, A., & Slater, M. (2010). Testing the continuum of delusional beliefs: An experimental study using virtual reality. Journal of Abnormal Psychology, 119(1), 83–92. https://doi.org/10.1037/a0017514
- Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., & Slater, M. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine, 47(14), 2393–2400. https://doi.org/10.1017/S003329171700040X
- Fusar-Poli, P., Bonoldi, I., Yung, A. R., Borgwardt, S., Kempton, M. J., Valmaggia, L., Barale, F., Caverzasi, E., & McGuire, P. (2012). Predicting psychosis: Meta-analysis of transition outcomes in individuals at high clinical risk. Archives of General Psychiatry, 69(3), 220–229. https://doi.org/10.1001/archgenpsychiatry.2011.1472
- Garety, P. A., Kuipers, E., Fowler, D., Freeman, D., & Bebbington, P. (2001). A cognitive model of the positive symptoms of psychosis. Psychological Medicine, 31(2), 189–195. https://doi.org/10.1017/S0033291701003312
- Gayer-Anderson, C., & Morgan, C. (2013). Social networks, support and early psychosis: A systematic review. Epidemiology and Psychiatric Sciences, 22(2), 131–146. https://doi.org/10.1017/S2045796012000406
- Geraets, C. N., van Beilen, M., Pot-Kolder, R., Counotte, J., van der Gaag, M., & Veling, W. (2018). Social environments and interpersonal distance regulation in psychosis: A virtual reality study. Schizophrenia Research, 192, 96–101. https://doi.org/10.1016/j.schres.2017.04.034
- Girvan, C. (2018). What is a virtual world? Definition and classification. Educational Technology Research and Development, 66(5), 1087–1100. https://doi.org/10.1007/s11423-018-9577-y
- Greenwood, K. E., Morris, R., Smith, V., Jones, A.-M., Pearman, D., & Wykes, T. (2016). Virtual shopping: A viable alternative to direct assessment of real life function? Schizophrenia Research, 172(1–3), 206–210. https://doi.org/10.1016/j.schres.2016.02.029
- Gregg, L., & Tarrier, N. (2007). Virtual reality in mental health: A review of the literature. Social Psychiatry and Psychiatric Epidemiology, 42(5), 343–354. https://doi.org/10.1007/s00127-007-0173-4
- Gutierrez-Maldonado, J., Rus-Calafell, M., Marquez-Rejon, S., & Ribas-Sabate, J. (2012). Associations between facial emotion recognition, cognition and alexithymia in patients with schizophrenia: Comparison of photographic and virtual reality presentations. Studies in Health Technology and Informatics, 181, 88–92. https://doi.org/10.3233/978-1-61499-121-2-88
- Han, K., Kim, I. Y., & Kim, J.-J. (2012). Assessment of cognitive flexibility in real life using virtual reality: A comparison of healthy individuals and schizophrenia patients. Computers in Biology and Medicine, 42(8), 841–847. https://doi.org/10.1016/j.compbiomed.2012.06.007
- Han, K., Shin, J., Yoon, S. Y., Jang, D. P., & Kim, J. J. (2014). Deficient gaze pattern during virtual multiparty conversation in patients with schizophrenia. Computers in Biology and Medicine, 49(1), 60–66. https://doi.org/10.1016/j.compbiomed.2014.03.012
- Hartley, S., Barrowclough, C., & Haddock, G. (2013). Anxiety and depression in psychosis: A systematic review of associations with positive psychotic symptoms. Acta Psychiatrica Scandinavica, 128(5), 327–346. https://doi.org/10.1111/acps.12080
- Harvey, P., Khan, A., Atkins, A., & Keefe, R. (2019). Virtual reality functional capacity assessment in patients with schizophrenia: Correlates of performance of solitary and socially relevant tasks. Schizophrenia Bulletin, 45 (Supplement_2)S, S175–S175. https://doi.org/10.1093/schbul/sbz021.216
- Hesse, K., Schroeder, P. A., Scheeff, J., Klingberg, S., & Plewnia, C. (2017). Experimental variation of social stress in virtual reality - Feasibility and first results in patients with psychotic disorders. Journal of Behavior Therapy and Experimental Psychiatry, 56, 129–136. https://doi.org/10.1016/j.jbtep.2016.11.006
- Hirschfeld, R. M. A., Montgomery, S. A., Keller, M. B., Kasper, S., Schatzberg, A. F., Moller, H.-J., Healy, D., Baldwin, D., Humble, M., Versiani, M., Montenegro, R., & Bourgeois, M. (2000). Social functioning in depression: A review. The Journal of Clinical Psychiatry, 61(4), 268–275. https://doi.org/10.4088/JCP.v61n0405
- Humm, L. B., Olsen, D., Be, M., Fleming, M., & Smith, M. (2014). Simulated job interview improves skills for adults with serious mental illnesses. Annual Review of CyberTherapy and Telemedicine, 12, 50–54. https://doi.org/10.3233/978-1-61499-401-5-50
- Jongeneel, A., Pot-Kolder, R., Counotte, J., van der Gaag, M., & Veling, W. (2018). Self-esteem moderates affective and psychotic responses to social stress in psychosis: a virtual reality study. Schizophrenia Research, 202, 80–85. https://doi.org/10.1016/j.schres.2018.06.042
- Keefe, B. D., Wincenciak, J., Jellema, T., Ward, J. W., & Barraclough, N. E. (2016). Action adaptation during natural unfolding social scenes influences action recognition and inferences made about actor beliefs. Journal of Vision, 16(9), 9. https://doi.org/10.1167/16.9.9
- Khan, K. S., Ter Riet, G., Glanville, J., Sowden, A. J., & Kleijnen, J. (2001). Undertaking systematic reviews of research on effectiveness: CRD's guidance for carrying out or commissioning reviews. NHS Centre for Reviews and Dissemination.
- Kim, E., Jung, Y.-C., Ku, J., Kim, J.-J., Lee, H., Kim, S. Y., Kim, S. I., & Cho, H.-S. (2009). Reduced activation in the mirror neuron system during a virtual social cognition task in euthymic bipolar disorder. Progress in Neuro-Psychopharmacology & Biological Psychiatry, 33(8), 1409–1416. https://doi.org/10.1016/j.pnpbp.2009.07.019
- Kim, K., Kim, J.-J., Kim, J., Park, D.-E., Jang, H. J., Ku, J., Kim, C.-H., Kim, I. Y., & Kim, S. I. (2007). Characteristics of social perception assessed in schizophrenia using virtual reality. Cyberpsychology & Behavior, 10(2), 215–219. https://doi.org/10.1089/cpb.2006.9966
- Kim, K., Kim, J., Kim, J., Park, D., Ku, J., Jang, H., & Kim, S. (2005). Investigation of social cue perception in schizophrenia using virtual reality. Annual Review of CyberTherapy and Telemedicine, 3, 135–142. https://doi.org/10.1037/e705572011-052
- Kim, E., Ku, J., Kim, J. J., Lee, H., Han, K., Kim, S. I., & Cho, H. S. (2009). Nonverbal social behaviors of patients with bipolar mania during interactions with virtual humans. The Journal of Nervous and Mental Disease, 197(6), 412–418. https://doi.org/10.1097/NMD.0b013e3181a61c3d
- Ku, J., Han, K., Lee, H. R., Jang, H. J., Kim, K. U., Park, S. H., Kim, J. J., Kim, C. H., Kim, I. Y., & Kim, S. I. (2007). VR-based conversation training program for patients with schizophrenia: A preliminary clinical trial. Cyberpsychology & Behavior, 10(4), 567–574. https://doi.org/10.1089/cpb.2007.9989
- Ku, J., Kim, J., Jang, H., Park, S., Kim, S., Kim, C., & Kim, S. (2005). Relationship between social response to virtual avatar and symptom severity of patients with schizophrenia. Annual Review of CyberTherapy and Telemedicine, 3, 143–149. https://doi.org/10.1037/e705572011-056
- Lake, C. R., & Hurwitz, N. (2006). Schizoaffective disorders are psychotic mood disorders; there are no schizoaffective disorders. Psychiatry Research, 143(2–3), 255–287. https://doi.org/10.1016/j.psychres.2005.08.012
- Lambe, S., Knight, I., Kabir, T., West, J., Patel, R., Lister, R., Rosebrock, L., Rovira, A., Garnish, B., Freeman, J., M. Clark, D., Waite, F., & Freeman, D. (2020). Developing an automated VR cognitive treatment for psychosis: Gamechange VR therapy. Journal of Behavioral and Cognitive Therapy, 30(1), 33–40. https://doi.org/10.1016/j.jbct.2019.12.001
- Lutgens, D., Gariepy, G., & Malla, A. (2017). Psychological and psychosocial interventions for negative symptoms in psychosis: Systematic review and meta-analysis. The British Journal of Psychiatry : The Journal of Mental Science, 210(5), 324–332. https://doi.org/10.1192/bjp.bp.116.197103
- Marcos-Pablos, S., González-Pablos, E., Martín-Lorenzo, C., Flores, L. A., Gómez-García-Bermejo, J., & Zalama, E. (2016). Virtual avatar for emotion recognition in patients with schizophrenia: A pilot study. Frontiers in Human Neuroscience, 10, 12. https://doi.org/10.3389/fnhum.2016.00421
- McDonnell, J., Stahl, D., Day, F., McGuire, P., & Valmaggia, L. (2018). Interpersonal sensitivity in those at clinical high risk for psychosis mediates the association between childhood bullying victimisation and paranoid ideation: A virtual reality study. Schizophrenia Research, 192, 89–95. https://doi.org/10.1016/j.schres.2017.04.029
- Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
- Nijman, S. A., Veling, W., Greaves-Lord, K., Vermeer, R. R., Vos, M., Zandee, C. E. R., Zandstra, D. C., Geraets, C. N. W., & Pijnenborg, G. H. M. (2019). Dynamic Interactive Social Cognition Training in Virtual Reality (DiSCoVR) for social cognition and social functioning in people with a psychotic disorder: Study protocol for a multicenter randomized controlled trial. BMC Psychiatry, 19(1), 272. https://doi.org/10.1186/s12888-019-2250-0
- Oker, A., Prigent, E., Courgeon, M., Eyharabide, V., Urbach, M., Bazin, N., Amorim, M.-A., Passerieux, C., Martin, J.-C., & Brunet-Gouet, E. (2015). How and why affective and reactive virtual agents will bring new insights on social cognitive disorders in schizophrenia? An illustration with a virtual card game paradigm. Frontiers in Human Neuroscience, 9, 133. https://doi.org/10.3389/fnhum.2015.00133
- Osborne, K. J., Vargas, T., & Mittal, V. A. (2020). Early childhood social communication deficits in youth at clinical high-risk for psychosis: Associations with functioning and risk. Development and Psychopathology, 32(2), 559–514. https://doi.org/10.1017/S0954579419000385
- Park, K. M., Kim, J. J., Ku, J., Kim, S. Y., Lee, H. R., Kim, S. I., & Yoon, K. J. (2009). Neural basis of attributional style in schizophrenia. Neuroscience Letters, 459(1), 35–40. https://doi.org/10.1016/j.neulet.2009.04.059
- Park, K. M., Ku, J., Choi, S. H., Jang, H. J., Park, J. Y., Kim, S. I., & Kim, J. J. (2011). A virtual reality application in role-plays of social skills training for schizophrenia: A randomized, controlled trial. Psychiatry Research, 189(2), 166–172. https://doi.org/10.1016/j.psychres.2011.04.003
- Park, S.-H., Ku, J., Kim, J.-J., Jang, H. J., Kim, S. Y., Kim, S. H., Kim, C.-H., Lee, H., Kim, I. Y., & Kim, S. I. (2009). Increased personal space of patients with schizophrenia in a virtual social environment. Psychiatry Research, 169(3), 197–202. https://doi.org/10.1016/j.psychres.2008.06.039
- Park, K. M., Ku, J., Park, I. H., Park, J. Y., Kim, S. I., & Kim, J. J. (2009). Improvement in social competence in patients with schizophrenia: A pilot study using a performance-based measure using virtual reality. Human Psychopharmacology: Clinical and Experimental, 24(8), 619–627. https://doi.org/10.1002/hup.1071
- Park, S., Shin, J., Han, K., Shin, Y. B., & Kim, J. J. (2014). Effects of intimacy on the social decision in patients with schizophrenia. Schizophrenia Research, 153, S225. https://doi.org/10.1016/S0920-9964(14)70647-4
- Pot-Kolder, R. M. C. A., Geraets, C. N. W., Veling, W., van Beilen, M., Staring, A. B. P., Gijsman, H. J., Delespaul, P. A. E. G., & van der Gaag, M. (2018). Virtual-reality-based cognitive behavioural therapy versus waiting list control for paranoid ideation and social avoidance in patients with psychotic disorders: A single-blind randomised controlled trial. The Lancet Psychiatry, 5(3), 217–226. https://doi.org/10.1016/S2215-0366(18)30053-1
- Pot-Kolder, R., Veling, W., Counotte, J., & Van Der Gaag, M. (2017). Self-reported cognitive biases moderate the associations between social stress and paranoid ideation in a virtual reality experimental study. Schizophrenia Bulletin, 44(4), 749–756. https://doi.org/10.1093/schbul/sbx119
- Pot-Kolder, R., Veling, W., Geraets, C., Lokkerbol, J., Smit, F., Jongeneel, A., Ising, H., & van der Gaag, M. (2020). Cost-effectiveness of virtual reality cognitive behavioral therapy for psychosis: Health-economic evaluation within a randomized controlled trial. Journal of Medical Internet Research, 22(5), e17098. https://doi.org/10.2196/17098
- Realpe, A., Elahi, F., Bucci, S., Birchwood, M., Vlaev, I., Taylor, D., & Thompson, A. (2020). Co-designing a virtual world with young people to deliver social cognition therapy in early psychosis. Early Intervention in Psychiatry, 14(1), 37–43. https://doi.org/10.1111/eip.12804
- Reininghaus, U. A., Morgan, C., Simpson, J., Dazzan, P., Morgan, K., Doody, G. A., Bhugra, D., Leff, J., Jones, P., Murray, R., Fearon, P., & Craig, T. K. J. (2008). Unemployment, social isolation, achievement-expectation mismatch and psychosis: Findings from the AESOP Study. Social Psychiatry and Psychiatric Epidemiology, 43(9), 743–751. https://doi.org/10.1007/s00127-008-0359-4
- Riches, S., Bird, L., Chan, N., Garety, P., Rus-Calafell, M., & Valmaggia, L. (2020). Subjective experience of paranoid ideation in a virtual reality social environment: A mixed methods cross-sectional study. Clinical Psychology and Psychotherapy, 27(3), 337–345. https://doi.org/10.1002/cpp.2431
- Riva, G. (2005). Virtual reality in psychotherapy: Review. Cyberpsychology & Behavior: Behavior, 8(3), 220–230. https://doi.org/10.1089/cpb.2005.8.220
- Rus-Calafell, M., Garety, P., Sason, E., Craig, T., & Valmaggia, L. (2018). Virtual reality in the assessment and treatment of psychosis: A systematic review of its utility, acceptability and effectiveness. Psychological Medicine, 48(3), 362–391. https://doi.org/10.1017/S0033291717001945
- Rus-Calafell, M., Gutierrez-Maldonado, J., & Ribas-Sabate, J. (2014). A virtual reality-integrated program for improving social skills in patients with schizophrenia: A pilot study. Journal of Behavior Therapy and Experimental Psychiatry, 45(1), 81–89. https://doi.org/10.1016/j.jbtep.2013.09.002
- Rus-Calafell, M., Gutiérrez-Maldonado, J., & Ribas-Sabaté, J. (2013). Neurocognition, presence and acceptance of a VR programme for psychotic patients: A correlational study. Annual Review of CyberTherapy and Telemedicine, 11, 141–145. https://doi.org/10.3233/978-1-61499-282-0-141
- Schrank, B., Riches, S., Bird, V., Murray, J., Tylee, A., & Slade, M. (2014). A conceptual framework for improving well-being in people with a diagnosis of psychosis. Epidemiology and Psychiatric Sciences, 23(4), 377–387. https://doi.org/10.1017/S2045796013000462
- Seidel, R. J., & Chatelier, P. R. (2013). Virtual Reality, Training’s Future? Perspectives on virtual reality and related emerging technologies (Vol. 6). Springer Science & Business Media.
- Shin, J. E., Choi, S.-H., Lee, H., Shin, Y. S., Jang, D.-P., & Kim, J.-J. (2015). Involvement of the dorsolateral prefrontal cortex and superior temporal sulcus in impaired social perception in schizophrenia. Progress in Neuro-Psychopharmacology & Biological Psychiatry, 58, 81–88. https://doi.org/10.1016/j.pnpbp.2014.12.006
- Slater, M., & Sanchez-Vives, M. V. (2016). Enhancing our lives with immersive virtual reality. Frontiers in Robotics and AI, 3, 74. https://doi.org/10.3389/frobt.2016.00074
- Smith, M. J., Fleming, M. F., Wright, M. A., Jordan, N., Humm, L. B., Olsen, D., & Bell, M. D. (2015). Job offers to individuals with severe mental illness after participation in virtual reality job interview training. Psychiatric Services, 66(11), 1173–1179. https://doi.org/10.1176/appi.ps.201400504
- Smith, M. J., Fleming, M. F., Wright, M. A., Roberts, A. G., Humm, L. B., Olsen, D., & Bell, M. D. (2015). Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophrenia Research, 166(1–3), 86–91. https://doi.org/10.1016/j.schres.2015.05.022
- Smith, M. J., Ginger, E., Wright, M., Wright, K., Bell, M., & Fleming, M. (2014). Virtual reality job interview training. Schizophrenia Research, 153, S186. https://doi.org/10.1007/s10803-014-2113-y
- Smith, M. J., Ginger, E. J., Wright, M., Wright, K., Boteler Humm, L., Olsen, D., Bell, M. D., & Fleming, M. F. (2014). Virtual reality job interview training for individuals with psychiatric disabilities. The Journal of Nervous and Mental Disease, 202(9), 659–667. https://doi.org/10.1097/NMD.0000000000000187
- Smith, M. J., Humm, L. B., Fleming, M. F., Jordan, N., Wright, M. A., Ginger, E. J., Wright, K., Olsen, D., & Bell, M. D. (2015). Virtual reality job interview training for veterans with posttraumatic stress disorder. Journal of Vocational Rehabilitation, 42(3), 271–279. https://doi.org/10.3233/JVR-150748
- Sohn, B. K., Hwang, J. Y., Park, S. M., Choi, J.-S., Lee, J.-Y., Lee, J. Y., & Jung, H.-Y. (2016). Developing a virtual reality-based vocational rehabilitation training program for patients with schizophrenia. Cyberpsychology, Behavior and Social Networking, 19(11), 686–691. https://doi.org/10.1089/cyber.2016.0215
- Song, Y., Xiang, Y.-T., Huang, Y., Wang, X., Wang, X., Zhang, F., Kwan, J. S. H., Chan, O. C., Wang, Z., Ungvari, G. S., Correll, C. U., & Zaroff, C. (2015). Impairments in negative facial emotion recognition in chinese schizophrenia patients detected with a newly designed task. The Journal of Nervous and Mental Disease, 203(9), 718–724. https://doi.org/10.1097/NMD.0000000000000358
- Souto, T., Baista, A., Tavares, D., Queirós, C., & Marques, A. (2013). Facial emotional recognition in schizophrenia: Preliminary results of the Virtual Reality Program for Facial Emotional Recognition. [Reconhecimento emocional de faces na esquizofrenia: Resultados preliminares do Programa de Realidade Virtual para o Reconhecimento Emocional de Faces. Revista de Psiquiatria Clinica, 40(4), 129–134. https://doi.org/10.1590/S0101-60832013000400001
- Souto, T., Silva, H., Leite, A., Baptista, A., Queirós, C., & Marques, A. (2020). Facial emotion recognition: Virtual reality program for facial emotion recognition—A trial program targeted at individuals with schizophrenia. Rehabilitation Counseling Bulletin, 63(2), 79–90. https://doi.org/10.1177/0034355219847284
- Tan, B.-L., Lee, S.-A., & Lee, J. (2018). Social cognitive interventions for people with schizophrenia: A systematic review. Asian Journal of Psychiatry, 35, 115–131. https://doi.org/10.1016/j.ajp.2016.06.013
- Thirioux, B., Tandonnet, L., Jaafari, N., & Berthoz, A. (2014). Disturbances of spontaneous empathic processing relate with the severity of the negative symptoms in patients with schizophrenia: A behavioural pilot-study using virtual reality technology. Brain and Cognition, 90, 87–99. https://doi.org/10.1016/j.bandc.2014.06.006
- Thomas, B., Ciliska, D., Dobbins, M., & Micucci, S. (2004). A process for systematically reviewing the literature: Providing the research evidence for public health nursing interventions. Worldviews on Evidence-Based Nursing, 11(3), 176–184. https://doi.org/10.1111/j.1524-475X.2004.04006.x
- Thompson, A. D., Bartholomeusz, C., & Yung, A. R. (2011). Social cognition deficits and the 'ultra high risk' for psychosis population: A review of literature. Early Intervention in Psychiatry, 5(3), 192–202. https://doi.org/10.1111/j.1751-7893.2011.00275.x
- Thompson, A., Elahi, F., Realpe, A., Birchwood, M., Taylor, D., Vlaev, I., Leahy, F., & Bucci, S. (2020). A feasibility and acceptability trial of social cognitive therapy in early psychosis delivered through a virtual world: The VEEP study. Frontiers in Psychiatry, 11, 219. https://doi.org/10.3389/fpsyt.2020.00219
- Tsang, M. M. Y., & Man, D. W. K. (2013). A virtual reality-based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophrenia Research, 144(1–3), 51–62. https://doi.org/10.1016/j.schres.2012.12.024
- Valmaggia, L. R., Day, F., Garety, P., Freeman, D., Antley, A., Slater, M., Swapp, D., Myin-Germeys, I., & McGuire, P. (2015). Social defeat predicts paranoid appraisals in people at high risk for psychosis. Schizophrenia Research, 168(1–2), 16–22. https://doi.org/10.1016/j.schres.2015.07.050
- Valmaggia, L. R., Freeman, D., Green, C., Garety, P., Swapp, D., Antley, A., Prescott, C., Fowler, D., Kuipers, E., Bebbington, P., Slater, M., Broome, M., & McGuire, P. K. (2007). Virtual reality and paranoid ideations in people with an 'at-risk mental state' for psychosis. The British Journal of Psychiatry, 51(SUPPL. 51), s63–s68. https://doi.org/10.1192/bjp.191.51.s63
- Valmaggia, L. R., Freeman, D., Green, C., Garety, P., Swapp, D., Antley, A., Prescott, C., Fowler, D., Kuipers, E., Bebbington, P., Slater, M., Broome, M., & McGuire, P. K. (2007). Virtual reality and paranoid ideations in people with an 'at-risk mental state' for psychosis. British Journal of Psychiatry, 191(S51), S63–S68. https://doi.org/10.1192/bjp.191.51.s63
- Valmaggia, L., Latif, L., Kempton, M., & Rus-Calafell, M. (2016). Virtual reality in the psychological treatment for mental health problems: An systematic review of recent evidence. Psychiatry Research, 236, 189–195. https://doi.org/10.1016/j.psychres.2016.01.015
- Veling, W., Counotte, J., Pot-Kolder, R., Van Os, J., & Van Der Gaag, M. (2016). Childhood trauma, psychosis liability and social stress reactivity: A virtual reality study. Psychological Medicine, 46(16), 3339–3348. https://doi.org/10.1017/S0033291716002208
- Veling, W., Moritz, S., & van der Gaag, M. (2014). Brave New worlds-review and update on virtual reality assessment and treatment in psychosis. Schizophrenia Bulletin, 40(6), 1194–1197. https://doi.org/10.1093/schbul/sbu125
- Veling, W., Pot-Kolder, R., Counotte, J., & Van Der Gaag, M. (2014). Psychosis liability, paranoia and distress in experimental virtual reality social environments. Schizophrenia Research, 153, S377–S378. https://doi.org/10.1016/S0920-9964(14)71065-5
- Ventura, J., Welikson, T., Ered, A., Subotnik, K. L., Keefe, R. S., Hellemann, G. S., & Nuechterlein, K. H. (2020). Virtual reality assessment of functional capacity in the early course of schizophrenia: Associations with cognitive performance and daily functioning. Early Intervention in Psychiatry, 14(1), 106–114. https://doi.org/10.1111/eip.12831
- Viertiö, S., Tuulio-Henriksson, A., Perälä, J., Saarni, S. I., Koskinen, S., Sihvonen, M., Lönnqvist, J., & Suvisaari, J. (2012). Activities of daily living, social functioning and their determinants in persons with psychotic disorder. European Psychiatry, 27(6), 409–415. https://doi.org/10.1016/j.eurpsy.2010.12.005
- Voges, M., & Addington, J. (2005). The association between social anxiety and social functioning in first episode psychosis. Schizophrenia Research, 76(2–3), 287–292. https://doi.org/10.1016/j.schres.2005.01.001
- Yager, J. A., & Ehmann, T. S. (2006). Untangling social function and social cognition: A review of concepts and measurement. Psychiatry: Interpersonal and Biological Processes, 69(1), 47–68. https://doi.org/10.1521/psyc.2006.69.1.47