Ecological and ethical issues in virtual reality research: A call for increased scrutiny

ABSTRACT

We argue that moral judgment studies currently conducted utilizing virtual reality (VR) devices must confront a dilemma as a result of how virtual environments are designed and how those environments are experienced. We first begin by describing the contexts present in paradigmatic cases of naturalistic moral judgments. We then compare these contexts to current traditional (vignette-based) and VR-based moral judgment research. We show that, contra to paradigmatic cases, vignette-based and VR-based moral judgment research often fails to accurately model the situational features of paradigmatic moral judgments. In particular, we compare and contrast six recent VR studies to support our view that only simulations high in context-realism and perspectival-fidelity can produce “virtually-real experiences.” After analyzing the constituents of a virtually-real experience, we go on to propose guidelines for the creation of VR studies. These guidelines serve two purposes. First, we aim to increase the ecological validity of such studies to advance our understanding of moral judgments. Second, we believe that such guidelines should inform how Institutional Review Boards assess VR research. We show that our guidelines are urgently needed, given the current lax review standards in place.

KEYWORDS:

• Ecological validity
• institutional review boards
• human subjects
• moral psychology
• subject harm
• virtual reality

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. I would like to thank Scott LaBarge, Javiera Perez-Gomez, Miles Elliott, Carl Maggio, Leilan Nishi, attendees of the 3rd Annual Latinx Philosophy Conference held at Rutgers New Brunswick, and two anonymous reviewers for their feedback on this article.

2. This is true even though moral judgments may occasionally be primed in this way (i.e., via e-mail or letter). For such a study to escape the ecological validity question, the study would have to narrow its scope such that it aims to model only those moral judgments prompted by such long-form narratives.

3. We say more about the naturalistic contexts of moral judgments in just a moment.

4. In other words, the subject does not need to realize that she is feeling good and therefore that she is now more likely to rely on stereotypes when she makes judgments.

5. As mentioned earlier, readers who are dubious about the effects or magnitude of implicit bias are free to discard this aspect of the example. What matters is that at least some of the features of a moral judgment are subdoxastic.

6. Darley and Batson (1973) found that the greater time-pressure their subjects were under, the less likely they were to help the homeless confederate.

7. Subjects were, in other words, not capable of imaging the relevant subdoxastic influences present in the experimental context (Ramirez, 2017).

8. For example, a within-subjects design where subjects make several moral judgments either of the same iterated scenario or of several different scenarios will be one in which an accurate representation of the consequences of an individual’s decisions may affect the nature of future decisions. We thank an anonymous reviewer for pushing us here to distinguish between these features and to explain their relative significance.

9. As this example should help to make clear, virtually-real experience is not identical with the concept of immersion or of “presence” as it is understood by psychologists (Ahn et al., 2016; Cummings & Bailenson, 2016). The first-person shooting experience may well be immersive and may include the feeling of “being present” in the environment. However, if the experience contains the sort of context-unreal design elements that we discuss below, such a simulation is unlikely to produce a virtually-real experience (indeed the lack of subject trauma is perhaps the most tell-tale sign that she did not have virtually-real experiences of killing her virtual enemies).

10. As we will show, some structural features of a simulation will also affect its degree of context-realism. For example, the presence of nondiegetic voice-over in a simulation will likely detract both from the perspectival-fidelity of the simulation and from the degree of context-realism present in it.

11. It seems plausible to suggest that perspectival-fidelity will vary as our perspectives change. For example, in a world where most of us make use of augmented reality (AR) devices, a simulation that includes AR meta-information may therefore be more perspectivally-faithful than one without. Additionally, perspectival-fidelity will be dependent on an individual’s mode of conceiving the world. A “color-blind” subject may experience a simulation as higher in perspectival-fidelity than a neurotypical subject. A subject who is deaf may find elements of an experience less faithful than a subject who is not, depending on how well the simulation recreates their own perspectives.

12. Here again we press the distinction between immersion, presence, and virtually-real experience. Won and colleagues (2015) claim, for example, that subjects can successfully feel in control of a virtual lobster body (that the experience is one in which they feel present), we argue that such experiences are unlikely to be experienced as virtually-real. Although presence and virtually-real experience may overlap (a highly present experience is also likely a virtually-real experience) they are not identical.

13. Context-realism is subjective in one important sense. It is dependent on how a particular subject understands the possibilities available to her in the world. For example, a subject who believes in angels and demons is likely to experience a simulation of a Christian supernatural nature as more context-real than a subject who does not believe in angels and demons, belongs to a different religious group, or who is an atheist.

14. Patil et al.'s (2014) study examines many different versions of trolley problem-like judgments (involving flaming cars, sharks, and so on). Though we only examine their trolley problem environment, our concerns with ecological validity extend, in very similar ways, to all of their virtual environments.

15. Although Slater and colleagues’ study is a one-shot within-subject design, the fact that subjects had to make many decisions to shock (or not shock) the virtual learner may appear to make it like Patil et al. (2014) study. However, although in the case of Patil and colleagues we argued that a visual depiction of the bloody consequences of their decisions worked to detract from the context-realism of their environment, we believe the opposite is true in Navarette and colleagues’ environment. It is part of the nature of the Milgram environment itself to test how far subjects are willing to go when delivering increasingly painful shocks to a learner, the same is not the case in trolley problem-like scenarios. We thank an anonymous reviewer for helping us to make this distinction.

16. In our own work, we have found it very difficult (we believe next-to-impossible) to design a context-real and perspectivally faithful VR simulation of the bridge variant of the trolley problem. Specifically, it is difficult to design a version of this variant that does not break context-realism in a way that undermines ecological validity in the ways that Francis et al. (2016) own version does. One pressing question with respect to the bridge variant is how to create a simulation such that subjects spontaneously generate the idea that pushing the man onto the tracks (as the original thought-experiment is written) is a viable option to save the five. By resorting to nondiegetic voice-over to prompt the choice in their subjects, the context-realism and perspectival fidelity of the scenario are both diminished (we conjecture it is diminished to such a degree that it affects the generalizability of the data but this is merely a conjecture). We believe that this speaks to problems embedded in the structure of the bridge variant of the trolley problem itself, though explaining exactly why lies beyond the scope of this paper.

17. Specifically, we argue that TEP should modify how IRBs interpret 45 CFR 46 §46.111 (2009). In particular, we believe that the way that IRBs currently assess risk, as evidenced by the approval of studies like Slater et al.’s (2006) and Sütfeld’s (2017), indicate that VR experiences which include perspectivally faithful and context-real designs involving death or harm to virtual agents are not seen as posing ethically significant harms on participating subjects. The Belmont Report cautions researchers and IRBs to ensure that “the method of ascertaining risks should be explicit, especially where there is no alternative to the use of such vague categories as small or slight risk. It should also be determined whether an investigator’s estimates of the probability of harm or benefits are reasonable, as judged by known facts or other available studies” (National Commission, 1978). We argue that the methods for assessing the probability of harm in such studies is not reasonable and that TEP should be used as a tool to help revise these estimates. We thank an anonymous reviewer for asking us to clarify this aspect of our argument.

18. In realistic-seeming nightmares, for example, we might be relieved, upon waking, to realize that “it was all a dream” and that “none of that really happened.” However, while that fact might diminish (or extinguish) the first sort of harm (the harm of believing that you hurt someone), it often does little to diminish or extinguish the other sort of harm. For example, realistic dreams in which we disrespect our parents, cheat on our partners, or intentionally harm or kill others might continue to bother us long after we awake. If they do this, it is likely because we believe, or are at least worried about the possibility, that we might have learned something ugly or distasteful about the kind of people we are given our behavior in the dream. I thank Javiera Perez-Gomez for pressing me on this issue.

Additional information

Funding

This work was supported by the Markkula Center for Applied Ethics [14013];Oculus Education.

Notes on contributors

Erick Jose Ramirez

Erick Jose Ramirez is assistant professor of philosophy at Santa Clara University. His research centers of theoretical and applied questions regarding empathy. One branch of this research focuses on moral concepts and the role played by empathic distress in producing such concepts. Another branch explores metaphilosophical questions regarding the limitations of empathic perspective taking for philosophical and psychological research that uses thought experiments as a source of data.