The Effects of Walk-in-Place and Overground Walking on the Acquisition of Spatial Information by People With Visual Impairment in Virtual Reality Wayfinding

Abstract

Although virtual reality (VR) wayfinding systems are available for people with visual impairment, there is a lack of studies providing the choice of VR environments can affect their spatial information acquisition. To address these issues, we designed two VR-based wayfinding systems for people with visual impairment. We compared the efficacy of walk-in-place, where the user walks in a virtual environment by walking on a treadmill in the real world, and overground walking, where the user walks in a virtual environment by walking in a constrained analogous real walking space without walls and obstacles. By conducting user experiments on these two VR-based wayfinding systems, we evaluated how people with visual impairments acquired spatial information regarding paths and obstacles. Our results showed that people with visual impairment memorized the paths more accurately after the wayfinding with walk-in-place than with the overground walking. Conversely, they memorized the obstacles on the path more accurately after the wayfinding task with the overground walking than the walk-in-place. Lastly, we discussed the rationale for these results of task on the walk-in-place and the overground walking.

Keywords:

• Virtual reality (VR)
• people with visual impairment
• wayfinding
• spatial memory
• walk-in-place
• overground walking

1. Introduction

People with visual impairment tend to walk only familiar routes, such as always using the same subway exit (Alwi & Noh, 2013). As walking on unfamiliar roads can be dangerous for them, it is desirable to train people with visual impairment in advance about spatial information, such as the route and location of obstacles (Riazi et al., 2016). Although a common way for them to familiarize themselves with the new route is to walk outdoors with a wayfinding assistant or a guide dog, it is neither always feasible nor completely safe. To overcome these limitations, many researchers have developed a virtual reality (VR) wayfinding training systems to help people with visual impairment learn the spatial information of a new road through virtual simulations (Ricci, Boldini, Beheshti, et al., 2023; Ricci, Boldini, Ma, et al., 2023; Siu et al., 2020; Zhang et al., 2020; Zhao et al., 2018). As an extension of previous studies, this study develops two VR wayfinding systems and compares the effectiveness of the systems in the acquisition of spatial information by people with visual impairment.

Researchers have developed a variety of map systems that help people with visual impairment get information about the routes. For example, a Braille map system helps people with visual impairment to tactilely learn the directions with a unique Braille pattern (Ivanchev et al., 2014). The augmented reality map system with a touchpad can recognize the touch and provide verbally audible information about the touched location on the map (Albouys-Perrois et al., 2018; Brock et al., 2015; Wang et al., 2009). Another map system provides ambient sounds at the pointing location. (Papadopoulos et al., 2017). However, these systems require people with visual impairment to transform and apply the spatial information obtained in the 2D map to the 3D real-world situation when they actually walk on the road. Additionally, the spatial information obtained from the 2D map does not provide detailed information that can be known by actually walking on the road, such as the locations of various obstacles.

Through VR, the trained sense of space can be intuitively applied when walking on a real road, and hence, the obstacles, which are difficult to show on a map, can be provided. There are several ways to move in VR, such as by using a controller (Cirio et al., 2013) or trackball (Bozgeyikli et al., 2016), moving their arms (Usoh et al., 1999), or doing walk-in-place (Bozgeyikli et al., 2019). As locomotion methods of our system, we implemented walk-in-place, which enables people to move in an infinite virtual space by dragging the foot on a limited area, but different from a natural gait, and overground walking which applies the original gait, but in the limited space by a real environment. Previous studies presented opinions on two locomotion methods (Kreimeier, Karg, et al., 2020; Kreimeier, Ullmann, et al., 2020) and stated that walk-in-place was less intuitive and required more physical effort, compared to overground walking. Conversely, overground walking was more realistic (Kreimeier, Karg, et al., 2020). However, a comprehensive study is needed on whether there is a difference in memorizing spatial information by people with visual impairment between walk-in-place and overground walking. In this study, we investigated how effectively people with visual impairment were able to acquire spatial information about routes and obstacles after conducting a wayfinding task using walk-in-place and overground walking.

The research questions for this study are as follows.

• Research Question 1: When people with visual impairment memorize the routes after wayfinding task, is there a difference in the accuracy of the memory during walk-in-place and overground walking?

• Research Question 2: When people with visual impairment memorize the obstacles encountered on the route after wayfinding task, is there a difference in the accuracy of the memory during walk-in-place and overground walking?

To answer the first research question, we conducted the first experiment wherein people with visual impairment did wayfinding for five routes either by walk-in-place on the VR treadmill, or overground walking using VR trackers. After the experiment, they answered the route they had walked by touching and choosing one of the four route blocks, which were printed using a 3D printer. To answer the second research question, we conducted the second experiment wherein people with visual impairment did wayfinding for a route with five obstacles either by walk-in-place on the VR treadmill or by overground walking using VR trackers. After the experiment, they were asked to mark the locations of the five obstacles they experienced with bricks, on a Lego^TM block, which represented the route they walked.

In this study, we conducted experiments with two different gait methods: walk-in-place and overground walking, in a virtual environment and studied how different gaits affected ability of people with visual impairment to memorize the routes and obstacles on the route. The results of the experiments showed that people with visual impairment memorized the route information more accurately by walk-in-place than by overground walking. Conversely, they remembered the location of obstacles more accurately by overground walking than by walk-in-place. Based on these results, we discussed how characteristics of the locomotion method had an effect on acquiring spatial information for people with visual impairment.

2. Related studies

We reviewed related studies of VR wayfinding for people with visual impairment.

2.1. Wayfinding for people with visual impairment

The lack of spatial information forces people with visual impairment to move only along familiar paths during wayfinding (Andrade et al., 2021). Therefore, it is necessary to update their cognitive maps, which is a mental image formed by gathering spatial information about the structure of the environment, location of objects, and the spatial relationship between the objects (May et al., 2020). This cognitive map is also a representation of the distance and direction among landmarks, paths, and objects (Fazzi & Petersmeyer, 2001). To walk independently and confidently within an environment, people with visual impairment need a cognitive map of their particular environments (May et al., 2020). Therefore, it is necessary to effectively utilize the visual, auditory, tactile, and kinesthetic senses to acquire spatial information.

Among the various senses, hearing has been used in VR wayfinding systems for people with visual impairment (Siu et al., 2020; Zhao et al., 2018). Zhao et al. produced a VR Canetroller that supports an auditory interface on the cane mainly used by people with visual impairment in virtual wayfinding systems (Zhao et al., 2018). Canetroller was designed to detect and interact with objects based on spatial sound in VR and gain an understanding of their environment. Additionally, several studies show that people with visual impairment can form a cognitive map using tactile information. In the case of Kim et al., people with visual impairment were able to recognize braille on the road utilizing the tactile sense transmitted through a white cane in VR, which confirmed that the information learned from VR could be useful in actual wayfinding (Kim, 2020). In the case of Zhang et al., an MR cane was developed that simultaneously provides auditory and tactile feedback whenever the virtual cane contacts a virtual object (Zhang et al., 2020).

In terms of kinesthetic sense, the wayfinding of people with visual impairment is closely related to the rotational ability of the body (Ferrell, 2011). In efficient walking, the pelvis turns and the legs step forward so that the body continues to face forward. People with visual impairment often have poor body rotational ability (Ferrell, 2011) because their vision cannot contribute to balancing the body (Gipsman, 1981). Therefore, they try to stabilize their bodies either consciously or unconsciously in various ways (Gipsman, 1981). For example, some use shorter steps to reduce the time it takes to balance the body between steps (Gipsman, 1981). Some try dragging their feet or extending their stride length to increase the time their soles are in touch with the ground. Variations like these indicate that it is necessary to compare the effects of the walk-in-place approach with a VR treadmill and of the overground walking approach with a VR tracker in perspective of spatial acquisition. Kreimeier et al. tried to compare the walk-in-place and overground walking from the perspective of subjective evaluations of people with visual impairment (Kreimeier, Karg, et al., 2020). However, it is difficult to find a study that compares the walk-in-place to overground walking in terms of memorizing the path and obstacles, which are important elements in wayfinding task.

2.2. Gait of people with visual impairment

As people walk alone in our proposed system, there are some distinguishable characteristics from walking with a walking assistant or a guide dog. First, people with visual impairment often do not walk straight alone because of veering, which refers to getting out of the intended path in a restricted state of vision (Cornell & Heth, 2004; Guth & LaDuke, 1994, 1995). Veering is caused due to unrecognizable movement noise that occurs during individual gait (Kallie et al., 2007). Veering may cause distorted memory of route and obstacles owing to the difference between the perceived and actual movement. One of the main ways to prevent veering is to use a guide dog or a walking assistant (Ahmetovic et al., 2019); however, veering can only be overcome when they detect their own direction through objects in the surrounding environment (Cornell & Heth, 2004). Second, people with visual impairment also need mobility monitoring, which means focusing on the gait itself to walk safely and efficiently. In the previous study, because of mobility monitoring, people with simulated visual impairment had less obstacle memory than people with normal vision (Rand et al., 2015). One of the solutions for reducing this distraction is to navigate with a walking assistant.

These kinesthetic characteristics of gait are influenced by VR locomotion. For example, overground walking evokes less mobility monitoring than walk-in-place does because overground walking demands the same gait people with visual impairment are used to Kreimeier, Karg, et al., (2020). However, there are lack of studies on how VR locomotion affects the acquisition of spatial information based on these characteristics. However, this study shows how walk-in-place and overground walking affects the acquisition of spatial information for people with visual impairment.

2.3. Spatial sound in VR wayfinding

Existing VR systems have mainly developed visually-oriented display functions, making it difficult for people with visual impairment to access VR systems (Creed et al., 2023; Sylaiou & Fidas, 2022). Therefore, researchers have helped people with visual impairment recognize their spatial location in VR using head-related transfer function (HRTF) (Kolarik et al., 2016; Zhang et al., 2023) –– a 3D acoustic technology used for delivering various spatial information in VR wayfinding systems for people with visual impairment. One of the spatial information with HRTF was an obstacle collision sound evoked by a VR white cane at a location where it collided with an obstacle (Siu et al., 2020; Zhao et al., 2018). Some used HRTF to implement echolocation by which sounds are reflected differently depending on the surrounding VR environment (Andrade et al., 2018; Wu et al., 2017). Echolocation helped people with visual impairment understand the surrounding environment by recognizing various sound reflections (echoes) according to surrounding physical characteristics (Kolarik et al., 2014). Some used HRTF to generate the sound of the destination where people with visual impairment should arrive (Kreimeier & Götzelmann, 2019). Others applied HRTF to the sound of cars passing through virtual streets so that people with visual impairment can recognize virtual cars passing-by, simulating real environments of crosswalk (Thevin et al., 2020).

Similar to previous studies (Andrade et al., 2018; Kreimeier & Götzelmann, 2019; Kreimeier, Karg, et al., 2020; Wu et al., 2017), we used HRTF to implement spatial obstacle sounds in the VR wayfinding system for people with visual impairment. However, unlike previous studies, which used HRTF for the sound of collisions (Siu et al., 2020; Zhao et al., 2018) or for the sound that only indicated the location of the destination (Kreimeier & Götzelmann, 2019; Kreimeier, Karg, et al., 2020), this study used HRTF to the sound generation of a moving guide dog. The guide dog informs people with visual impairment of the direction required for the next step so that they can change directions in real-time.

2.4. VR locomotion for people with visual impairment

Among various locomotion methods in virtual environments, controller-based moving methods allow people with visual impairment to move in a specific direction and speed using various devices such as keyboards (Connors et al., 2014; Maidenbaum et al., 2013), joysticks (Picinali et al., 2014; Sánchez et al., 2012) or Nintendo Nunchuck (Lahav et al., 2018) instead of their legs. Conversely, the teleport-based moving method guides the user to teleport a location that is indicated by the player (i.e., point & teleport). However, the teleport-based locomotion methods rely heavily on visual information, making it difficult for people with visual impairment to use this method (Thevin et al., 2020). Instead, some VR systems apply motion-based locomotion methods, allowing them to move in virtual environments by using their legs (Siu et al., 2020; Thevin et al., 2020; Zhao et al., 2018). Motion-based locomotion methods (e.g., walk-in-place (Lee & Hwang, 2019), redirected walking (Nescher et al., 2016), overground walking (Kreimeier, Karg, et al., 2020)) detect the movement of the head or legs of the player so that they can move in a virtual environment with the gaits they are used to. To use these methods, HTC Vive calculates steps of the player based on their head movement (Lee & Hwang, 2019) or via VR trackers that are attached directly to the leg (Siu et al., 2020; Zhao et al., 2018). In contrast, using a smartphone’s camera and inertial sensors with Google’s ARCore, the head movement can be tracked and calculated while walking in VR (Thevin et al., 2020). Additionally, recent related studies confirm that even people with visual impairment can walk in place with the VR Treadmill (Kreimeier & Götzelmann, 2019; Kreimeier, Karg, et al., 2020). Kreimeier et al. tested four VR mobile devices (Virtuix Omni, Virtualizer, Vive tracker, and Joystick) by people with visual impairment (Kreimeier, Karg, et al., 2020). The Vive tracker was attached to the leg so that participants could move in VR as they would walk in the real world. Virtuix Omni and Virtualizer are VR treadmills people can use during walk-in-place. The participants reported that they felt that Virtuix Omni, which supported walk-in-place, was faster and safer than Vive tracker, which supported overground walking. However, this study simply reports what the participants felt when they walked using given locomotion methods, and it is not known how each walking method influenced their memory about the route. In our study, we used two locomotion methods (walk-in-place and overground walking) with two different locomotion devices (Virtuix Omni and Vive Tracker) for wayfinding task in virtual environments. Through the research, we obtained an objective evaluation for wayfinding as well as a subjective evaluation of the participants using each locomotion method.

3. Implementation

We designed a VR wayfinding system for people with visual impairment using VR locomotion and HRTF. We built two different VR systems: a VR treadmill for walking in place (Virtuix Omni) and a VR tracker for overground walking (Vive tracker). All sounds in both VR locomotion environments have been filtered with head-related transfer function (HRTF) technology, which can lead people with visual impairment in the right direction by the sound of a guide dog.

3.1. Apparatus

In the walk-in-place environment, participants can walk around in place using the Virtuix Omni treadmill (Virtuix, 2022). Conversely, in the overground walking environment, participants can walk on the ground by wearing the Vive tracker (Vive, 2022). A Vive Pro^TM head-mounted display (HMD) was used in both environments to reflect the head movements of the participants. Although people with visual impairment have difficulty viewing the display of the HMD, they still wear the HMD to implement HRTF by determining the position and orientation of their ears. The VR controller is also implemented as a virtual white cane to detect virtual obstacles. Moreover, this controller vibrates when the virtual white cane hits a virtual object, such as a wall or obstacles.

The left of Figure 1 shows the completed appearance of the walk-in-place environment. To develop walk-in-place environment, we used Virtuix Omni as a VR treadmill. There are various VR treadmills such as Kat Walk Mini S (KATVR, 2021), Infinadeck (INFINADECK, 2020), Cyberith Virtualizer (Cyberith, 2021), and SpaceWalkerVR (Keles, 2015). However, we used the Virtuix Omni as it holds the body with a soft belt around the waist and a height-adjustable ring, thereby preventing users from falling or protruding from the treadmill. The Virtuix Omni has a low-friction Omni shoe and a smooth surface Omni body to facilitate easy walk-in-place (i.e., without actual forward movement). Additionally, tracking pods, which are inertial measurement unit (IMU)-based tracking devices comprising a gyroscope, an accelerometer, and a geomagnetic sensor, were attached to the Omni shoes to track foot movements. The sensors detect the speed and direction of the participants’ feet by measuring acceleration and inertia.

Figure 1. The Virtuix Omni and HTC Vive tracker used in the experiments. The red circles on the left and right indicate the IMU sensor used for Virtuix Omni and the HTC Vive tracker.

The right of Figures 1 and 2 show the completed appearance of the overground walking environment. The overground walking was developed using Vive trackers as the VR tracker. Two Vive trackers were used to track the movements of both feet by attaching straps to the user’s ankles. To track the Vive trackers, Vive base stations were installed at both ends of the floor space. With these setups, the VR system with overground walking could track the gait in real space. For the overground walking environment, a 5 × 5 m² actual safety zone was created to minimize the risk of participants falling, and participants were allowed to move within the 3 × 4 m² space.

Figure 2. Base station used for overground walking environment. It tracks the feet of the participant.

3.2. 3D spatial audio system

To implement the 3D spatial audio system, we used Google Resonance Audio. This software development kit (SDK) simulates spatial sound transmitted in the real world, giving the user the effect of hearing the sound from a specific location (Google, 2022). Google Resonance Audio was applied to all sounds of the virtual objects in our wayfinding system, allowing the participants to hear spatial sounds. The wayfinding system included the sound of a guide dog, the sound of interaction with the white cane (wall, floor, obstacle), and the sound of interaction with the avatar (wall, obstacle). During wayfinding, the sound of the guide dog continuously occurs until participants arrive to the destination. The sound of the guide dog was implemented as 3 octaves and C major based on a previous finding by Shi’s study (Shi et al., 2019) to give people with visual impairment a comfortable sound. The characteristic of the sound is monotone and the pitch of the sound is always the same, but the volume and arrival time of the sound to each ear varies depending on the position of the guide dog by HRTF. Through the sound of the guide dog, participants are aware of which direction they should go. The sound of the white cane hitting the floor was recorded by tapping a short and simple wooden stick, while the bumping sound of the wall was indicated by tapping the wooden stick three times in a row. The sound of a collision between obstacles and the stick was implemented depending on the material of the respective obstacles. We used virtual object sounds to represent seven materials: bush, cloth, glass, plastic, stone, iron, and wood. The sound of a collision with a participant used the animation-style bumping sound.

3.3. Software

For wayfinding applications, we created various virtual routes based on space of 3 × 4 m² routes in VR. We used the Unity Omni SDK for the treadmill and the SteamVR for the tracker. We attached the “Audio Listener” object to the virtual avatar and produced the spatial sound depending on the HMD location and direction. To detect collisions between the virtual avatar and the virtual wall, we attached a collider component to the virtual body of the participant to recognize collisions. The guide dog moved to the destination while maintaining a distance of 0.5 m from the participant to prevent a collision with the avatar. We also developed the narrow moveable area for the guide dog so that it may move only along the center of the route. A total of 10 routes were created as wayfinding tasks for Experiment 1. All routes included two to three turns. For Experiment 2, we chose two routes and added five obstacles. We added to all environment objects such as floors, walls, and all obstacles (e.g. bushes, stones, trees, glass, iron, plastic, and cloth) Unity’s collider components to generate each collision sound when they are tapped with the virtual white cane.

4. Experiments

We conducted two experiments to study how the respective locomotion method affected the acquisition of various spatial information. First, we conducted virtual wayfinding task for people with visual impairment to memorize the paths using two different locomotion methods, walk-in-place and overground walking. Second, we conducted virtual wayfinding task for people with visual impairment to memorize obstacles using those two different locomotion methods.

4.1. Participants with visual impairment

A total of sixteen people with visual impairment were recruited. Eleven (V1 to V11) participated in both Experiments 1 and 2. Participants V12 to V16 only participated in Experiment 2. Each experiment was conducted after a sufficient tutorial was provided to familiarize the participants with the locomotion method. Participants were aged between 41 to 79 years (M = 53.43). Four participants had previously experienced VR, while only one (V5) had previous experience on a VR Treadmill.

The navigation habits of participants with visual impairment were studied. Seven participants with visual impairment navigated routes with a wayfinding assistant. Three people with visual impairments relied on their residual vision during navigation. One participant received remote assistance from family members using a smartphone camera to show the surroundings. Additionally, seven of them used a white cane for navigation. One participant used the tactical senses of their feet to check their current position on the memorized route.

For the personal history of rehabilitation, seven participants with visual impairment were trained to use white canes. Seven participants with visual impairment had experienced blindness rehabilitation related to mobility, such as how to walk from one place to another, how to walk around the schoolyard, how to move in case of fire emergencies, and how to get help when lost.

All participants used blindfolds during the experiments to reduce the effect of various visual ability. Detailed information about all participants is shown in Table A1. The experiments were approved by the Institutional Review Board (IRB No. HYU-2019-08-001-1)

4.2. Experiment 1: Comparing walk-in-place and overground walking regarding the memory of the route

The goal of the first experiment was to determine which wayfinding environments between overground walking and walk-in-place is better for people with visual impairment to memorize the path taken. In this experiment, the virtual guide dog continuously informed the participants of the direction along the path with spatial sound. This progressive updating of directions was used considering it is difficult to navigate a route by depending on sounds that only indicate the final destination (Kreimeier & Götzelmann, 2019). In Experiment 1, the participants walked five paths in VR. After walking each path, participants selected one of four 3D-printed objects that accurately represented the path they had walked (see Figure 3). The research question for Experiment 1 was as follows.

Figure 3. The ten routes used in Experiment 1. Participants with visual impairment are presented with four 3D-printed route blocks, and they choose a one block that shows the route they walked during the experiment by touching with hands. For a given path, the path may vary depending on the initial direction the participant is facing; therefore, an arrow is added to the starting point to make the direction right.

• RQ 1: When people with visual impairment memorize the routes after wayfinding task, is there a difference in the accuracy of the memory by walk-in-place and overground walking?

4.2.1. Experiment 1 setting

The basic experimental environment setting was the same as described in the Section 3. We used an additional monitor to continuously monitor the location of the participants in the VR environment (see Figure 4(a)). The view on the monitor was the same as that on the HMD, but the participants could not see that because they were blind-folded (see Figure 4(b)). For reporting the memory of the route they had walked, each path was printed by a 3D printer so that the participants could select the path they had taken through tactile feedback (Figure 3). The ten routes used in Experiment 1 comprised left or right turns two to three times.

Figure 4. (a) The experiment apparatus in the walk-in-place environment. The monitor shows the surrounding environment according to the direction the participant’s head is facing so that the researcher can check the current status. (b) The scene in the monitor. The participants cannot see this scene because they are blind-folded.

4.2.2. Experimental procedure and the collected data

Each participant signed a consent form and answered demographic questions, such as their age and gender. All participants wore eye-patches on both eyes to control errors and variations caused by various levels of visual impairment. Before starting the main experiment, the participants practiced for 20 min to become accustomed to each locomotion method. Then, the participants engaged in wayfinding task on the five paths. Wayfinding task was conducted by walking the given path with the guide dog. After the wayfinding task on each path, 3D-printed block of four different paths were presented to the participants, and they chose one block that represented the route they walked. To minimize the learning effect, the order of the walking paths was randomized by the Latin Square. The order of two locomotion methods, walk-in-place and overground walking, was also counter-balanced. After each wayfinding task, the participants answered the questionnaire. After finishing all the tasks, researchers interviewed participants on what they felt during the experiment. Figure 5 shows a flowchart of Experiment 1.

Figure 5. Experiment Flow of Experiment 1 (orange: first locomotion device experiment, green: second locomotion device experiment).

Correct answer rates, which was calculated by answering five questions given after walking each route with each locomotion method, were used to determine how well participants remembered the route after walking the virtual path. Completion time was the time elapsed while walking the route from start to end. We also collected the number of collisions with the virtual wall. Lastly, the number of errors was measured as the number of facing wrong directions, and the error time was measured as the time taken in the wrong direction.

A questionnaire was used in this research. The questionnaire asked 30 questions in 9 categories. The wayfinding category (W) questions ask whether the wayfinding system is generally useful (Darken & Sibert, 1996; Kalawsky, 1999; Wickens & Baker, 1995; Woodrow & Thomas, 1995). Mobility category (M) questions ask whether the locomotion method is easy to use (Kalawsky, 1999; Slater et al., 1995). The white cane category (C) questions were whether it was easy to find obstacles with the controller (Kalawsky, 1999; Kontarinis & Howe, 1995; Shneiderman & Plaisant, 2010). The audio category (A) questions asked that all collision sounds, and the spatial audio of the guide dog sounds are spatially informative (Gabbard, 1997; Richard et al., 1996; Woodrow & Thomas, 1995). Presence category (P) (Kalawsky, 1999; Witmer & Singer, 1998), ease category (E) (Gabbard, 1997; Kennedy et al., 1993; Zhai et al., 1996), distress category (D)(Kennedy et al., 1993), and chain reaction category (R) (Kennedy et al., 1993) questions asked about the overall usefulness of this VR system from various perspectives. Finally, the satisfaction category (S) questions asked whether the wayfinding task was successful. All items are answered on a 5-point Likert scale. Each item of the questionnaire is listed in Table A2. Additionally, line data which showed how participants walked in the virtual environment was recorded.

4.2.3. Experiment 1 results

We collected data for Experiment 1 from eleven participants. In this study, a within-subject design was used wherein participants were involved in both locomotion environments. Therefore, participants were measured repeatedly over five paths with both locomotion methods. To examine the significant differences between walk-in-place and overground walking, the Wilcoxon signed-rank test was used because of the discrete variables. The correct memory rate for the route is shown on the left side of Figure 6, whereas Figure 7 shows the remaining results from Experiment 1.

Figure 6. Correct memory from experiments 1 and 2 between two locomotion methods. Walk-in-place showed significantly better results than overground walking for memorizing routes. Conversely, overground walking showed significantly better results than walk-in-place for memorizing obstacles.

Figure 7. Results of Experiment 1 between two locomotion methods. (b, c) The number of collisions and errors show significant differences with p < 0.05. (d) The error times show significant differences with p < 0.001.

The rate of correct memory showed a significant difference between ground walking and walk-in-place (Z=-2.041, p = 0.041). Higher memorization accuracy was observed for walk-in-place (M = 71%) than for overground walking (M = 53%). In other measures, the number of collisions for walk-in-place was significantly lower (M = 2.18) than that for overground walking (M = 3.48), (Z=-2.097, p = 0.036). The number of errors for walk-in-place was significantly lower (M = 2.80) than that for overground walking (M = 3.74) (Z = −2.469, p = 0.014). The error time was significantly shorter in walk-in-place (M = 3.83) than that in overground walking (M = 8.64), (Z=-4.790, p < 0.001). According to the questionnaire results, there were no significant differences between the two walking methods.

Additionally, to determine the trade-off between speed and accuracy, we used Spearman correlation with the average speed and correct answer rate for each walking method. The average speed for each participant was the average of the speeds across the five paths, and the speed for each path was calculated as the length of each path divided by the completion time. There was no significant negative correlation between the average speed and correct answer rate in walk-in-place and overground walking in Experiment 1 (Walk-in-place: r_s(9) = −0.269, p = 0.212/Overground walking: r_s(9) = −0.229, p = 0.249).

One interesting finding of line data about how participant walked during Experiment 1 was that V6 continued to attempt to pass through walls while overground walking, indicating that people with visual impairment attempted to walk in a straight line but actually walked in a curved direction. This phenomenon is referred to as veering, which is the gap between perceived and actual walking due to imperceptible movement noise (Kallie et al., 2007). Conversely, V6 did not attempt to pass through a wall while walk-in-place. This result can be interpreted as no veering with walk-in-place, considering the orientation of the participant was fixed by a soft belt around the waist and a height-adjustable ring of the Virtuix Omni. Figure 8 shows how V6 did walk-in-place and overground walking in Experiment 1.

Figure 8. The line data of V6 in Experiment 1. Left image shows the path that V6 walked using walk-in-place and right image shows the path that V6 walked using overground walking.

4.3. Experiment 2: Comparing walk-in-place and overground walking regarding the memory of the obstacles

The goal of the second experiment was to find out whether overground walking or walk-in-place was better in terms of memorizing obstacles on the route in wayfinding task for people with visual impairment. The likelihood of surprise obstacles always exists for people with visual impairment and collision can lead to injury. Therefore, many people with visual impairment use a white cane to check if there are any obstacles in front of them. Accordingly, many researchers have developed a virtual white cane (Siu et al., 2020; Zhang et al., 2020; Zhao et al., 2018). When people with visual impairment recognize an obstacle with a white cane, they notice the presence of the obstacle as well as the obstacle’s material and location. In our system, by vibration and spatial sound on a VR white cane, people with visual impairment can sense the position and material of the obstacle. In Experiment 2, we attempted to determine how well the participants could remember the detected obstacles while wayfinding using the two different locomotion methods. Research question for Experiment 2 was as follows.

• RQ 2: When people with visual impairment memorize the obstacles encountered on the route after wayfinding, is there a difference in the accuracy of the memory by walk-in-place and by overground walking?

4.3.1. Experiment 2 setting

Experiment 2 used the same systems as Experiment 1, but only two routes in Experiment 1 were used and five virtual obstacles were added on those routes. The placement and types of added obstacles are shown in the two images on the left in Figure 9. After wayfinding task, participants expressed their perceived obstacles by placing Lego^TM blocks, as shown in the two images on the right in Figure 9.

Figure 9. The location of obstacles on the paths used in Experiment 2 and the block models used in the evaluation. (a, b) The locations and kinds of obstacles on the paths. (c, d) Participants marked the location of each obstacle using Lego^TM blocks.

4.3.2. Experimental procedures and collected data

Each participant signed the consent form and wore eye patches on both eyes to control errors that could be caused by the various levels of vision impairment among the participants. Before starting the main experiment, the participants had a 20 min practice session, familiarizing them with the environment and obstacles in VR wayfinding. During the experiment, each participant walked one walk-in-place path and one overground walking path for wayfinding task. The order of locomotion methods and path for each method was balanced according to the Latin Square. The main task of the experiment was to memorize the positions of the five obstacles. After wayfinding, participants had to place the blocks on the cloned route indicating the locations of the identified obstacles. We collected data regarding how accurately participants positioned the blocks in both routes. The participants answered the questionnaire after all blocks were placed. Then, the researchers interviewed the participants on what they felt during the experiment. Figure 10 shows the experiment flow of Experiment 2.

Figure 10. Experiment Flow of Experiment 2 (orange: first locomotion device experiment, green: second locomotion device experiment).

Data collection was the same as in Experiment 1. However, the correct memory was collected differently from Experiment 1 by checking whether the position of the detected obstacle was correct. Among the answers, only the answer that detected the obstacle and placed it in the correct location or the answer that not detected the obstacle and placed it considered as the correct answer. Additionally, only when an obstacle was marked at the correct location but not detected, or when an obstacle was detected but marked at the wrong location, it was determined as an incorrect answer.

4.3.3. Experiment 2 results

We conducted the within-subject study with sixteen participants (V1–V16). To examine the significant differences between walk-in-place and overground walking, the Wilcoxon signed-rank test was used because of the discrete variables. To assess obstacle memorization, five researchers individually judged whether all 16 participants responded to the correct locations of the obstacles (See Table A3). Fleiss kappa showed a strong agreement between the assessments of the five raters (κ=.83, p < 0.001). Participants could memorize the position of the obstacles on the route more accurately with overground walking than with walk-in-place. The results for the correct memory of the obstacles are shown on the right side of Figure 6, and the remaining results in Experiment 2 are depicted in Figure 11.

Figure 11. Results of Experiment 2 between two locomotion methods. (b) The number of collisions show significant differences with p < 0.05.

All sixteen participants walked two different paths with five obstacles in each path using walk-in-place and overground walking. Therefore, each environment (walk-in-place and overground walking) had a maximum of 80 correctly recognized obstacles (16 people and 5 obstacles). The results showed that participants correctly recognized 44 obstacles in the walk-in-place environment and 62 obstacles in the overground walking environment. Using the Wilcoxon signed-rank test, the results indicated a significant difference in the number of correctly recognized obstacles between walk-in-place and overground walking experiences (Z=-3, p = 0.003). The rate of correctly recognizing obstacles was higher for overground walking (62/80 = 77.50%) than for walk-in-place (44/80 = 55%). Although the number of collisions for walk-in-place (M = 2.63) was significantly lower than that for overground walking (M = 6.81, Z = −2.219, p = 0.026), other measurements did not show significant differences between the two environments. We found that the overground walking environment allows people with visual impairment to remember obstacles better than the walk-in-place environment. The questionnaire results indicated significant difference in E2 question (i.e., “Fatigue did not increase with time while I was using the device”). Walk-in-place (M = 4.25, SE = 0.250) caused more fatigue in the participants than overground walking (M = 4.69, SE = 0.120) (Z=−2.070, p = 0.038). However, there was no significant difference in the rest of the questions in the questionnaire.

Moreover, we used Spearman correlation with the average speed and correct answer rate for each walking method to determine the trade-off between speed and accuracy. The average speed for each participant was calculated as the length of each path divided by the completion time. While there was no significant negative correlation between the average speed and answer rate in overground walking (r_s(14) = −0.132, p = 0.313), a significant negative correlation between the two variables in walk-in-place was observed in Experiment 2(r_s(14) = −0.453, p = 0.039).

Because participants V1–V11 engaged in Experiments 1 and 2 and V12–V16 participated in Experiment 2, the effect of learning by participating in both experiments on the results of Experiment 2 must be considered. To analyze the learning effect, the answer rates of V1–V11 and V12–V16 in Experiment 2 were compared using the Mann-Whitney U test. The results showed no significant differences in the answer rate among the V1–V11 group and the V12–V16 group in walk-in-place(mean answer rate of V1-V11: 0.51; mean answer rate of V12-V16: 0.64; Comparison in walk-in-place: U = 597.5; p = 0.278) or overground walking(mean answer rate of V1-V11: 0.73; mean answer rate of V12-V16: 0.88; Comparison in overground walking: U = 582.5; p = 0.132).

In the interview, participants mentioned the advantages and disadvantages of the two wayfinding methods. V7 commented that the walk-in-place was fun and gave him a sense of stability. However, the gait in the walk-in-place was unnatural, and the number of steps and distance were difficult to predict. V9 reported that in walk-in-place, it was difficult to predict how far he moved. Participants reported that they could walk more comfortably in overground walking without additional physical burden than walk-in-place. V2 said that overground walking was easier than walking in place because it was easy for him to apply the familiar sense of walking. However, it was difficult to walk in a straight line due to veering. V8 reported that walking in a straight line was somewhat daunting.

5. Discussion

In this study, we confirmed whether the people with visual impairment memorized the path they walked and the obstacles on the path using walk-in-place and overground walking, which are locomotion methods where the people with visual impairment can move in virtual reality using their legs. The major difference between the two methods lies in the gait variability, which refers to the extent to which movements are made during walking deviation (Hollman et al., 2016). The results showed that walk-in-place had lower gait variability than overground walking. The fluctuations of walk-in-place in the time taken to move one step and the maximum tilt of the trunk in the sagittal plane were smaller than those of overground walking (Dingwell et al., 2001; Hollman et al., 2016; Terrier & Dériaz, 2011). As a result of Experiment 1, walk-in-place made people with visual impairment memorize the path more accurately than overground walking, indicating that the lower the variability of gait, such as walk-in-place, the better the route is remembered owing to the low cognitive load characteristics. In an experiment where the cognitive task of answering an array of numbers by subtracting seven from a given random number was performed with walk-in-place and overground walking, the cognitive performance of the walk-in-place environment was significantly better than the performance of the overground walking environment (Penati et al., 2020).

For people with visual impairment who cognitively require more concentration on walking than people with normal vision, the result of Experiment 1 can interpreted as they also felt less cognitive load with walk-in-place than overground walking. The cognitive load for checking their own gait is called mobility monitoring, and this load is required for people with visual impairment owing to reduced visual ability (Rand et al., 2015). This load can be alleviated with a walking assistant. Rand et al. found that among people who had impaired visual ability by low vision glasses, those who memorized obstacles without a walking assistant had lower memory accuracy for obstacles than those who memorized obstacles with the help of a walking assistant (Rand et al., 2015). In our experiment which people with visual impairment walk alone, walk-in-place helps people with visual impairment to memorize the road by lowering the cognitive load on gait. In the interview, V6 mentioned that walk-in-place was able to memorize the route better because this gait method was more stable than overground walking. Additionally, the phenomenon of getting out of the intended path in a restricted state of vision is called veering, which was observed in overground walking in Experiment 1 (Kallie et al., 2007). However, in many existing studies on gait variability of overground walking, it is difficult to research veering because it does not occur in people with normal visual ability in a controlled environment (Ekvall Hansson et al., 2021; Kaipust et al., 2012; Nohelova et al., 2021). Moreover, existing studies on walk-in-place mainly use treadmills that move in a straight line, so it is difficult to study the directional variability of walking (Houdijk et al., 2012; Penati et al., 2020; Smulders et al., 2009). However, Kallie et al. confirmed that people with visual impairment had about 1.3 degrees of variability in walking direction (Kallie et al., 2007). In Experiment 1, overground walking confirmed that veering occurred due to the variability of walking direction, and confirmed that people with visual impairment collided with the wall or missed the dog by veering. For example, V1 veered to the right from a straight line and wandered off by hitting the virtual wall in overground walking environment. Because veering has a characteristic in which the direction people with visual impairment perceive and the actual movement direction are different (Kallie et al., 2007), we confirmed that people with visual impairment had answered the wrong path with overground walking more than walk-in-place. However, in walk-in-place environment, people with visual impairment did not veer from virtual reality and memorized the path more accurately than in an overground walking environment. Walk-in-place in this study was implemented with Virtuix Omni, so it was possible to change direction. Additionally, because the safety parts of the device held the torso of people with visual impairment, they moved in a straight line without veering. V1 went all the way in a straight line with Virtuix Omni, then turned at once to match the turn of the virtual guide dog. This phenomenon is also confirmed in Figure 8, which shows the line data that recorded the walking direction of V6.

In Experiment 2, the obstacle memory accuracy was higher in overground walking than in walk-in-place considering overground walking is easier to proceed obstacle negotiation than walk-in-place. Obstacle negotiation or obstacle avoidance refers to the process of identifying an obstacle and stepping on it or avoiding it (Hill et al., 1997, 1999; Hofstad et al., 2006; Hofstad et al., 2009; Vrieling et al., 2007). Obstacle avoidance is one of the characteristics to be considered in wayfinding because it can prevent falls and injuries caused by obstacles. To avoid an obstacle while walking, it is necessary to reduce the speed and step size while walking (Galna et al., 2009; McFadyen & Prince, 2002). In overground walking, it is easy to change the speed and reduce the step size during walking, but it is not easy to control the gait speed and stride length in walk-in-place because the ability to control both gait speed and stride length is limited (Nevisipour et al., 2023). In this study, the locomotion difference between walk-in-place and overground walking seemed to have affected the perception of obstacles. By analyzing the video of wayfinding, it was confirmed that V12 reacted differently to obstacles according to each locomotion method in the process of navigating obstacles. When using overground walking, V12 identified the location of the obstacle by striking it a few more times where it heard the sound of the cane hitting it. But when walk-in-place was used, V12 kept going forward despite the sound of the cane hitting an obstacle.

The results of Experiment 2 can be explained by the differences in the vestibular feedback that each walking method can support. Overground walking provides more accurate vestibular feedback for calculating the position than walk-in-place does because the vestibular feedback of feet movement in walk-in-place makes it difficult to estimate the length that the participants have walked, as reported by V9. V9 stated that walk-in-place made it difficult to predict how far one had moved. A related study reported that people with visual impairment are required to change their motion-based perception of position and apply it to their own spatial mechanisms (Seemungal et al., 2007). In other words, if people with visual impairment do not receive correct or sufficient vestibular feedback to estimate length, their ability to infer distance is impaired. Experiment 2 required complex vestibular information to calculate the relative positions of the obstacles. The lack of translational vestibular information from walk-in-place results in less accurate memorization of obstacle positions compared to overground walking.

The results confirmed that the characteristics of the locomotion method had an effect on memorizing roads and obstacles. First, because walk-in-place has lower mobility monitoring than overground walking, it makes people with visual impairment pay less attention to walking, helping them memorize the route better. Conversely, in memorizing obstacles, overground walking is easier to control the walking speed and length to avoid obstacles than walk-in-place, so it helps take action to perceive the position of an obstacle.

This result shows that walk-in-place is a better alternative for acquiring route information than overground walking. As walk-in-place is not affected by real-space constraints, people with visual impairment can walk in unlimited virtual environments. Further, it requires less real estate to develop a virtual environment than does overground walking. However, overground walking is a better choice for acquiring detailed spatial information (e.g., obstacle position). Compared with walk-in-place, overground walking was closer to real-world walking, and participants could remember obstacle positions more accurately with overground walking than with walk-in-place in this study.

These results can be extended to construct cognitive maps or graphs (Peer et al., 2021). In the context of the research by Peer et al., a cognitive map represents spatial information based on Euclidean coordinates such as the latitude and longitude. The cognitive graph represents the location based on distinct links between local cues. The task of Experiment 1 required building a cognitive map that relied heavily on a Euclidean grid by memorizing the order of rotation directions. However, the task in Experiment 2 required creating a cognitive graph to memorize the positions of route landmarks that were obstacles by linking information together. This result implies that walk-in-place is better than overground walking for memorizing cognitive maps such as grid-like cities. Otherwise, overground walking is superior to walk-in-place walking in terms of memorizing cognitive graphs such as cities with complex buildings.

6. Limitations and future work

A limitation of our virtual guide dog is that it is different from a real guide dog in its functionality. Our virtual guide dog continuously produces spatial sounds to assist people with visual impairment in determining their direction, whereas a real guide dog only produces sounds to alert individuals of immediate danger (e.g., approaching cars or people) or if it is separated from the owner. Additionally, a real guide dog can provide force feedback via a harness to guide them, which our virtual guide dog cannot provide. As a suggestion for future studies, we propose the development of a virtual guide dog that can provide force feedback to assist people with visual impairment in navigation. Another limitation is that all participants were blindfolded during wayfinding, contrary to real-life scenarios. The reason for blindfolding was to create a consistent visual environment for comparing the two walking methods. Therefore, this study could be extended to compare people with various visual conditions, including those who have been blind for a few years and those who have been chronically blind without a blindfold. The questionnaire used herein lacked validity and unidimensionality. A new questionnaire was created by adopting questions from previous studies to analyze the perceived feelings concerning each walking method. However, due to the limited number of participants (16), it was difficult to proceed with exploratory or confirmatory factor analysis and Rasch analysis, which estimate the reliability of items and participants. Therefore, we suggest conducting the experiment with a larger number of participants in future studies. With a sufficient number of participants, Rasch analysis can be employed to verify the psychological characteristics of respondents, appropriateness and difficulty of questions, and suitability of response categories based on item response theory (IRT). We suggest conducting a study to compare different walking methods that restrict various parts of the vestibular feedback to acquire spatial information. In this context, a previous study demonstrated the utility of various walking methods in virtual reality, such as stepper machines or redirected walking, but did not compare their effectiveness in terms of memorizing spatial information (Bozgeyikli et al., 2019).

7. Conclusion

This study compares two locomotion methods (walk-in-place and overground walking) in VR wayfinding systems for people with visual impairment to acquire spatial information (routes and obstacles). In this system, the people with visual impairment acquire spatial information about the virtual environment through wayfinding by moving their legs. The results showed that while people with visual impairment memorized routes better with walk-in-place than overground walking due to less cognitive load on mobility monitoring and less veering in walk-in-place. On the other hand, they memorized the location of obstacles better with overground walking than walk-in-place owing to better obstacle negotiation and obstacle avoidance ability in overground walking. Veering in overground walking causes a distorted cognitive map owing to the difference between the perceived and actual movements. Difficulty of controlling gait speed and step length in walk-in-place makes obstacle perception hard due to abnormal gait by Virtuix Omni. Based on these results, we suggest that walk-in-place is effective for memorizing a route whereas overground walking is effective for memorizing an obstacle. We believe our findings will be helpful to those designing VR wayfinding systems for people with visual impairment.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (No. 2018R1D1A1A02085645). This research is also supported by the Culture, Sports and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture, Sports and Tourism in 2023 (Project Number: RS-2023-00224524).

Notes on contributors

Sangsun Han

Sangsun Han is formerly ME-PhD student in Human-Computer Interaction from Hanyang University. After graduation, he is currently the post-doc researcher in bionics research center of Korea Institutes of Science and Technology. He is interested in digital therapies based on XR(eXtended Reality).

Pilhyoun Yoon

Pilhyoun Yoon graduated from Hanyang University’s HCI PhD program and is currently a professor at Inha Technical college of Digital Marketing Engineering. His research interests include authentic learning, virtual learning, and distance education using VR and AR.

Xiangshi Ren

Xiangshi Ren is a lifetime tenured professor in the School of Information and founding director of the Center for Human-Engaged Computing (CHEC) at Kochi University of Technology, Japan. His research interests include all aspects of Human-Computer Interaction and Human-Engaged Computing (HEC).

Kibum Kim

Kibum Kim is a full professor in the Department of Human-Computer Interaction at Hanyang University in South Korea. He researches in the areas of HCI, CSCW (Computer-Supported Cooperative Work), CSCL (Computer-Supported Collaborative Learning), VR, and computer education.

Appendices

Table A1. Demographic information of participants with visual impairment.

ID	Age/Sex	Description of the visual condition
V1	63/M	Low vision for 25 years (with light perception)
V2	47/M	Blind for 3 years
V3	58/M	Low vision for almost 30 years (with shape perception)
V4	74/M	Low vision for 43 years (with light perception)
V5	72/M	Blind for 60 years
V6	58/F	Blind for 29 years
V7	54/F	Low vision from birth (with shape perception)
V8	50/F	Low vision from birth (with shape perception)
V9	60/M	Low vision from birth (with shape perception), no vision perception from left eye
V10	59/F	Blind for 15 years
V11	67/F	Low vision for 18 years (with light perception)
V12	41/F	Blind for 11 years
V13	58/M	Low vision for 10 years (with shape perception), no vision perception from right eye
V14	45/F	Low vision for 20 years (with light and shape perception)
V15	79/M	Low vision for 15 years, no vision perception from left eye
V16	50/M	Low vision for 17 years (with shape perception)

Table A2. Questionnaire with 5-point likert scale used in both experiments.

Questions		Strongly disagree	Disagree	Neutral	Agree	Strongly agree
W1.	I wasn’t confused in the virtual wayfinding environment during use.	1	2	3	4	5
W2.	There were very few cases where I didn’t know my location during use (Kalawsky, 1999).	1	2	3	4	5
W3.	There is a function to help you know your location(Darken & Sibert, 1996).	1	2	3	4	5
W4.	The virtual wayfinding environment gives information that helps me to where I want to go (Wickens & Baker, 1995).	1	2	3	4	5
W5.	The audio feedback was helpful to get information about the direction of the guide dog (Woodrow & Thomas, 1995).	1	2	3	4	5
M1.	It was easy to change or rearrange position (Kalawsky, 1999).	1	2	3	4	5
M2.	My movement was not restricted by the exercise equipment (treadmill, tracker).(Slater et al., 1995).	1	2	3	4	5
M3.	Moving in virtual wayfinding environment was free enough (Kalawsky, 1999).	1	2	3	4	5
C1.	It was easy to use and control the input device (Controller) (Kalawsky, 1999).	1	2	3	4	5
C2.	The input device (Controller) was easy to use so that I understood the spatial state of the virtual cane and myself (Shneiderman & Plaisant, 2010).	1	2	3	4	5
C3.	The feedback (feeling) after using the input device (Controller) was well presented to me (Kontarinis & Howe, 1995).	1	2	3	4	5
A1.	The auditory output was perfectly integrated with my activity in VR (Gabbard, 1997).	1	2	3	4	5
A2.	The auditory output had no lag or inconvenient characteristics (Richard et al., 1996).	1	2	3	4	5
A3.	The auditory output was meaningful, timely, and useful (Woodrow & Thomas, 1995).	1	2	3	4	5
P1.	I felt that I was part of VR while using the VR device (Kalawsky, 1999).	1	2	3	4	5
P2.	I was able to use the controller naturally and to react to the virtual environment fluidly (Witmer & Singer, 1998).	1	2	3	4	5
P3.	I thought that the VR experience in this user study was similar to what I had experienced before (Witmer & Singer, 1998).	1	2	3	4	5
E1.	I did not feel significant discomfort while using the device (Gabbard, 1997).	1	2	3	4	5
E2.	Fatigue did not increase with time while I was using the device (Gabbard, 1997).	1	2	3	4	5
E3.	Overwhelming physical fatigue was not evoked while using device (Zhai et al., 1996).	1	2	3	4	5
D1.	I did not feel overwhelming discomfort while in VR (Kennedy et al., 1993).	1	2	3	4	5
D2.	I did not feel any headache while in VR (Kennedy et al., 1993).	1	2	3	4	5
D3.	I did not sweat a lot (Kennedy et al., 1993).	1	2	3	4	5
D4.	I did not feel nauseous (Kennedy et al., 1993).	1	2	3	4	5
R1.	I want to use this VR system again	1	2	3	4	5
R2.	I did not feel any headache or nausea after using the VR device	1	2	3	4	5
R3.	I did not experience vertigo (loss of orientation to vertical upright) after using the VR device (Kennedy et al., 1993)	1	2	3	4	5
S1.	The performance of virtual wayfinding for spatial acquisition was successful	1	2	3	4	5
S2.	It seems that the wayfinding ability improved through this VR system	1	2	3	4	5
S3.	I would like to recommend this system to other people with visual impairment.	1	2	3	4	5

Table A3. Individual results of obstacle detection (0: incorrect; 1: correct).

Treadmill		Evaluators
Participant	Obstacle	A	B	C	D	E
V1	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	1	1	1	1	1
	5	1	1	1	1	1
V2	1	0	1	0	0	0
	2	1	1	1	1	1
	3	1	1	1	1	1
	4	0	0	0	0	0
	5	0	0	0	0	0
V3	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	1	0	1
V4	1	0	0	0	0	0
	2	0	0	0	0	0
	3	1	0	1	1	1
	4	1	1	1	1	1
	5	0	0	0	0	0
V5	1	0	1	0	0	0
	2	1	1	1	1	1
	3	1	1	1	1	1
	4	0	1	1	1	1
	5	0	0	0	0	0
V6	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V7	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V8	1	0	0	0	0	0
	2	0	1	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V9	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	1	1	1	1	1
	5	0	0	0	0	0
V10	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V11	1	0	0	0	0	0
	2	1	1	1	1	1
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V12	1	0	0	0	0	0
	2	0	0	0	0	0
	3	1	1	1	1	1
	4	0	0	0	0	0
	5	0	0	0	0	0
V13	1	1	1	1	1	1
	2	1	1	1	1	1
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V14	1	0	0	0	0	0
	2	0	0	0	0	0
	3	1	1	0	0	1
	4	0	0	0	0	0
	5	0	0	0	0	0
V15	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	1	0
	4	1	1	0	0	1
	5	0	0	0	0	0
V16	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
Tracker		Evaluators
Participant	Obstacle	A	B	C	D	E
V1	1	0	0	0	1	0
	2	1	0	1	1	1
	3	1	1	1	1	1
	4	0	0	0	0	0
	5	0	0	0	0	0
V2	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	1	1	1	1	1
	5	0	0	0	0	0
V3	1	0	0	0	0	0
	2	0	1	0	0	0
	3	1	0	1	1	1
	4	1	1	1	1	1
	5	0	0	0	0	0
V4	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	1	0	0
	5	0	0	1	1	1
V5	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V6	1	0	0	0	0	0
	2	1	1	1	1	1
	3	0	0	0	0	0
	4	1	1	1	1	1
	5	0	0	1	1	1
V7	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V8	1	0	0	0	0	0
	2	0	0	0	0	0
	3	1	1	1	1	1
	4	0	0	0	0	0
	5	0	0	0	0	0
V9	1	0	0	0	0	0
	2	1	1	1	1	1
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V10	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	1	1	1	1	1
	5	0	0	0	0	0
V11	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V12	1	0	0	1	1	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	1	1	1	0	1
	5	1	1	1	1	1
V13	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V14	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	1	1	1	1	1
	5	0	0	0	0	0
V15	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0
V16	1	0	0	0	0	0
	2	0	0	0	0	0
	3	0	0	0	0	0
	4	0	0	0	0	0
	5	0	0	0	0	0