The Effect of Audiovisual Spatial Design on User Experience of Bare-Hand Interaction in VR

References

• Alkemade, R., Verbeek, F. J., & Lukosch, S. G. (2017). On the efficiency of a VR hand gesture-based interface for 3D object manipulations in conceptual design. International Journal of Human-Computer Interaction, 33(11), 882–901. https://doi.org/10.1080/10447318.2017.1296074
   Web of Science ®Google Scholar
• Andre, C. R., Corteel, E., Embrechts, J.-J., Verly, J. G., & Katz, B. F. G. (2014). Subjective evaluation of the audiovisual spatial congruence in the case of stereoscopic-3D video and Wave Field Synthesis. International Journal of Human-Computer Studies, 72(1), 23–32. https://doi.org/10.1016/j.ijhcs.2013.09.004
   Web of Science ®Google Scholar
• Argelaguet, F., Hoyet, L., Trico, M., Lecuyer, A. (2016). The role of interaction in virtual embodiment: Effects of the virtual hand representation. 2016 IEEE Virtual Reality (VR), 3–10. https://doi.org/10.1109/VR.2016.7504682
   Google Scholar
• Bavelier, D., Dye, M. W. G., & Hauser, P. C. (2006). Do deaf individuals see better? Trends in Cognitive Sciences, 10(11), 512–518. https://doi.org/10.1016/j.tics.2006.09.006
   PubMed Web of Science ®Google Scholar
• Chen, Q. (2012). Cognitive neural mechanisms of three-dimensional spatial attention. 2012 Abstracts of the 15th National Academic Congress of Psychology, 56. https://kns.cnki.net/KCMS/detail/detail.aspx?dbcode=CPFD&dbname=CPFD0914&filename=ZGXG201211001012&v=
   Google Scholar
• Cummings, J. J., & Bailenson, J. N. (2016). How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology, 19(2), 272–309. https://doi.org/10.1080/15213269.2015.1015740
   Web of Science ®Google Scholar
• Fan, L., Wang, S.-Y., Wang, Y.-Q., Zou, X., & Dou, J.-S. (2011). A new species of the genus Phyllostachys (Coleoptera, Staphylinidae) from China. Ergonomics and cognitive load study of fNIRS-based AR-guided puncture training system. Packaging Engineering, 42(20), 146–151. https://doi.org/10.19554/j.cnki.1001-3563.2021.20.014
   Google Scholar
• Fechter, M., Schleich, B., & Wartzack, S. (2022). Comparative evaluation of WIMP and immersive natural finger interaction: A user study on CAD assembly modeling. Virtual Reality, 26(1), 143–158. https://doi.org/10/gpqb99
   Web of Science ®Google Scholar
• Finnegan, D. J., O’Neill, E., Proulx, M. J. (2016). Compensating for distance compression in audiovisual virtual environments using incongruence. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 200–212. https://doi.org/10/gppv7s
   Google Scholar
• Fu, Q., Lv, J., Zhao, Z., & Yue, D. (2020). Research on optimization method of VR Task Scenario resources driven by user cognitive needs. Information, 11(2), 64. https://doi.org/10/gppskh
   Google Scholar
• Geiger, A., Bewersdorf, I., Brandenburg, E., & Stark, R. (2018). Visual feedback for grasping in virtual reality environments for an interface to instruct digital human models. In T. Ahram & C. Falcão (Eds.), Advances in usability and user experience (Vol. 607, pp. 228–239). Springer International Publishing. https://doi.org/10.1007/978-3-319-60492-3_22
   Google Scholar
• Godfroy, M., Roumes, C., & Dauchy, P. (2003). Spatial variations of visual – Auditory fusion areas. Perception, 32(10), 1233–1245. https://doi.org/10.1068/p3344
   PubMed Web of Science ®Google Scholar
• Herff, C., Heger, D., Fortmann, O., Hennrich, J., Putze, F., & Schultz, T. (2013). Mental workload during n-back task – Quantified in the prefrontal cortex using fNIRS. Frontiers in Human Neuroscience, 7(2013), 935. https://doi.org/10.3389/fnhum.2013.00935
   PubMed Web of Science ®Google Scholar
• Izzetoglu, K., Bunce, S., Izzetoglu, M., Onaral, B., Pourrezaei, K. (2003). FNIR spectroscopy as a measure of cognitive task load. Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439), 3431–3434. https://doi.org/10/cx2c95
   Google Scholar
• Jeong, K., Lee, J., & Kim, J. (2018). A study on new virtual reality system in Maze Terrain. International Journal of Human-Computer Interaction, 34(2), 129–145. https://doi.org/10.1080/10447318.2017.1331535
   Web of Science ®Google Scholar
• Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C. V., & Cramer, S. C. (2014). Free-hand interaction with leap motion controller for stroke rehabilitation. CHI ’14 Extended Abstracts on Human Factors in Computing Systems, 1663–1668. https://doi.org/10.1145/2559206.2581203
   Google Scholar
• Kim, H., & Choi, Y. (2019). Performance comparison of user interface devices for controlling mining software in virtual reality environments. Applied Sciences, 9(13), 2584. https://doi.org/10.3390/app9132584
   Google Scholar
• Kovács, E., & Antal, M. (2012). Challenges of virtual environment in education. EDULEARN12 Proceedings, 2109–2117.
   Google Scholar
• Lewald, J., Ehrenstein, W. H., & Guski, R. (2001). Spatio-temporal constraints for auditory–visual integration. Behavioural brain Research, 121(1–2), 69–79. https://doi.org/10.1016/S0166-4328(00)00386-7
   PubMed Web of Science ®Google Scholar
• Lewis, J. R. (2015). Introduction to the special issue on usability and user experience: Methodological evolution. International Journal of Human-Computer Interaction, 31(9), 555–556. https://doi.org/10.1080/10447318.2015.1065689
   Web of Science ®Google Scholar
• Li, Q., Feng, J., Guo, J., Wang, Z., Li, P., Liu, H., & Fan, Z. (2020). Effects of the multisensory rehabilitation product for home-based hand training after stroke on cortical activation by using NIRS methods. Neuroscience Letters, 717(2020), 134682. https://doi.org/10.1016/j.neulet.2019.134682
   PubMed Web of Science ®Google Scholar
• Lin, J., Yang, W., Gao, X., & Liao, M. (2015). Learning to assemble building blocks with a leap motion controller. In F. W. B. Li, R. Klamma, M. Laanpere, J. Zhang, B. F. Manjón, & R. W. H. Lau (Eds.), Advances in Web-Based Learning—ICWL 2015. (Vol. 9412, pp. 258–263). Springer International Publishing. https://doi.org/10.1007/978-3-319-25515-6_25
   Google Scholar
• Liu, D., & Rau, P.-L P. (2020). Spatially incongruent sounds affect visual localization in virtual environments. Attention, Perception & Psychophysics, 82(4), 2067–2075. https://doi.org/10/gppv8j
   PubMed Web of Science ®Google Scholar
• Lv, Z., Zhou, Y., Liu, H., Shu, X., Zhang, N. (2020). A TCN-based primary ambient extraction in generating ambisonics audio from Panorama Video. 2020 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), 1–6. https://doi.org/10/gpp3jn
   Google Scholar
• Masurovsky, A., Chojecki, P., Runde, D., Lafci, M., Przewozny, D., & Gaebler, M. (2020). Controller-free hand tracking for grab-and-place tasks in immersive virtual reality: Design elements and their empirical study. Multimodal Technologies and Interaction, 4(4), 91. https://doi.org/10/gppn28
   Google Scholar
• McKendrick, R., Parasuraman, R., Murtza, R., Formwalt, A., Baccus, W., Paczynski, M., & Ayaz, H. (2016). Into the Wild: Neuroergonomic differentiation of hand-held and augmented reality wearable displays during outdoor navigation with functional near infrared spectroscopy. Frontiers in Human Neuroscience, 10(2016), 216. https://doi.org/10.3389/fnhum.2016.00216
   PubMed Web of Science ®Google Scholar
• McMahan, R. P., Lai, C., & Pal, S. K. (2016). Interaction fidelity: The Uncanny valley of virtual reality interactions. In S. Lackey & R. Shumaker (Eds.), Virtual, augmented and mixed reality (Vol. 9740, pp. 59–70). Springer International Publishing. https://doi.org/10.1007/978-3-319-39907-2_6
   Google Scholar
• Sagnier, C., Loup-Escande, E., Lourdeaux, D., Thouvenin, I., & Vallery, G. (2020). User acceptance of virtual reality: An extended technology acceptance model. International Journal of Human-Computer Interaction, 36(11), 993–1007. https://doi.org/10.1080/10447318.2019.1708612
   Web of Science ®Google Scholar
• Sato, M., Sato, I., Kato, S., & Kaufman, A. E. (2019). Examination of perception with auditory stimuli when grasping a virtual soft object with a bare hand. In P. Y. Lau, K. Hayase, Q. Kemao, W.-N. Lie, Y.-L. Lee, S. Srisuk, & L. Yu (Eds.), International Workshop on Advanced Image Technology (IWAIT) 2019 (p. 17). SPIE. https://doi.org/10.1117/12.2520986
   Google Scholar
• Scholkmann, F., Kleiser, S., Metz, A. J., Zimmermann, R., Pavia, J. M., Wolf, U., & Wolf, M. (2014). A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology 22.
   Google Scholar
• Shattuck, D. W., Mirza, M., Adisetiyo, V., Hojatkashani, C., Salamon, G., Narr, K. L., Poldrack, R. A., Bilder, R. M., & Toga, A. W. (2008). Construction of a 3D probabilistic atlas of human cortical structures. NeuroImage, 39(3), 1064–1080. https://doi.org/10.1016/j.neuroimage.2007.09.031
   PubMed Web of Science ®Google Scholar
• Simon, H. A. (1981). Information-processing models of cognition. Journal of the American Society for Information Science, 32(5), 364–377. https://doi.org/10.1002/asi.4630320517
   Google Scholar
• Spence, C. (2013). Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule: Spatial coincidence and multisensory integration. Annals of the New York Academy of Sciences, 1296(1), 31–49. https://doi.org/10.1111/nyas.12121
   PubMedGoogle Scholar
• Sreng, J., Lecuyer, A., Megard, C., & Andriot, C. (2006). Using visual cues of contact to improve interactive manipulation of virtual objects in industrial assembly/maintenance simulations. IEEE transactions on Visualization and Computer Graphics, 12(5), 1013–1020. https://doi.org/10/c52kgw
   PubMed Web of Science ®Google Scholar
• Sun, C.-Y., & Liu, D.-Z. (2013). Comparison of subjective rating scales for cognitive load. Psychological Science, 36(1), 194–201. https://doi.org/10.16719/j.cnki.1671-6981.2013.01.014
   Google Scholar
• Tsang, P. S., & Velazquez, V. L. (1996). Diagnosticity and multidimensional subjective workload ratings. Ergonomics, 39(3), 358–381. https://doi.org/10.1080/00140139608964470
   PubMed Web of Science ®Google Scholar
• Turner, A., Berry, J., & Holliman, N. (2011). Can the perception of depth in stereoscopic images be influenced by 3D sound? (A. J. Woods, N. S. Holliman, & N. A. Dodgson, Eds., Proceedings Volume 7863, Stereoscopic Displays and Applications XXII, p. 786307). https://doi.org/10.1117/12.871960
   Google Scholar
• Uga, M., Dan, I., Sano, T., Dan, H., & Watanabe, E. (2014). Optimizing the general linear model for functional near-infrared spectroscopy: An adaptive hemodynamic response function approach. Neurophotonics, 1(1), 015004. https://doi.org/10.1117/1.NPh.1.1.015004
   PubMed Web of Science ®Google Scholar
• Van der Stoep, N., Nijboer, T. C. W., & Van der Stigchel, S. (2013). Exogenous orienting of crossmodal attention in 3-D space: Support for a depth-aware crossmodal attentional system. Psychonomic Bulletin & Review 21(3), 708–714. https://doi.org/10.3758/s13423-013-0532-y
   Web of Science ®Google Scholar
• Vosinakis, S., & Koutsabasis, P. (2018). Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift. Virtual Reality, 22(1), 47–62. https://doi.org/10/gppsjp
   Web of Science ®Google Scholar
• Vosinakis, S., Koutsabasis, P., Makris, D., Sagia, E. (2016). A kinesthetic approach to digital heritage using leap motion: The cycladic sculpture application. 2016 8th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), 1–8. https://doi.org/10.1109/VS-GAMES.2016.7590334
   Google Scholar
• Wallace, M. T., Roberson, G. E., Hairston, W. D., Stein, B. E., Vaughan, J. W., & Schirillo, J. A. (2004). Unifying multisensory signals across time and space. Experimental brain Research, 158(2), 252–258. https://doi.org/10.1007/s00221-004-1899-9
   PubMed Web of Science ®Google Scholar
• Wickens, C. D. (2008). Multiple resources and mental workload. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 449–455. https://doi.org/10.1518/001872008X288394
   PubMed Web of Science ®Google Scholar
• Wozniak, P., Vauderwange, O., Mandal, A., Javahiraly, N., & Curticapean, D. (2016). Possible applications of the LEAP Motion controller for more interactive simulated experiments in Augmented or Virtual Reality. In G. G. Gregory (Ed.), Optics education and outreach Iv (Vol. 9946, p. 99460). Spie-Int Soc Optical Engineering. https://doi.org/10/gppvmz
   Google Scholar
• Yu, D., Wang, S., Song, F., Liu, Y., Zhang, S., Wang, Y., Xie, X., & Zhang, Z. (2022). Research on user experience of the video game difficulty based on flow theory and fNIRS. Behaviour & Information Technology, 1–17. https://doi.org/10.1080/0144929X.2022.2043442
   Web of Science ®Google Scholar
• Zhao, M. Y., Ong, S. K., & Nee, A. Y. C. (2016). An augmented reality-assisted therapeutic healthcare exercise system based on bare-hand interaction. International Journal of Human-Computer Interaction, 32(9), 708–721. https://doi.org/10.1080/10447318.2016.1191263
   Web of Science ®Google Scholar
• Zhou, X., Qin, H., Xiao, W., Jia, L., & Xue, C. (2020). A comparative usability study of bare hand three-dimensional object selection techniques in virtual environment. Symmetry, 12(10), 1723. https://doi.org/10.3390/sym12101723
   Google Scholar