Initial validation of the GUESS-18 for usability in virtual reality gaming environments: a pilot study

References

• Arh, T., and B. J. Blažič. 2008. “A Case Study of Usability Testing - The SUMI Evaluation Approach of the EducaNext Portal.” WSEAS Transactions on Information Science & Applications 5 (2): 175–181.
   Google Scholar
• Atmaja, P. W., and S. Sugiarto. 2021. “Balancing Entertainment, Cost, and Educational Strength: A Design Framework for Medium-Coupling Educational Games.” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control 4: 27–40. doi:10.22219/kinetik.v6i1.1158.
   Google Scholar
• Baker, H. 2021. Beat Saber Passes $100 Million In Revenue On Quest Platform Alone. https://uploadvr.com/beat-saber-100-million-revenue/#:∼:text=InFebruary%2C Facebook revealed that, Store (Quest and Rift).
   Google Scholar
• Bateye, O., F. A. Fatoye, A. S. Adejumobi, F. S. Olatoye, M. O. Makinde, C. E. Mbada, A. J. Salami, O. O. Dada, I. P. Gambo, and A. C. Odole. 2020. “Development and Feasibility Testing of a Back Extension-Glide Virtual Reality Game for Low-Back Pain.” International Journal of Telemedicine and Clinical Practices 3 (3): 243. doi:10.1504/ijtmcp.2020.10026527.
   Google Scholar
• Bente, G., and J. Breuer. 2010. “Why so Serious? On the Relation of Serious Games and Learning.” Eludamos - Journal for Computer Game Culture 4 (1): 7–24. http://www.eludamos.org/index.php/eludamos/article/viewArticle/vol4no1-2.
   Google Scholar
• Briggs, N. E., and R. C. MacCallum. 2003. “Recovery of Weak Common Factors by Maximum Likelihood and Ordinary Least Squares Estimation.” Multivariate Behavioral Research 38 (1): 25–56. doi:10.1207/S15327906MBR3801_2.
   PubMed Web of Science ®Google Scholar
• Brooke, J. 1986. “SUS: A “Quick and Dirty” Usability Scale.” In Usability Evaluation in Industry: 1st edition, edited by Patrick W. Jordan, B. Thomas, Ian Lyall McClelland, Bernard. Weerdmeester: CRC Press, 207–212. doi:10.1201/9781498710411-35.
   Google Scholar
• Buttussi, F., and L. Chittaro. 2018. “Effects of Different Types of Virtual Reality Display on Presence and Learning in a Safety Training Scenario.” IEEE Transactions on Visualization and Computer Graphics 24 (2): 1063–1076.
   Google Scholar
• Carroll, M., E. Osborne, and C. Yildirim. 2019. “Effects of VR Gaming and Game Genre on Player Experience.” 2019 IEEE Games, Entertainment, Media Conference, GEM 2019. doi:10.1109/GEM.2019.8811554.
   Google Scholar
• Chin, J. P., V. A. Diehl, and K. L. Norman. 1988. “Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface.” Conference on Human Factors in Computing Systems - Proceedings, Part F1302, 213–218. doi:10.1145/57167.57203.
   Google Scholar
• Fang, Y., and C. Lin. 2019. “The Usability Testing of VR Interface for Tourism Apps.” Applied Sciences 9 (16): 3215. doi:10.3390/app9163215.
   Google Scholar
• Fox, J., L. Pittaway, and I. Uzuegbunam. 2018. “Simulations in Entrepreneurship Education: Serious Games and Learning through Play.” Entrepreneurship Education and Pedagogy 1 (1): 61–89. doi:10.1177/2515127417737285.
   Google Scholar
• Fussell, S. G., J. L. Derby, J. K. Smith, W. J. Shelstad, J. D. Benedict, B. S. Chaparro, R. Thomas, and A. R. Dattel. 2019. “Usability Testing of a Virtual Reality Tutorial.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 63 (1): 2303–2307. doi:10.1177/1071181319631494.
   Google Scholar
• Fussell, S. G., and M. P. Hight. 2021. “Usability Testing of a VR Flight Training Program.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 65 (1): 1124–1128. doi:10.1177/1071181321651096.
   Google Scholar
• Genie, G. M. 2020. What Does Mixed Reality Mean For the Video Game Industry? https://www.gamemarketinggenie.com/blog/mixed-reality-video-game-industry
   Google Scholar
• Haleem, A., and M. Javaid. 2018. “Role of CT and MRI in the Design and Development of Orthopaedic Model Using Additive Manufacturing.” Journal of Clinical Orthopaedics and Trauma 9 (3): 213–217. doi:10.1016/j.jcot.2018.07.002.
   PubMedGoogle Scholar
• Javaid, M., and A. Haleem. 2020. “Virtual Reality Applications toward Medical Field.” Clinical Epidemiology and Global Health 8 (2): 600–605. doi:10.1016/j.cegh.2019.12.010.
   Web of Science ®Google Scholar
• Keebler, J., W. Shelstad, D. Smith, B. Chaparro, and M. Phan. 2020. “Validation of the GUESS-18: A Short Version of the Game User Experience Satisfaction Scale (GUESS).” Journal of Usability Studies 16 (1): 49.
   Web of Science ®Google Scholar
• Kennedy, R. S., N. E. Lane, K. S. Berbaum, and M. G. Lilienthal. 1993. “Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness.” The International Journal of Aviation Psychology 3 (3): 203–220. doi:10.1207/s15327108ijap0303_3.
   Google Scholar
• Khooshabeh, P., I. Choromanski, C. Neubauer, D. M. Krum, R. Spicer, and J. Campbell. 2017. “Mixed Reality Training for Tank Platoon Leader Communication Skills.” 2017 IEEE Virtual Reality (VR), 333–334. doi:10.1109/VR.2017.7892312.
   Google Scholar
• Kim, G., and Biocca, F. 2018. Immersion in Virtual Reality Can Increase Exercise Motivation and Physical Performance. In Virtual, Augmented and Mixed Reality: Applications in Health, Cultural Heritage, and Industry, edited by J. Chen and G. Fragomeni. VAMR 2018. Lecture Notes in Computer Science, vol 10910. Springer, Cham. doi:10.1007/978-3-319-91584-5_8
   Google Scholar
• Lewis, J. R. 2002. “Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies.” International Journal of Human-Computer Interaction 14 (3–4): 463–488. doi:10.1207/s15327590ijhc143&4_11.
   Web of Science ®Google Scholar
• Lewis, J. R. 2018. “The System Usability Scale : Past, Present, and Future.” International Journal of Human–Computer Interaction 34 (7): 577–590. doi:10.1080/10447318.2018.1455307.
   Google Scholar
• Lewis, J. R., and J. Sauro. 2009. “The Factor Structure of the System Usability Scale.” Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 5619 LNCS, 94–103. doi:10.1007/978-3-642-02806-9_12.
   Google Scholar
• Liang, J., D. Xian, X. Liu, J. Fu, X. Zhang, B. Tang, and J. Lei. 2018. “Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation.” JMIR MHealth and UHealth 6 (11): e11066. doi:10.2196/11066.
   Web of Science ®Google Scholar
• Macchiarella, N. D., P. K. Arban, and S. M. Doherty. 2006. “Transfer of Training from Flight Training Devices to Flight for ab-Initio Pilots.” International Journal of Applied Aviation Studies 6 (2): 299–314.
   Google Scholar
• Manero, A., J. Sparkman, M. Dombrowski, R. Buyssens, and P. A. Smith. 2018. “Developing and Training Multi-Gestural Prosthetic Arms.” Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Vol. 10909 LNCS. Springer International Publishing. doi:10.1007/978-3-319-91581-4_32.
   Google Scholar
• Martin, A., S. Cross, and C. Attoe. 2020. “The Use of in Situ Simulation in Healthcare Education: Current Perspectives.” Advances in Medical Education and Practice 11: 893–903. doi:10.2147/AMEP.S188258.
   PubMed Web of Science ®Google Scholar
• Mittmann, G., S. Zehetmayer, and B. Schrank. 2021. “Study Protocol for a Randomized Controlled Trial to Evaluate the Effectiveness of a Serious Game Targeting Interpersonal Emotion Regulation in Early Adolescents.” Trials 22(1): 741.
   Google Scholar
• Mossel, A., C. Schoenauer, M. Froeschl, A. Peer, and J. Goellner. 2021. “Immersive Training of First Responder Squad Leaders in Untethered Virtual Reality.” Virtual Reality 25 (3): 745–759. doi:10.1007/s10055-020-00487-x.
   Web of Science ®Google Scholar
• Mossel, A., A. Vienna, A. Peer, and J. Göllner. 2017. “Requirements Analysis on a Virtual Reality Training System for CBRN Crisis Preparedness.” Proceedings of the 59th Annual Meeting of the ISSS, vol. 1 (1): 1–20.
   Google Scholar
• Naeini, S., S. Mostowfi, and H. S. Naeini. 2015. “Using QUIS as a Measurement Tool for User Satisfaction Evaluation (Case Study: Vending Machine) Fashion Design View Project Ergonomic Product Design Methodology View Project Using QUIS as a Measurement Tool for User Satisfaction Evaluation (Case Study: Ve).” International Journal of Information Science 2015 (1): 14–23. doi:10.5923/j.ijis.20150501.03.
   Google Scholar
• Nielsen, J. 1994. “Enhancing the Explanatory Power of Usability Heuristics.” Conference on Human Factors in Computing Systems - Proceedings, 152–158. doi:10.1145/191666.191729.
   Google Scholar
• Pal, D., and V. Vanijja. 2020. “Perceived Usability Evaluation of Microsoft Teams as an Online Learning Platform during COVID-19 Using System Usability Scale and Technology Acceptance Model in India.” Children and Youth Services Review 119 (September): 105535. doi:10.1016/j.childyouth.2020.105535.
   PubMedGoogle Scholar
• Pallavicini, F., and A. Pepe. 2019. “Comparing Player Experience in Video Games Played in Virtual Reality or on Desktop Displays.” Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, 195–210. doi:10.1145/3341215.3355736.
   Google Scholar
• Pallavicini, F., A. Pepe, and E. Minissi. 2019. “Gaming in Virtual Reality : What Changes in Terms of Usability, Emotional Response and Sense of Presence Compared to Non-Immersive Video Games ?” Simulation and Gaming 15(2): 136–159. doi:10.1177/1046878119831420.
   Google Scholar
• Phan, M. H., J. R. Keebler, and B. S. Chaparro. 2016. “The Development and Validation of the Game User Experience Satisfaction Scale (GUESS).” Human Factors 58 (8): 1217–1247. doi:10.1177/0018720816669646.
   PubMed Web of Science ®Google Scholar
• Porras, D., H. Sharon, R. Inzelberg, Y. Ziv-Ner, G. Zeilig, and M. Plotnik. 2019. “Advanced Virtual Reality-Based Rehabilitation of Balance and Gait in Clinical Practice.” Therapeutic Advances in Chronic Disease 10: 2040622319868379. doi:10.1177/2040622319868379.
   Web of Science ®Google Scholar
• Samuels, P. 2016. “Advice on Reliability Analysis with Small Samples.” Technical Report (August 2015): 2–6. 10.13140/RG.2.1.1495.5364.
   Google Scholar
• Schorer, A., and A. Protopsaltis. 2021. “Learners’ User Experience Assessment of a Serious Game for Social Innovation Education.” Proceedings of the European Conference on Games-Based Learning, 2021 September, 622–630. doi:10.34190/GBL.21.051.
   Google Scholar
• Schutte, N. S., and E. J. Stilinović. 2017. “Facilitating Empathy through Virtual Reality.” Motivation and Emotion 41 (6): 708–712. doi:10.1007/s11031-017-9641-7.
   Web of Science ®Google Scholar
• Sharar, S., W. Miller, A. Teeley, M. Soltani, H. Hoffman, M. Jensen, and D. Patterson. 2008. “Applications of Virtual Reality for Pain Management in Burn-Injured Patients.” Expert Review of Neurotherapeutics 8 (11): 1667–1674. doi:10.1586/14737175.8.11.1667.Applications.
   PubMedGoogle Scholar
• Shelstad, W. J., B. S. Chaparro, and J. R. Keebler. 2019. “Assessing the User Experience of Video Games: Relationships between Three Scales.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 63 (1): 1488–1492. doi:10.1177/1071181319631300.
   Google Scholar
• Shelstad, W. J., A. A. Hosein, J. R. Keebler, and B. S. Chaparro. 2020. “Using User Experience Scales to Predict Video Game Play and Purchasing Intent.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 64 (1): 2065–2069. doi:10.1177/1071181320641500.
   Google Scholar
• Shelstad, W. J., D. C. Smith, and B. S. Chaparro. 2017. “Gaming on the Rift: How Virtual Reality Affects Game User Satisfaction.” Proceedings of the Human Factors and Ergonomics Society 61(1): 2072–2076. doi:10.1177/1541931213602001.
   Google Scholar
• Singh, R., M. Javaid, R. Kataria, M. Tyagi, A. Haleem, and R. Suman. 2020. “Significant Applications of Virtual Reality for COVID-19 Pandemic.” Diabetes and Metabolic Syndrome: Clinical Research and Reviews 14(4): 661–664.
   Google Scholar
• Sinlapanuntakul, W., J. L. Derby, and B. S. Chaparro. 2022. “Understanding the Effects of Mixed Reality on Video Game Satisfaction, Enjoyment, and Performance.” Simulation and Gaming 53 (3): 237–252. doi:10.1177/10468781221094473.
   Google Scholar
• Sinlapanuntakul, W., W. J. Shelstad, J. L. Derby, and B. S. Chaparro. 2020. “Gameplay in Mixed Reality : How It Differs from Mobile Gameplay.” Poster presented at Discovery Day, ­Embry-Riddle Aeronautical University. Available at: https://commons.erau.edu/discovery-day/db-discovery-day-2021/poster-session/20/
   Google Scholar
• Suhr, D. D. 2006. “Exploratory or Confirmatory Factor Analysis?” Rehabilitation 45 (4): 243–248. doi:10.1055/s-2006-940029.
   PubMedGoogle Scholar
• Susi, T., M. Johannesson, and P. Backlund. 2007. “Serious Games - An Overview.” 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference, UEMCON 2017, 2018 January, 488–491. doi:10.1109/UEMCON.2017.8249059.
   Google Scholar
• Van Ommen, C., and B. S. Chaparro. 2021. “Assessing Video Game Satisfaction of Gamers with Disabilities.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 65 (1): 822–826. doi:10.1177/1071181321651319.
   Google Scholar
• Waller, D., E. Hunt, and D. Knapp. 1998. “The Transfer of Spatial Knowledge in Virtual Environment Training.” Presence: Teleoperators and Virtual Environments 7 (2): 129–143. doi:10.1162/105474698565631.
   Web of Science ®Google Scholar
• Wold, S., K. Esbensen, and P. Geladi. 1987. “Principal Component Analysis.” Chemometrics and Intelligent Labratory Systems 2: 37–52. doi:10.1007/978-0-387-87811-9_2.
   Web of Science ®Google Scholar
• Yildirim, C., M. Carroll, D. Hufnal, T. Johnson, and S. Pericles. 2018. “Video Game User Experience: To VR, or Not to VR?.” 2018 IEEE Games, Entertainment, Media Conference, GEM 2018, 125–131. doi:10.1109/GEM.2018.8516542.
   Google Scholar
• Yoshimura, B. 2021. “Making Health Education Palatable : A Usability Study of a Digital Game about Nutrition.” Master’s Project, University of Hawai’i Manoa Available at https://scholarspace.manoa.hawaii.edu/bitstreams/10484141-8f3a-4fa7-ad3c-5d8777df6d26/download
   Google Scholar
• Zainuddin, Z., S. K. W. Chu, M. Shujahat, and C. J. Perera. 2020. “The Impact of Gamification on Learning and Instruction: A Systematic Review of Empirical Evidence.” Educational Research Review 30: 100326 (March). doi:10.1016/j.edurev.2020.100326.
   Google Scholar