Comparison of Panel and Pie Menus for Mid-Air Interaction in a Photo Booth: Muscle Activity, Performance, and User Experience

References

• Ackad, C., Clayphan, A., Tomitsch, M., & Kay, J. (2015, September). An in-the-wild study of learning mid-air gestures to browse hierarchical information at a large interactive public display [Paper presentation]. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 1227–1238). https://dl.acm.org/doi/abs/10.1145/2750858.2807532
   Google Scholar
• Alenabi, T., Whittaker, R. L., Kim, S. Y., & Dickerson, C. R. (2019). Arm posture influences on regional supraspinatus and infraspinatus activation in isometric arm elevation efforts. Journal of Electromyography and Kinesiology, 44(2019), 108–116. https://doi.org/10.1016/j.jelekin.2018.12.005
   PubMed Web of Science ®Google Scholar
• Asakawa, D. S., Becker, M. G., Asaro, J. M., & Hein, J. L. (2022). Shoulder, elbow, and wrist joint angle excursions vary by gesture during touchscreen interaction. Journal of Electromyography and Kinesiology, 62(2022), 102377. https://doi.org/10.1016/j.jelekin.2019.102377
   PubMed Web of Science ®Google Scholar
• Bailly, G., Lecolinet, E., & Nigay, L. (2017). Visual menu techniques. ACM Computing Surveys, 49(4), 1–41. https://doi.org/10.1145/3002171
   Web of Science ®Google Scholar
• Barbero, M., Merletti, R., & Rainoldi, A. (2012). Atlas of muscle innervation zones: Understanding surface electromyography and its applications. Springer Science & Business Media.
   Google Scholar
• Bianchi-Berthouze, N., Kim, W. W., & Patel, D. (2007). Does body movement engage you more in digital game play? and why? In Affective Computing and Intelligent Interaction: Second International Conference, ACII 2007 Lisbon, Portugal, September 12-14, 2007 Proceedings 2 (pp. 102–113). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-74889-2_10
   Google Scholar
• Boettcher, C. E., Ginn, K. A., & Cathers, I. (2008). Standard maximum isometric voluntary contraction tests for normalizing shoulder muscle EMG. Journal of Orthopaedic Research, 26(12), 1591–1597. https://doi.org/10.1002/jor.20675
   PubMed Web of Science ®Google Scholar
• Brand, D., Büchele, K., & Meschtscherjakov, A. (2016, October). Pointing at the HUD: Gesture interaction using a leap motion [Paper presentation]. Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 167–172). https://doi.org/10.1145/3004323.3004343
   Google Scholar
• Burden, A. (2010). How should we normalize electromyograms obtained from healthy participants? What we have learned from over 25 years of research. Journal of Electromyography and Kinesiology, 20(6), 1023–1035. https://doi.org/10.1016/j.jelekin.2010.07.004
   PubMed Web of Science ®Google Scholar
• Callahan, J., Hopkins, D., Weiser, M., & Shneiderman, B. (1988, May). An empirical comparison of pie vs. linear menus [Paper presentation]. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 95–100). https://doi.org/10.1145/57167.57182
   Google Scholar
• Chang, Y. H., & Hwang, T. P. (2011). Effects of Menu Types and Item Lengths on Operation Efficiency. In Design, user experience, and usability. Theory, methods, tools and practice: First International Conference, DUXU 2011, Held as Part of HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part II 1 (pp. 376–383). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-21708-1_43
   Google Scholar
• Chen, Z., Ma, X., Peng, Z., Zhou, Y., Yao, M., Ma, Z., Wang, C., Gao, Z., & Shen, M. (2018). User-defined gestures for gestural interaction: Extending from hands to other body parts. International Journal of Human–Computer Interaction, 34(3), 238–250. https://doi.org/10.1080/10447318.2017.1342943
   Web of Science ®Google Scholar
• Choi, E., Kwon, S., Lee, D., Lee, H., & Chung, M. K. (2014). Towards successful user interaction with systems: Focusing on user-derived gestures for smart home systems. Applied Ergonomics, 45(4), 1196–1207. https://doi.org/10.1016/j.apergo.2014.02.010
   PubMed Web of Science ®Google Scholar
• Chu, S., & Tanaka, J. (2015). Design of a motion-based gestural menu-selection interface for a self-portrait camera. Personal and Ubiquitous Computing, 19(2), 415–424. https://doi.org/10.1007/s00779-014-0776-1
   Web of Science ®Google Scholar
• Chu, S., Zhang, F., Ji, N., Jin, Z., & Pan, R. (2017, May). Pan-and-tilt self-portrait system using gesture interface [Paper presentation]. 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS) (pp. 599–605). IEEE. https://doi.org/10.1109/ICIS.2017.7960063
   Google Scholar
• Coenen, J., Claes, S., & Moere, A. V. (2017, June). The concurrent use of touch and mid-air gestures or floor mat interaction on a public display [Paper presentation]. Proceedings of the 6th ACM International Symposium on Pervasive Displays (pp. 1–9). https://doi.org/10.1145/3078810.3078819
   Google Scholar
• Cook, C. S., & McDonagh, M. J. N. (1996). Measurement of muscle and tendon stiffness in man. European Journal of Applied Physiology and Occupational Physiology, 72(4), 380–382. https://doi.org/10.1007/BF00599700
   PubMed Web of Science ®Google Scholar
• Coppola, S. M., Lin, M. Y., Schilkowsky, J., Arezes, P. M., & Dennerlein, J. T. (2018). Tablet form factors and swipe gesture designs affect thumb biomechanics and performance during two-handed use. Applied Ergonomics, 69(2018), 40–46. https://doi.org/10.1016/j.apergo.2017.12.015
   PubMed Web of Science ®Google Scholar
• David-John, B., Peacock, C., Zhang, T., Murdison, T. S., Benko, H., & Jonker, T. R. (2021). Towards gaze-based prediction of the intent to interact in virtual reality [Paper presentation]. ACM Symposium on Eye Tracking Research and Applications (pp. 1–7). https://doi.org/10.1145/3448018.3458008
   Google Scholar
• Deng, S., Jiang, N., Chang, J., Guo, S., & Zhang, J. J. (2017). Understanding the impact of multimodal interaction using gaze informed mid-air gesture control in 3D virtual objects manipulation. International Journal of Human-Computer Studies, 105(2017), 68–80. https://doi.org/10.1016/j.ijhcs.2017.04.002
   Web of Science ®Google Scholar
• Finstad, K. (2010). The usability metric for user experience. Interacting with Computers, 22(5), 323–327. https://doi.org/10.1016/j.intcom.2010.04.004
   Web of Science ®Google Scholar
• Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381–391. https://doi.org/10.1037/h0055392
   PubMedGoogle Scholar
• Forman, D. A., Forman, G. N., Robathan, J., & Holmes, M. W. (2019). The influence of simultaneous handgrip and wrist force on forearm muscle activity. Journal of Electromyography and Kinesiology, 45(2019), 53–60. https://doi.org/10.1016/j.jelekin.2019.02.004
   PubMed Web of Science ®Google Scholar
• Gebhardt, S., Pick, S., Leithold, F., Hentschel, B., & Kuhlen, T. (2013). Extended pie menus for immersive virtual environments. IEEE Transactions on Visualization and Computer Graphics, 19(4), 644–651. https://doi.org/10.1109/TVCG.2013.31
   PubMed Web of Science ®Google Scholar
• Gerr, F., Fethke, N. B., Merlino, L., Anton, D., Rosecrance, J., Jones, M. P., Marcus, M., & Meyers, A. R. (2014). A prospective study of musculoskeletal outcomes among manufacturing workers: I. Effects of physical risk factors. Human Factors, 56(1), 112–130. https://doi.org/10.1177/0018720813491114
   PubMed Web of Science ®Google Scholar
• Gonçalves, J. S., Moriguchi, C. S., Takekawa, K. S., Coury, H. J. C. G., & de Oliveira Sato, T. (2017). The effects of forearm support and shoulder posture on upper trapezius and anterior deltoid activity. Journal of Physical Therapy Science, 29(5), 793–798. https://doi.org/10.1589/jpts.29.793
   PubMedGoogle Scholar
• Halaki, M., & Ginn, K. (2012). Normalization of EMG signals: To normalize or not to normalize and what to normalize to. Computational Intelligence in Electromyography Analysis-A Perspective on Current Applications and Future Challenges, 10(2012), 49957. https://doi.org/10.5772/49957
   Google Scholar
• Harvey, R., & Peper, E. (1997). Surface electromyography and mouse use position. Ergonomics, 40(8), 781–789. https://doi.org/10.1080/001401397187775
   PubMed Web of Science ®Google Scholar
• Hincapié-Ramos, J. D., Guo, X., Moghadasian, P., & Irani, P. (2014, April). Consumed endurance: A metric to quantify arm fatigue of mid-air interactions [Paper presentation]. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1063–1072). https://doi.org/10.1145/2556288.2557130
   Google Scholar
• Holmes, M. W., Tat, J., & Keir, P. J. (2015). Neuromechanical control of the forearm muscles during gripping with sudden flexion and extension wrist perturbations. Computer Methods in Biomechanics and Biomedical Engineering, 18(16), 1826–1834. https://doi.org/10.1080/10255842.2014.976811
   PubMed Web of Science ®Google Scholar
• Huang, J., Mao, L., Qi, M., Zhang, D., An, M., Han, R., & Ji, T. (2022). Differences in muscle activity, kinematics, user performance, and subjective assessment between touchscreen and mid-air interactions on a tablet. Behaviour & Information Technology, 41(14), 3028–3043. https://doi.org/10.1080/0144929X.2021.1970227
   Web of Science ®Google Scholar
• Huang, J., Qi, M., Mao, L., An, M., Ji, T., & Han, R. (2021). User-defined gestures for mid-air interaction: A comparison of upper limb muscle activity, wrist kinematics, and subjective preference. International Journal of Human–Computer Interaction, 37(16), 1516–1537. https://doi.org/10.1080/10447318.2021.1898825
   Web of Science ®Google Scholar
• Huang, S., Ranganathan, S. P., & Parsons, I. (2020). To touch or not to touch? Comparing Touch, mid-air gesture, midair haptics for public display in post COVID-19 society [Paper presentation]. SIGGRAPH Asia 2020 Posters (SA ’20), New York, NY, USA (Article 5, pp. 1–2). Association for Computing Machinery. https://doi.org/10.1145/3415264.3425438
   Google Scholar
• Huang, J., Zhang, H., Mao, L., Zhang, D., Li, J., Ji, T., & Han, R. (2022). The effect of tablet computer configurations and touchscreen gestures on human biomechanics, performance, and subjective assessment. International Journal of Human–Computer Interaction, (2022), 1–17. https://doi.org/10.1080/10447318.2022.2111051
   Web of Science ®Google Scholar
• Janssen, M. M., Harlaar, J., & de Groot, I. J. (2015). Surface EMG to assess arm function in boys with DMD: A pilot study. Journal of Electromyography and Kinesiology, 25(2), 323–328. https://doi.org/10.1016/j.jelekin.2015.01.008
   PubMed Web of Science ®Google Scholar
• Kim, K., Kim, J., Choi, J., Kim, J., & Lee, S. (2015). Depth camera-based 3D hand gesture controls with immersive tactile feedback for natural mid-air gesture interactions. Sensors, 15(1), 1022–1046. https://doi.org/10.3390/s150101022
   PubMed Web of Science ®Google Scholar
• Kim, Y., Lee, G. A., Jo, D., Yang, U., Kim, G., & Park, J. (2011, January). Analysis on virtual interaction-induced fatigue and difficulty in manipulation for interactive 3D gaming console [Paper presentation]. 2011 IEEE International Conference on Consumer Electronics (ICCE) (pp. 269–270). IEEE. https://doi.org/10.1109/ICCE.2011.5722577
   Google Scholar
• Komerska, R., & Ware, C. (2004, March). A study of haptic linear and pie menus in a 3D fish tank VR environment [Paper presentation]. 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS'04. Proceedings (pp. 224–231). IEEE. https://doi.org/10.1109/HAPTIC.2004.1287200
   Google Scholar
• Koutsabasis, P., & Vogiatzidakis, P. (2019). Empirical research in mid-air interaction: A systematic review. International Journal of Human–Computer Interaction, 35(18), 1747–1768. https://doi.org/10.1080/10447318.2019.1572352
   Web of Science ®Google Scholar
• Lee, M., Kwahk, J., Han, S. H., & Lee, H. (2020). Relative pointing interface: A gesture interaction method based on the ability to divide space. International Journal of Industrial Ergonomics, 75(2020), 102878. https://doi.org/10.1016/j.ergon.2019.102878
   Web of Science ®Google Scholar
• Li, W., Wan, X., Shi, Y., Yao, N., Wang, C., & Gao, Z. (2021). Depth and breadth of pie menus for mid-air gesture interaction. International Journal of Human–Computer Interaction, 37(2), 131–140. https://doi.org/10.1080/10447318.2020.1809245
   Web of Science ®Google Scholar
• Lund, A. M. (2001). Measuring usability with the use questionnaire12. Usability Interface, 8(2), 3–6. https://scholar.google.com/scholar_lookup?hl=zh-CN&volume=8&publication_year=2001&pages=3-6&journal=Usability+Interface&issue=2&author=A.+M.+Lund&title=Measuring+usability+with+the+USE+questionnaire
   Google Scholar
• Lyu, F., Li, H., Fu, Q., Liu, Y., Huang, J., & Deng, Z. (2023). Effects of spatial constraints and ages on children’s upper limb performance in mid-air gesture interaction. International Journal of Human-Computer Studies, 170(2023), 102952. https://doi.org/10.1016/j.ijhcs.2022.102952
   Web of Science ®Google Scholar
• Mendes, D., Medeiros, D., Sousa, M., Ferreira, R., Raposo, A., Ferreira, A., & Jorge, J. (2017, March). Mid-air modeling with Boolean operations in VR [Paper presentation]. 2017 IEEE Symposium on 3D User Interfaces (3DUI) (pp. 154–157). IEEE. https://doi.org/10.1109/3DUI.2017.7893332
   Google Scholar
• Monnai, Y., Hasegawa, K., Fujiwara, M., Yoshino, K., Inoue, S., & Shinoda, H. (2014, October). HaptoMime: Mid-air haptic interaction with a floating virtual screen [Paper presentation]. Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (pp. 663–667). https://doi.org/10.1145/2642918.2647407
   Google Scholar
• Monteiro, P., Coelho, H., Gonçalves, G., Melo, M., & Bessa, M. (2019). Comparison of radial and panel menus in virtual reality. IEEE Access, 7(2019), 116370–116379. https://doi.org/10.1109/ACCESS.2019.2933055
   Web of Science ®Google Scholar
• Morris, M. R., Danielescu, A., Drucker, S., Fisher, D., Lee, B., Schraefel, M. C., & Wobbrock, J. O. (2014). Reducing legacy bias in gesture elicitation studies. Interactions, 21(3), 40–45. https://doi.org/10.1145/2591689
   Google Scholar
• Murano, P., & Khan, I. N. (2015). Pie menus or linear menus, which is better? Journal of Emerging Trends in Computing and Information Sciences, 6(9), 476–481. https://scholar.google.com.hk/scholar?hl=zh-CN&as_sdt=0%2C5&q=Pie+menus+or+linear+menus%2C+which+is+better%3F&btnG=
   Google Scholar
• Ni, T., Bowman, D. A., North, C., & McMahan, R. P. (2011). Design and evaluation of freehand menu selection interfaces using tilt and pinch gestures. International Journal of Human-Computer Studies, 69(9), 551–562. https://doi.org/10.1016/j.ijhcs.2011.05.001
   Web of Science ®Google Scholar
• Pereira, A., Wachs, J. P., Park, K., & Rempel, D. (2015). A user-developed 3-D hand gesture set for human–computer interaction. Human Factors, 57(4), 607–621. https://doi.org/10.1177/0018720814559307
   PubMed Web of Science ®Google Scholar
• Pourmemar, M., & Poullis, C. (2019, November). Visualizing and interacting with hierarchical menus in immersive augmented reality [Paper presentation]. The 17th International Conference on Virtual-Reality Continuum and Its Applications in Industry (pp. 1–9). https://doi.org/10.1145/3359997.3365693
   Google Scholar
• Qin, J., Chen, H., & Dennerlein, J. T. (2013). Wrist posture affects hand and forearm muscle stress during tapping. Applied Ergonomics, 44(6), 969–976. https://doi.org/10.1016/j.apergo.2013.03.013
   PubMed Web of Science ®Google Scholar
• Schwartz, C., Tubez, F., Wang, F. C., Croisier, J. L., Brüls, O., Denoël, V., & Forthomme, B. (2017). Normalizing shoulder EMG: An optimal set of maximum isometric voluntary contraction tests considering reproducibility. Journal of Electromyography and Kinesiology, 37(2017), 1–8. https://doi.org/10.1016/j.jelekin.2017.08.005
   PubMed Web of Science ®Google Scholar
• Urbina, M. H., Lorenz, M., & Huckauf, A. (2010, March). Pies with EYEs: The limits of hierarchical pie menus in gaze control [Paper presentation]. Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications (pp. 93–96). https://doi.org/10.1145/1743666.1743689
   Google Scholar
• Vogiatzidakis, P., & Koutsabasis, P. (2021, November). Mid-air gestures for manipulation of multiple targets in the physical space: Comparing the usability of two interaction models [Paper presentation]. CHI Greece 2021: 1st International Conference of the ACM Greek SIGCHI Chapter (pp. 1–9). https://doi.org/10.1145/3489410.3489425
   Google Scholar
• Vogiatzidakis, P., & Koutsabasis, P. (2022). ‘Address and command’: Two-handed mid-air interactions with multiple home devices. International Journal of Human-Computer Studies, 159(2022), 102755. https://doi.org/10.1016/j.ijhcs.2021.102755
   Web of Science ®Google Scholar
• Walter, R., Bailly, G., Valkanova, N., & Müller, J. (2014, September). Cuenesics: Using mid-air gestures to select items on interactive public displays [Paper presentation]. Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & Services (pp. 299–308). https://doi.org/10.1145/2628363.2628368
   Google Scholar
• Wang, W., Qin, X., Zheng, C., Wang, H., Li, J., & Niu, J. (2018). Mechanical energy expenditure-based comfort evaluation model for gesture interaction. Computational Intelligence and Neuroscience, 2018(2018), 9861697–9861611. https://doi.org/10.1155/2018/9861697
   Web of Science ®Google Scholar
• Wu, S., Gable, T., May, K., Choi, Y. M., & Walker, B. N. (2016). Comparison of surface gestures and air gestures for in-vehicle menu navigation. Archives of Design Research, 29(4), 65–80. https://doi.org/10.15187/adr.2016.11.29.4.65
   Google Scholar
• Yang, H. H., Chen, Z. N., & Hung, C. W. (2017). Performance of smartphone users with half-pie and linear menus. Behaviour & Information Technology, 36(9), 935–954. https://doi.org/10.1080/0144929X.2017.1312529
   Web of Science ®Google Scholar