Differences in muscle activity, kinematics, user performance, and subjective assessment between touchscreen and mid-air interactions on a tablet

References

• Akyol, S., U. Canzler, K. Bengler, and W. Hahn. 2000, November. “Gesture Control for Use in Automobiles.” In MVA, 349–352.
   Google Scholar
• Alenabi, T., R. L. Whittaker, S. Y. Kim, and C. R. Dickerson. 2019. “Arm Posture Influences on Regional Supraspinatus and Infraspinatus Activation in Isometric arm Elevation Efforts.” Journal of Electromyography and Kinesiology 44: 108–116. doi:10.1016/j.jelekin.2018.12.005.
   PubMed Web of Science ®Google Scholar
• Ardito, C., M. F. Costabile, and H. C. Jetter. 2014. “Gestures That People Can Understand and use.” Journal of Visual Languages & Computing 25 (5): 572–576. doi:10.1016/j.jvlc.2014.07.002.
   Web of Science ®Google Scholar
• Asakawa, D. S., M. G. Becker, J. M. Asaro, and J. L. Hein. 2019. “Shoulder, Elbow, and Wrist Joint Angle Excursions Vary by Gesture During Touchscreen Interaction.” Journal of Electromyography and Kinesiology 2019: 102377. doi:10.1016/j.jelekin.2019.102377.
   Web of Science ®Google Scholar
• Asakawa, D. S., G. H. Crocker, A. Schmaltz, and D. L. Jindrich. 2017. “Fingertip Forces and Completion Time for Index Finger and Thumb Touchscreen Gestures.” Journal of Electromyography and Kinesiology 34: 6–13. doi:10.1016/j.jelekin.2017.02.007.
   PubMed Web of Science ®Google Scholar
• Asakawa, D. S., J. T. Dennerlein, and D. L. Jindrich. 2017. “Index Finger and Thumb Kinematics and Performance Measurements for Common Touchscreen Gestures.” Applied Ergonomics 58: 176–181. doi:10.1016/j.apergo.2016.06.004.
   PubMed Web of Science ®Google Scholar
• Aslan, I., I. Buchwald, P. Koytek, and E. André. 2016, October. “Pen+ mid-air: An Exploration of mid-air Gestures to Complement pen Input on Tablets.” Proceedings of the 9th nordic conference on human-computer interaction. (pp. 1–10). doi:10.1145/2971485.2971511
   Google Scholar
• Barbero, M., R. Merletti, and A. Rainoldi. 2012. Atlas of Muscle Innervation Zones: Understanding Surface Electromyography and its Applications. New York: Springer Science & Business Media.
   Google Scholar
• Bianchi-Berthouze, N. 2013. “Understanding the Role of Body Movement in Player Engagement.” Human–Computer Interaction 28 (1): 40–75. doi:10.1080/07370024.2012.688468.
   Web of Science ®Google Scholar
• Billinghurst, S. S., and K. P. L. Vu. 2015. “Touch Screen Gestures for web Browsing Tasks.” Computers in Human Behavior 53: 71–81. doi:10.1016/j.chb.2015.06.012.
   Web of Science ®Google Scholar
• Bobeth, J., J. Schrammel, S. Deutsch, M. Klein, M. Drobics, C. Hochleitner, and M. Tscheligi. 2014, June. Tablet, gestures, remote control? Influence of age on performance and user experience with iTV applications. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video (pp. 139-146). doi: 10.1145/2602299.2602315.
   Google Scholar
• Boettcher, C. E., K. A. Ginn, and I. Cathers. 2008. “Standard Maximum Isometric Voluntary Contraction Tests for Normalizing Shoulder Muscle EMG.” Journal of Orthopaedic Research 26 (12): 1591–1597. doi:10.1002/jor.20675.
   PubMed Web of Science ®Google Scholar
• Bragdon, A., E. Nelson, Y. Li, and K. Hinckley. 2011, May. “Experimental Analysis of Touch-Screen Gesture Designs in Mobile Environments.” Proceedings of the SIGCHI Conference on Human Factors in Computing systems. pp. 403–412. doi:10.1145/1978942.1979000
   Google Scholar
• Chen, Z. 2017. “Recognition and Interpretation of Multi-Touchscreen Interaction.” (Doctoral dissertation). Rennes, INSA.
   Google Scholar
• Chen, J., and C. Or. 2017. “Assessing the use of Immersive Virtual Reality, Mouse and Touchscreen in Pointing and Dragging-and-Dropping Tasks among Young, Middle-Aged and Older Adults.” Applied Ergonomics 65: 437–448. doi:10.1016/j.apergo.2017.03.013.
   PubMed Web of Science ®Google Scholar
• Chen, K. B., A. B. Savage, A. O. Chourasia, D. A. Wiegmann, and M. E. Sesto. 2013. “Touch Screen Performance by Individuals with and Without Motor Control Disabilities.” Applied Ergonomics 44 (2): 297–302. doi:10.1016/j.apergo.2012.08.004.
   PubMed Web of Science ®Google Scholar
• Choi, E., S. Kwon, D. Lee, H. Lee, and M. K. Chung. 2014. “Towards Successful User Interaction with Systems: Focusing on User-Derived Gestures for Smart Home Systems.” Applied Ergonomics 45 (4): 1196–1207. doi:10.1016/j.apergo.2014.02.010.
   PubMed Web of Science ®Google Scholar
• Cockburn, A., D. Ahlström, and C. Gutwin. 2012. “Understanding Performance in Touch Selections: Tap, Drag and Radial Pointing Drag with Finger, Stylus and Mouse.” International Journal of Human-Computer Studies 70 (3): 218–233. doi:10.1016/j.ijhcs.2011.11.002.
   Web of Science ®Google Scholar
• Coenen, J., S. Claes, and A. V. Moere. 2017, June. The concurrent use of touch and mid-air gestures or floor mat interaction on a public display. In Proceedings of the 6th ACM International Symposium on Pervasive Displays (pp. 1-9). doi: 10.1145/3078810.3078819.
   Google Scholar
• Coppola, S. M., M. Y. Lin, J. Schilkowsky, P. M. Arezes, and J. T. Dennerlein. 2018. “Tablet Form Factors and Swipe Gesture Designs Affect Thumb Biomechanics and Performance During two-Handed use.” Applied Ergonomics 69: 40–46. doi:10.1016/j.apergo.2017.12.015.
   PubMed Web of Science ®Google Scholar
• Dauncey, T., H. P. Singh, and J. J. Dias. 2017. “Electrogoniometer Measurement and Directional Analysis of Wrist Angles and Movements During the Sollerman Hand Function Test.” Journal of Hand Therapy 30 (3): 328–336. doi:10.1016/j.jht.2016.06.011.
   PubMed Web of Science ®Google Scholar
• Dennerlein, J. T., I. Kingma, B. Visser, and J. H. van Dieën. 2007. “The Contribution of the Wrist, Elbow and Shoulder Joints to Single-Finger Tapping.” Journal of Biomechanics 40 (13): 3013–3022. doi:10.1016/j.jbiomech.2007.01.025.
   PubMed Web of Science ®Google Scholar
• Detjen, H., S. Geisler, and S. Schneegass. 2020, October. “Maneuver-based Control Interventions During Automated Driving: Comparing Touch, Voice, and Mid-Air Gestures as Input Modalities.” 2020 IEEE International Conference on systems, Man, and cybernetics (SMC). (pp. 3268–3274). IEEE. doi:10.1109/SMC42975.2020.9283431
   Google Scholar
• Doty, J. 2019. “Home Exercise Programs for Wrist and Hand Injuries.” Home Exercise Programs for Musculoskeletal and Sports Injuries: The Evidence-Based Guide for Practitioners, 49.
   Google Scholar
• Forman, D. A., G. N. Forman, J. Robathan, and M. W. Holmes. 2019. “The Influence of Simultaneous Handgrip and Wrist Force on Forearm Muscle Activity.” Journal of Electromyography and Kinesiology 45: 53–60. doi:10.1016/j.jelekin.2019.02.004.
   PubMed Web of Science ®Google Scholar
• Gentile, V., A. Adjorlu, S. Serafin, D. Rocchesso, and S. Sorce. 2019, June. Touch or touchless? evaluating usability of interactive displays for persons with autistic spectrum disorders. In Proceedings of the 8th ACM International Symposium on Pervasive Displays (pp. 1-7). doi: 10.1145/3321335.3324946.
   Google Scholar
• Gerr, F., N. B. Fethke, L. Merlino, D. Anton, J. Rosecrance, M. P. Jones, … A. R. Meyers. 2014. “A Prospective Study of Musculoskeletal Outcomes among Manufacturing Workers: I. Effects of Physical Risk Factors.” Human Factors 56 (1): 112–130. doi:10.1177/0018720813491114.
   PubMed Web of Science ®Google Scholar
• Gonçalves, J. S., C. S. Moriguchi, K. S. Takekawa, H. J. C. G. Coury, and T. de Oliveira Sato. 2017. “The Effects of Forearm Support and Shoulder Posture on Upper Trapezius and Anterior Deltoid Activity.” Journal of Physical Therapy Science 29 (5): 793–798. doi:10.1589/jpts.29.793.
   PubMedGoogle Scholar
• Gustafsson, E., P. Coenen, A. Campbell, and L. Straker. 2018. “Texting with Touchscreen and Keypad Phones-A Comparison of Thumb Kinematics, Upper Limb Muscle Activity, Exertion, Discomfort, and Performance.” Applied Ergonomics 70: 232–239. doi:10.1016/j.apergo.2018.03.003.
   PubMed Web of Science ®Google Scholar
• Habibi, P., and D. Chattopadhyay. 2021. “The Impact of Handedness on User Performance in Touchless Input.” International Journal of Human-Computer Studies 149: 102600. doi:10.1016/j.ijhcs.2021.102600.
   Web of Science ®Google Scholar
• Hagberg, M. 1979. “The Amplitude Distribution of Surface EMG in Static and Intermittent Static Muscular Performance.” European Journal of Applied Physiology and Occupational Physiology 40 (4): 265–272. doi:10.1007/bf00421518.
   PubMed Web of Science ®Google Scholar
• Hansson, GÅ, I. Balogh, K. Ohlsson, L. Rylander, and S. Skerfving. 1996. “Goniometer Measurement and Computer Analysis of Wrist Angles and Movements Applied to Occupational Repetitive Work.” Journal of Electromyography and Kinesiology 6 (1): 23–35. doi:10.1016/1050-6411(95)00017-8.
   PubMed Web of Science ®Google Scholar
• Hoffmann, F., M. I. Tyroller, F. Wende, and N. Henze. 2019, November. “User-defined Interaction for Smart Homes: Voice, Touch, or mid-air Gestures?” Proceedings of the 18th International Conference on Mobile and Ubiquitous multimedia. (pp. 1–7). doi:10.1145/3365610.3365624
   Google Scholar
• Hoggan, E., Nacenta, M., Kristensson, P. O., Williamson, J., Oulasvirta, A., & A Lehtiö. (2013). “Multi-touch Pinch Gestures: Performance and Ergonomics.” In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces (219–222). doi:10.1145/2512349.2512817.
   Google Scholar
• Holmes, M. W., J. Tat, and P. J. Keir. 2015. “Neuromechanical Control of the Forearm Muscles During Gripping with Sudden Flexion and Extension Wrist Perturbations.” Computer Methods in Biomechanics and Biomedical Engineering 18 (16): 1826–1834. doi:10.1080/10255842.2014.976811.
   PubMed Web of Science ®Google Scholar
• Huang, J., M. Qi, L. Mao, M. An, T. Ji, and R. Han. 2021. “User-Defined Gestures for Mid-Air Interaction: A Comparison of Upper Limb Muscle Activity, Wrist Kinematics, and Subjective Preference.” International Journal of Human–Computer Interaction, 1–22. doi:10.1080/10447318.2021.1898825.
   Web of Science ®Google Scholar
• Huang, S., S. P. Ranganathan, and I. Parsons. 2020. “To touch or not to touch? Comparing Touch, mid-air gesture, mid-air haptics for public display in post COVID-19 society.”In SIGGRAPH Asia 2020 Posters (SA '20). Association for Computing Machinery, New York, NY, USA, Article 5, 1–2. doi:10.1145/3415264.3425438.
   Google Scholar
• Jakobsen, M. R., Y. Jansen, S. Boring, and K. Hornbæk. 2015, September. Should I stay or should I go? Selecting between touch and mid-air gestures for large-display interaction. In IFIP Conference on Human-Computer Interaction (pp. 455-473). Springer, Cham. doi: 10.1007/978-3-319-22698-9_31.
   Google Scholar
• Janssen, M. M., J. Harlaar, and I. J. de Groot. 2015. “Surface EMG to Assess arm Function in Boys with DMD: a Pilot Study.” Journal of Electromyography and Kinesiology 25 (2): 323–328. doi:10.1016/j.jelekin.2015.01.008.
   PubMed Web of Science ®Google Scholar
• Jenkins, D. B. 2008. Hollinshead’s Functional Anatomy of the Limbs and Back-E-Book. Canada: Elsevier Health Sciences.
   Google Scholar
• Jeong, H., and Y. Liu. 2017. “Effects of Touchscreen Gesture’s Type and Direction on Finger-Touch Input Performance and Subjective Ratings.” Ergonomics 60 (11): 1528–1539. doi:10.1080/00140139.2017.1313457.
   PubMed Web of Science ®Google Scholar
• Jonsson, P., P. W. Johnson, M. Hagberg, and M. Forsman. 2011. “Thumb Joint Movement and Muscular Activity During Mobile Phone Texting–A Methodological Study.” Journal of Electromyography and Kinesiology 21 (2): 363–370. doi:10.1016/j.jelekin.2010.10.007.
   PubMed Web of Science ®Google Scholar
• Jung, M. C., and M. S. Hallbeck. 2002. “The Effect of Wrist Position, Angular Velocity, and Exertion Direction on Simultaneous Maximal Grip Force and Wrist Torque Under the Isokinetic Conditions.” International Journal of Industrial Ergonomics 29 (3): 133–143. doi:10.1016/S0169-8141(01)00058-0.
   Web of Science ®Google Scholar
• Kang, H., and G. Shin. 2014. “Hand Usage Pattern and Upper Body Discomfort of Desktop Touchscreen Users.” Ergonomics 57 (9): 1397–1404. doi:10.1080/00140139.2014.924574.
   PubMed Web of Science ®Google Scholar
• Kurtenbach, G. P. 1993. The Design and Evaluation of Marking Menus (pp. 1-173). Toronto: University of Toronto.
   Google Scholar
• Kurtenbach, G., and E. A. Hulteen. 1990. “Gestures in Human-Computer Communication.” In Laurel, B., Ed., The art of Human-Computer Interface Design. New York: Addison-Wesley Publishing Company, Inc.
   Google Scholar
• Leitão, R. A. 2012. Creating mobile gesture-based interaction design patterns for older adults: A study of tap and swipe gestures with Portuguese seniors.
   Google Scholar
• Li, W., X. Wan, Y. Shi, N. Yao, C. Wang, and Z. Gao. 2021. “Depth and Breadth of Pie Menus for Mid-air Gesture Interaction.” International Journal of Human–Computer Interaction 37 (2): 131–140. doi:10.1080/10447318.2020.1809245.
   Web of Science ®Google Scholar
• Liang, S. F. M., M. Menozzi, and Y. Y. R. Huang. 2019. “A Mechanism Based on Finger-Sliding Behavior for Designing Radial Menus.” International Journal of Industrial Ergonomics 74: 102869. doi:10.1016/j.ergon.2019.102869.
   Web of Science ®Google Scholar
• Malik, Z. H., and M. Arfan. 2019, March. Evaluation of accuracy: a comparative study between touch screen and midair gesture input. In Future of Information and Communication Conference (pp. 448-462). Springer, Cham. doi: 10.1007/978-3-030-12388-8_32.
   Google Scholar
• May, K. R., T. M. Gable, and B. N. Walker. 2014, September. “A Multimodal air Gesture Interface for in Vehicle Menu Navigation.” Adjunct Proceedings of the 6th International Conference on automotive user interfaces and interactive vehicular applications. (pp. 1–6). doi:10.1145/2667239.2667280
   Google Scholar
• Müller, J., L. Rieger, I. Aslan, C. Anneser, M. Sandstede, F. Schwarzmeier, … E. André. 2019, November. Mouse, Touch, Or Fich: Comparing Traditional Input Modalities To A Novel Pre-Touch Technique. In Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia (pp. 1-7). doi: 10.1145/3365610.3365622.
   Google Scholar
• Putz, R., and R. Pabst. 2006. Sobotta-Atlas of Human Anatomy: Head, Neck, Upper Limb, Thorax, Abdomen, Pelvis, Lower Limb; Two-volume set.
   Google Scholar
• Ryu, J., W. P. Cooney, L. J. Askew, K. N. An, and E. Y. Chao. 1991. “Functional Ranges of Motion of the Wrist Joint.” The Journal of Hand Surgery 16 (3): 409–419. doi:10.1016/0363-5023(91)90006-W
   PubMed Web of Science ®Google Scholar
• Schwartz, C., F. Tubez, F. C. Wang, J. L. Croisier, O. Brüls, V. Denoël, and B. Forthomme. 2017. “Normalizing Shoulder EMG: An Optimal set of Maximum Isometric Voluntary Contraction Tests Considering Reproducibility.” Journal of Electromyography and Kinesiology 37: 1–8. doi:10.1016/j.jelekin.2017.08.005.
   PubMed Web of Science ®Google Scholar
• Shin, D. 2019. “How do Technological Properties Influence User Affordance of Wearable Technologies?” Interaction Studies 20 (2): 307–338. doi:10.1075/is.16024.shi.
   Web of Science ®Google Scholar
• Shin, D. 2019. “How do Users Experience the Interaction with an Immersive Screen?” Computers in Human Behavior 98: 302–310. doi:10.1016/j.chb.2018.11.010.
   Web of Science ®Google Scholar
• Shin, D., M. Choi, J. H. Kim, and J. G. Lee. 2016. “Interaction, Engagement, and Perceived Interactivity in Single-Handed Interaction.” Internet Research 26 (5): 1134–1157. doi:10.1108/IntR-12-2014-0312.
   Web of Science ®Google Scholar
• Tu, H., Q. Huang, Y. Zhao, and B. Gao. 2020. “Effects of Holding Postures on User-Defined Touch Gestures for Tablet Interaction.” International Journal of Human-Computer Studies 141 (102451): 1–10. doi:10.1016/j.ijhcs.2020.102451.
   Web of Science ®Google Scholar
• Vogiatzidakis, P., and P. Koutsabasis. 2018. “Gesture Elicitation Studies for mid-air Interaction: A Review.” Multimodal Technologies and Interaction 2 (4): 65. doi:10.3390/mti2040065.
   Google Scholar
• Wang, W., X. Qin, C. Zheng, H. Wang, J. Li, and J. Niu. 2018. “Mechanical Energy Expenditure-Based Comfort Evaluation Model for Gesture Interaction.” Computational Intelligence and Neuroscience 2018: 1–11. doi:10.1155/2018/9861697.
   Web of Science ®Google Scholar
• Wu, S., T. Gable, K. May, Y. M. Choi, and B. N. Walker. 2016. “Comparison of Surface Gestures and air Gestures for in-Vehicle Menu Navigation.” Archives of Design Research 29 (4): 65–80. doi:10.15187/adr.2016.11.29.4.65.
   Google Scholar
• Wu, H., Y. Wang, J. Qiu, J. Liu, and X. Zhang. 2019. “User-defined Gesture Interaction for Immersive VR Shopping Applications.” Behaviour & Information Technology 38 (7): 726–741. doi:10.1080/0144929X.2018.1552313.
   Web of Science ®Google Scholar