Differences in muscle activity, kinematics, user performance, and subjective assessment between touchscreen and mid-air interactions on a tablet

ABSTRACT

Research on gesture input modalities, including touchscreen and mid-air interactions, is getting more widespread. A few studies have compared these two input modalities according to subjective methods. However, there is no empirical research on their differences in physiological measures. The purpose of this study was to quantify the differences between touchscreen and mid-air interactions when performing swipe gestures in four orthogonal directions on a tablet combining the indices of electromyography (EMG), electrogoniometry, user performance, and subjective assessment. Our results indicated that mid-air interaction obtained significantly lower muscular loads in the upper limb, smaller wrist joint excursions, shorter task completion time, and better subjective ratings than touchscreen interaction when users with elbow support performed swipe gestures on a tablet. We also found that performing swipe gestures in the vertical direction tended to possess higher muscular loads than in the horizontal direction during touchscreen interaction. Besides, we revealed that swipe right had the largest radial/ulnar deviation, and swipe down had the largest flexion/extension excursion. Furthermore, swipe up brought the worst subjective ratings among the four gesture types. These findings could provide a scientific basis for guiding the appropriate selection and use of the two input modalities in the future HCI field.

KEYWORDS:

• Mid-air interaction
• touchscreen interaction
• tablet devices
• EMG
• electrogoniometry

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Natural Science Foundation of China under grant number 61972340.