Wear + Touch: An Exploration of Wearables for Vibrotactile Feedback During Touchscreen Input

Abstract

Vibrotactile feedback is known to have a positive impact on the user performance and experience of touchscreen input. Unfortunately, not all touchscreen devices have built-in haptics capabilities, public displays being a notable example, while the design options for vibrotactile feedback on conventional mobile devices, such as smartphones and tablets, are still limited. In this work, we propose leveraging commodity wearable devices, such as smartwatches, rings, and armbands, to enable vibrotactile feedback as closely as possible to the touch point when the touchscreen does not natively support haptics, and to complement existing haptics capabilities when the touchscreen already features them. To this end, we introduce Wear + Touch, a set of four interaction techniques designed to augment touchscreen input with vibrations delivered through wearable devices, from substituting to complementing, extending, and expanding on-screen vibrotactile feedback. We demonstrate Wear + Touch with several practical implementations involving a large touchscreen display, smartphone, tablet, smartwatch, smart ring, and armband as well as a custom wearable device that we designed for vibrations delivered simultaneously on the user’s index finger, wrist, and forearm. We also report empirical results from two controlled experiments involving a total of 40 participants, conducted to understand the multi-faceted user experience of vibrotactile feedback delivered through the Wear + Touch techniques. In the first experiment, we reveal a favorable user experience of vibrations delivered on the index finger, wrist, and forearm, in this order, when the touchscreen lacks haptics capabilities. In the second experiment, we reveal benefits of reinforcing and expanding on-screen vibrations with vibrotactile feedback on the index finger, wrist, and arm, respectively. We use our findings to suggest future work opportunities for combined vibrotactile feedback involving both touchscreens and wearable devices.

Keywords:

• Touch input
• wearables
• vibrotactile feedback
• touchscreens
• watches
• rings
• armbands
• haptics

Disclosure statement

No potential competing interest was reported by the author(s).

Notes

1 https://us.amazon.com/Ring-R001-PB02-ZERO-M-Black/dp/B00U8IX7FO

2 https://www.rfc-editor.org/rfc/rfc6455

3 https://developer.mozilla.org/en-US/docs/Web/API/Vibration/_API

4 Adapted from https://github.com/Davigl/vue-music-player.

5 https://nfpshop.com/product/10mm-coin-vibration-motor-3mm-type-model-nfp-c1030

6 https://esp8266-shop.com/product/nodemcu-esp8266-esp-12e

7 https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API

8 https://leafletjs.com

9 https://www.ideum.com/products/touch-tables

10 https://cran.r-project.org/web/packages/ARTool/index.html

11 https://web.dev/explore/progressive-web-apps

Additional information

Funding

This work is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement [No. 860114].

Notes on contributors

Mihail Terenti

Mihail Terenti is a Marie Skłodowska-Curie Early Stage Researcher at the Ştefan cel Mare University of Suceava, Romania, where he works on multimodal feedback to augment the user experience of touch input.

Radu-Daniel Vatavu

Radu-Daniel Vatavu is a Professor of Computer Science at the Ştefan cel Mare University of Suceava, where he directs the Machine Intelligence and Information Visualization Research Laboratory. His research interests include human-computer interaction, augmented reality, ambient intelligence, and accessible computing.