SurveyMotion: what can we learn from sensor data about respondents’ completion and response behavior in mobile web surveys?

ABSTRACT

Participation in web surveys via smartphones increased continuously in recent years. The reasons for this increase are a growing proportion of smartphone owners and an increase in mobile Internet access. However, research has shown that smartphone respondents are frequently distracted and/or multitasking, which might affect completion and response behavior in a negative way. We propose ‘SurveyMotion (SMotion)’, a JavaScript-based tool for mobile devices that can gather information about respondents’ motions during web survey completion by using sensor data. Specifically, we collect data about the total acceleration (TA) of smartphones. We conducted a lab experiment and varied the form of survey completion (e.g. standing or walking). Furthermore, we employed questions with different response formats (e.g. radio buttons and sliders) and measured response times. The results reveal that SMotion detects higher TAs of smartphones for respondents with comparatively higher motion levels. In addition, respondents’ motion level affects response times and the quality of responses given. The SMotion tool promotes the exploration of how respondents complete mobile web surveys and could be employed to understand how future mobile web surveys are completed.

KEYWORDS:

• Acceleration
• JavaScript
• mobile sensors
• passive data collection
• response quality
• smartphones
• web survey

Acknowledgments

The authors would like to thank Melanie Revilla and the Advanced Survey Quality Methods research group (RECSM-Universitat Pompeu Fabra) as well as Annelies Blom (University of Mannheim) for their great support. We also are grateful to Björn Dauven (University of Göttingen) for his help during the data collection as well as the anonymous reviewers and the editors for their constructive suggestions for improving this article.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Elhoushi et al. (2017) provide a list of different sensor types, data acquisition approaches, and setups, including brief descriptions. For instance, the freely available application ‘CPU-Z (CPUID)’ for Android devices provides a list of sensors implemented in the device and displays sensor information in real time.

2. The International System unit for acceleration is ‘meter per second squared (m/s²)’.

3. An analysis of the user-agent-strings reveals that these participants used comparatively old devices and/or Internet browser versions. Here, 20 out of the 28 smartphones did not have a gyroscope, magnet sensor, or compass implemented, which hampers the proper gathering of the TA. Furthermore, eight smartphones did not have a browser version installed that supports the gathering of the TA, such as Chrome before version 30 (released in 2013) and iOS Safari before version 4.2 (released in 2010). Schlosser and Höhne (2018b) conducted a usability study with N = 1452 smartphone respondents to explore the technical potentials of measuring acceleration in mobile web surveys by means of SMotion. The study contained data from 29 different smartphone manufacturers, 208 different smartphone models, and 13 different Internet browsers. They found that only for 2.8% (n = 41) of the respondents, no acceleration could be gathered. Thus, the collection of JavaScript-based sensor data (i.e. acceleration) in mobile web surveys is an achievable and promising way to research respondents’ completion and response behavior.

4. It is evident that the sampling rate for each device is constant across all study-relevant questions.

Additional information

Notes on contributors

Jan Karem Höhne

Jan Karem Höhne is a postdoctoral researcher at the Collaborative Research Center 884 “Political Economy of Reforms” at the University of Mannheim (Germany) and a non-resident researcher at the “Research and Expertise Centre for Survey Methodology (RECSM)” at the Universitat Pompeu Fabra (Spain). His main research interests focus on survey methodology, web-survey design, passive data collection, and eye tracking.

Stephan Schlosser

Stephan Schlosser is a doctoral candidate and research associate at the Center of Methods in Social Sciences at the University of Göttingen (Germany). His main research interests focus on web-survey design and passive data collection.