When Smartphone Responses Are Not Unreliable

Abstract

As online surveys have become popular, concerns have emerged about the quality of responses submitted via mobile technology, such as limited text readability, increased distractions, etc. To assess the validity of this concern, this study examines the effect of medium (PC, smartphone) on the response reliability of Big-Five Mini-Markers for personality. Results provide little support for the concern, indicating that responses from smartphones are as reliable as those submitted from PCs. Conditions where mobile devices may not suffer with poor reliability are discussed.

Keywords:

• Data Quality
• Mobile Device
• Response Reliability
• Web Survey

Notes

[1] Full results (α per measured personality and condition, z-scores, SEs, and p values) can be obtained from the corresponding author on request.

[2] The distribution comprising sum of two independent z-scores was created and evaluated using R simulation.

[3] Current results should be interpreted cautiously, noting that the tests were somewhat underpowered with the relatively small total sample size. The sizes of the cells actually used for the tests, however, either exceeded or marginally fell short of the recommended minimum threshold (n ≈ 30; see Kim & Feldt, 2008).