Seeing (Movement) is Believing: The Effect of Motion on Perception of Automatic Systems Performance

Abstract

In this article, we report on one lab study and seven follow-up studies on a crowdsourcing platform designed to investigate the potential of animation cues to influence users’ perception of two smart systems: a handwriting recognition and a part-of-speech tagging system. Results from the first three studies indicate that animation cues can influence a participant’s perception of both systems’ performance. The subsequent three studies, designed to try and identify an explanation for this effect, suggest that this effect is related to the participants’ mental model of the smart system. The last two studies were designed to characterize the effect more in detail, and they revealed that different amounts of animation do not seem to create substantial differences and that the effect persists even when the system’s performance decreases, but only when the difference in performance level between the systems being compared is small.

Notes

¹ https://www.apertium.org.

² https://www.camscanner.com/.

³ https://www.mturk.com.

⁴ Casual interaction is when users are not able to, or do not want to, fully engage with their devices.

⁵ That is the identification of the syntactic role that each word has, e.g., verb, noun, adjective, etc.

⁶ https://www.turkprime.com/Home/FrequentlyAskedQuestions.

⁷ One participant in Study 6, and two in each of Study 7 and Study 8.

⁸ tThis information was displayed in the tasks’ description.

⁹ https://www.mturk.com.

¹⁰ For example, http://www.nltk.org/.

¹¹ This effect may be reminiscent of “priming,” however, we prefer to refer to “salience” rather than priming because priming generally refers to the situation where participants are shown a stimulus that causes an implicit memory effect (Meyer & Schvaneveldt, 1971). In our study, participants are explicitly asked to think, or remember, how a handwriting recognition system could work, so we do not consider this as related to implicit memory.

Additional information

Funding

This work was supported by a PhD scholarship by CONACyT, SICYT Morelos, and by the EPSRC A-IoT project (EP/N014243/2);Engineering and Physical Sciences Research Council [EP/N014243/2].

Notes on contributors

Pedro García García

Pedro García García ([email protected], URL) is a student with an interest in <Human-Computer Interaction; he is a PhD. Student in the electronics and computer science department of University of Southampton.

Enrico Costanza

Enrico Costanza ([email protected], https://hci.ecs.soton.ac.uk/enrico/) is a researcher with an interest in <Human-Computer Interaction; he is a Senior Lecture in the UCL Interaction Centre of University College London.

Jhim Verame

Jhim Verame ([email protected], URL) is a student with an interest in Human-Computer Interaction; he is a PhD. Student in the electronics and computer science department of University of Southampton.

Diana Nowacka

Diana Nowacka ([email protected], URL) is a researcher with an interest in Human-Computer Interaction; she is a Researcher Associate in the UCL Interaction Centre of University College London.

Sarvapali D. Ramchurn

Sarvapali D. Ramchurn ([email protected], URL) is a researcher with an interest in Human-Computer Interaction; he is an Associate Professor in the electronics and computer science department of University of Southampton.