Measuring the Effects of Usability Recommendations for Mobile Instant Messaging Apps on User Performance and User Behaviour

Abstract

Researchers worldwide have extensively investigated the influence of Human-Computer Interaction (HCI) on Mobile Instant Messaging applications, exploring aspects such as usability issues and user behaviour. However, traditional HCI challenges remain in these applications today (ie, usability issues and lack of specific usability guidelines). In this paper, we present a contextual and methodological contribution measuring user performance (with a controlled experiment on a set of structured tasks following a within-subjects design) and user behaviour (with a user satisfaction questionnaire) on Mobile Instant Messaging applications with end users. Most of the current studies on Mobile Instant Messaging are focused on only one application at a time and in a specific region of the world. This experiment originally measures over four apps: Three in-market apps (apps with good usability results, selected from previous studies) and one app developed following usability recommendations. In addition, this research originally adds to the current literature results with participants from two different countries. The results show, as practical implications, that the application of specific usability recommendations for Mobile Instant Messaging applications (eg, adding a new contact only with the ID, or providing a visual distinction between individual and group chats, among others) significantly facilitates user performance with these applications. However, further dissemination is needed among researchers and developers. The findings on user behaviour show that the number of close contacts using a particular Mobile Instant Messaging application strongly determines the user’s intention to continue using an application. And not the features offered or the acceptance of the user interface, as most current studies suggest.

Keywords:

• Usability recommendations
• mobile instant messaging
• mobile app
• user performance
• user behaviour

Acknowledgments

This work has been partly supported by the project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 739578 (RISE – Call: H2020-WIDESPREAD-01-2016-2017-TeamingPhase2) and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.

This research has been supported by a PMI research group from the University of Alcala (UAH).

The authors would like to thank Professor George Ghinea and his team, from Brunel University London, for their invaluable help in recruiting participants for this study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The participants of this study did not give written consent for their data to be shared publicly, so due to the sensitive nature of the research supporting data is not available.

Additional information

Funding

This research was funded by the FPU research staff education programme of the ‘Universidad de Alcalá’ (UAH), grant number [30400M000.541.A 640.06].

Notes on contributors

Sergio Caro-Alvaro

Sergio Caro-Alvaro is an Assistant Professor in the Computer Science Department at University of Alcala (Spain). He received his PhD degree in the Computer Science Department of the University of Alcala (Spain). His research interests include usability, user experience and gamification.

Eva García-López

Eva García-López has a PhD in Computer Science from the University of Alcala (Spain), where she is now an Associate Professor in the Computer Science Department. Her research interests include artificial intelligence and user experience. She has published more than 100 papers, and has participated in many research projects.

Antonio García-Cabot

Antonio García-Cabot is an Associate Professor in the Computer Science Department at the University of Alcala (Spain). He is the research leader of the “Intelligent Mobile Platforms” research group, he has published more than 100 papers in journals and conferences. His research interests include UX and Natural Language Processing.

Aekaterini Mavri

Aekaterini Mavri is a co-director at the Cyprus Interaction Lab, and a teaching staff member at the Department of Multimedia and Graphic Arts, Cyprus University of Technology. Aekaterini holds a Ph.D. in Educational Technology from CUT. Her interests center around User Experience, Learning Experience, Co-design, and blended learning communities.