Investigating the Influence of Different Factors on the UX Evaluation of a Mobile Application

References

• Alhadreti, O., & Mayhew, P. (2018). Rethinking thinking aloud: A comparison of three think-aloud protocols (pp. 1–12). ACM Press.
   Google Scholar
• Alves, R., Valente, P., & Nunes, N. J. (2014). The state of user experience evaluation practice (pp. 93–102). ACM Press.
   Google Scholar
• Ardito, C., Buono, P., Caivano, D., Costabile, M. F., & Lanzilotti, R. (2014). Investigating and promoting UX practice in industry: An experimental study. International Journal of Human-Computer Studies, 72(6), 542–551. https://doi.org/10.1016/j.ijhcs.2013.10.004
   Web of Science ®Google Scholar
• Bargas-Avila, J. A., & Hornbaek, K. (2011). Old wine in new bottles or novel challenges: A critical analysis of empirical studies of user experience (pp. 2689–2698). ACM.
   Google Scholar
• Bopp, J. A., Mekler, E. D., & Opwis, K. (2016). Negative emotion, positive experience? Emotionally moving moments in digital games (pp. 2996–3006). ACM.
   Google Scholar
• Borsci, S., Federici, S., Bacci, S., Gnaldi, M., & Bartolucci, F. (2015). Assessing user satisfaction in the era of user experience: Comparison of the sus, umux, and umux-lite as a function of product experience. International Journal of Human-Computer Interaction, 31(8), 484–495. https://doi.org/10.1080/10447318.2015.1064648
   Web of Science ®Google Scholar
• Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9
   PubMed Web of Science ®Google Scholar
• Cockburn, A., Quinn, P., & Gutwin, C. (2015). Examining the peak-end effects of subjective experience (pp. 357–366). ACM Press.
   Google Scholar
• Cockburn, A., Quinn, P., & Gutwin, C. (2017). The effects of interaction sequencing on user experience and preference. International Journal of Human-Computer Studies, 108(1), 89–104. https://doi.org/10.1016/j.ijhcs.2017.07.005
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). L. Erlbaum Associates.
   Google Scholar
• Cowen, A. S., & Keltner, D. (2017). Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proceedings of the National Academy of Sciences, 114(38), E7900–E7909. https://doi.org/10.1073/pnas.1702247114
   PubMed Web of Science ®Google Scholar
• Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
   Web of Science ®Google Scholar
• Cuomo, D. L., & Bowen, C. D. (1994). Understanding usability issues addressed by three user-system interface evaluation techniques. Interacting with Computers, 6(1), 86–108. https://doi.org/10.1016/0953-5438(94)90006-X
   Web of Science ®Google Scholar
• Díaz-Oreiro, I., López, G., Quesada, L., & Guerrero, L. A. (2019). Standardized questionnaires for user experience evaluation: A systematic literature review. Proceedings, 31(1), 1–12. https://doi.org/10.3390/proceedings2019031014
   Google Scholar
• Desurvire, H., Kondziela, J., & Atwood, M. E. (1992). What is gained and lost when using methods other than empirical testing (p. 125). ACM Press. https://doi.org/10.1145/1125021.1125115
   Google Scholar
• Fernandez, A., Abrahão, S., & Insfran, E. (2013). Empirical validation of a usability inspection method for model-driven web development. Journal of Systems and Software, 86(1), 161–186. https://doi.org/10.1016/j.jss.2012.07.043
   Web of Science ®Google Scholar
• Fernandez, A., Insfran, E., & Abrahão, S. (2011). Usability evaluation methods for the web: A systematic mapping study. Information and Software Technology, 53(8), 789–817. https://doi.org/10.1016/j.infsof.2011.02.007
   Web of Science ®Google Scholar
• Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology. General, 141(1), 2–18. https://doi.org/10.1037/a0024338
   PubMed Web of Science ®Google Scholar
• Gutwin, C., Rooke, C., Cockburn, A., Mandryk, R. L., & Lafreniere, B. (2016). Peak-end effects on player experience in casual games. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 5608–5619). https://doi.org/10.1145/2858036.2858419
   Google Scholar
• Hands, D. S., & Avons, S. E. (2001). Recency and duration neglect in subjective assessment of television picture quality. Applied Cognitive Psychology, 15(6), 639–657. https://doi.org/10.1002/acp.731
   Web of Science ®Google Scholar
• Hartson, H. R., Andre, T. S., & Williges, R. C. (2003). Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 15(1), 145–181. https://doi.org/10.1207/S15327590IJHC1501_13
   Web of Science ®Google Scholar
• Hasan, L., Morris, A., & Probets, S. (2012). A comparison of usability evaluation methods for evaluating e-commerce websites. Behaviour & Information Technology, 31(7), 707–737. https://doi.org/10.1080/0144929X.2011.596996
   Web of Science ®Google Scholar
• Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interaction, 19(4), 319–349. https://doi.org/10.1207/s15327051hci1904_2
   Web of Science ®Google Scholar
• Hassenzahl, M. (2018a). The thing and i (summer of ’17 remix). In M. Blythe & A. Monk (Eds.), Funology 2: From usability to enjoyment (pp. 17–31). Springer International Publishing. https://doi.org/10.1007/978-3-319-68213-6_2
   Google Scholar
• Hassenzahl, M. (2018b). The thing and i: Understanding the relationship between user and product. In M. Blythe & A. Monk (Eds.), Funology (Vol. 2, pp. 301–313). Springer.
   Google Scholar
• Hassenzahl, M., Burmester, M., & Koller, F. (2003). Attrakdiff: Ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualität. In Mensch & computer 2003 (pp. 187–196). Springer.
   Google Scholar
• Hassenzahl, M., Diefenbach, S., & Göritz, A. (2010). Needs, affect, and interactive products – Facets of user experience. Interacting with Computers, 22(5), 353–362. https://doi.org/10.1016/j.intcom.2010.04.002
   Web of Science ®Google Scholar
• Hassenzahl, M., Kekez, R., & Burmester, M. (2002). The importance of a software’s pragmatic quality depends on usage modes. In Proceedings of the 6th International Conference on Work With Display Units (WWDU 2002) (pp. 275–276).
   Google Scholar
• Hossain, M. A., & Quaddus, M. (2012). Expectation–confirmation theory in information system research: A review and analysis. Information Systems Theory, 28(1), 441–469. https://doi.org/10.1007/978-1-4419-6108-2_21
   Google Scholar
• Hvannberg, E. T., Law, E. L.-C., & Lárusdóttir, M. K. (2007). Heuristic evaluation: Comparing ways of finding and reporting usability problems. Interacting with Computers, 19(2), 225–240. https://doi.org/10.1016/j.intcom.2006.10.001
   Web of Science ®Google Scholar
• Jeffries, R., Miller, J. R., Wharton, C., & Uyeda, K. M. (1991). User interface evaluation in the real world: A comparison of four techniques (pp. 119–124). Association for Computing Machinery. https://doi.org/10.1145/108844.108862
   Google Scholar
• Johannessen, G. H. J., & Hornbaek, K. (2014). Must evaluation methods be about usability? Devising and assessing the utility inspection method. Behaviour & Information Technology, 33(2), 195–206. https://doi.org/10.1080/0144929X.2012.751708
   Web of Science ®Google Scholar
• Kujala, S., Mugge, R., & Miron-Shatz, T. (2017). The role of expectations in service evaluation: A longitudinal study of a proximity mobile payment service. International Journal of Human-Computer Studies, 98(1), 51–61. https://doi.org/10.1016/j.ijhcs.2016.09.011
   Google Scholar
• Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., & Sinnelä, A. (2011). Identifying hedonic factors in long-term user experience. In Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces (pp. 1–8). https://doi.org/10.1145/2347504.2347523
   Google Scholar
• Lallemand, C., Gronier, G., & Koenig, V. (2015). User experience: A concept without consensus? exploring practitioners’ perspectives through an international survey. Computers in Human Behavior, 43(1), 35–48. https://doi.org/10.1016/j.chb.2014.10.048
   Google Scholar
• Lallemand, C., Koenig, V., & Gronier, G. (2014). How relevant is an expert evaluation of user experience based on a psychological needs-driven approach? (pp. 11–20). ACM Press. https://doi.org/10.1145/2639189.2639214
   Google Scholar
• Law, E. L.-C., & Hvannberg, E. T. (2002). Complementarity and convergence of heuristic evaluation and usability test: A case study of universal brokerage platform (pp. 71–80). Association for Computing Machinery. https://doi.org/10.1145/572020.572030
   Google Scholar
• Maguire, M., & Isherwood, P. (2018). A comparison of user testing and heuristic evaluation methods for identifying website usability problems. In A. Marcus & W. Wang (Eds.), Design, user experience, and usability: Theory and practice (Vol. 10918, pp. 429–438). Springer International Publishing.
   Google Scholar
• Marques, L., Matsubara, P. G., Nakamura, W. T., Ferreira, B. M., Wiese, I. S., Gadelha, B. F., Zaina, L. M., Redmiles, D., & Conte, T. U. (2021). Understanding UX better: A new technique to go beyond emotion assessment. Sensors, 21(21), 7183. https://doi.org/10.3390/s21217183
   PubMed Web of Science ®Google Scholar
• Marques, L. C., Nakamura, W. T., Valentim, N., Rivero, L., & Conte, T. (2018). Do scale type techniques identify problems that affect user experience? User experience evaluation of a mobile application(s) (pp. 451–501). KSI Research Inc. https://doi.org/10.18293/SEKE2018-161
   Google Scholar
• Martens, D., & Maalej, W. (2019). Release early, release often, and watch your users’ emotions: Lessons from emotional patterns. IEEE Software, 36(5), 32–37. https://doi.org/10.1109/MS.2019.2923603
   Web of Science ®Google Scholar
• Matera, M., Costabile, M., Garzotto, F., & Paolini, P. (2002). Sue inspection: An effective method for systematic usability evaluation of hypermedia. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 32(1), 93–103. https://doi.org/10.1109/3468.995532
   Web of Science ®Google Scholar
• McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/BM.2012.031
   PubMed Web of Science ®Google Scholar
• Moreno, A. M., Seffah, A., Capilla, R., & Sanchez-Segura, M.-I. (2013). HCI practices for building usable software. Computer Magazine, 46(4), 100–102. https://doi.org/10.1109/MC.2013.133
   Web of Science ®Google Scholar
• Nakamura, W., Marques, L., Ferreira, B., Barbosa, S., & Conte, T. (2020). To inspect or to test? What approach provides better results when it comes to usability and UX? In Proceedings of the 22nd International Conference on Enterprise Information Systems – Volume 2: ICEIS (pp. 487–498). SciTePress. https://doi.org/10.5220/0009367904870498
   Google Scholar
• Nakamura, W. T., Marques, L. C., Redmiles, D., Oliveira, E. H. T. d., & Conte, T. (2022). Supplementary Material for the article entitled “Investigating the influence of different factors on the UX evaluation of a mobile application” (Technical Report). Retrieved from https://doi.org/10.6084/m9.figshare.20055833
   Google Scholar
• Nakamura, W. T., Marques, L. C., Rivero, L., Oliveira, E. H. T. d., & Conte, T. (2017). In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação - SBIE), Are generic ux evaluation techniques enough? A study on the Ux evaluation of the Edmodo learning management system (p. 1007).
   Google Scholar
• Nakamura, W. T., de Oliveira, E. H. T., & Conte, T. (2019). Negative emotions, positive experience: What are we doing wrong when evaluating the UX? (pp. 1–6). ACM Press.
   Google Scholar
• Nielsen, J. (1994). Heuristic evaluation. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 25–62). Springer.
   Google Scholar
• Pettersson, I., Lachner, F., Frison, A.-K., Riener, A., & Butz, A. (2018). A Bermuda triangle? A review of method application and triangulation in user experience evaluation (pp. 461:1–461:16). ACM.
   Google Scholar
• Rapp, A., Curti, L., & Boldi, A. (2021). The human side of human-chatbot interaction: A systematic literature review of ten years of research on text-based chatbots. International Journal of Human-Computer Studies, 151(1), 102630. https://doi.org/10.1016/j.ijhcs.2021.102630
   Google Scholar
• Rivero, L., & Conte, T. (2017). A systematic mapping study on research contributions on UX evaluation technologies (pp. 1–10). ACM Press. https://doi.org/10.1145/3160504.3160512
   Google Scholar
• Roto, V., & Lund, A. (2013). In CHI ‘13 Extended Abstracts on Human Factors in Computing Systems, On top of the user experience wave – How is our work changing? (pp. 2521–2524). Association for Computing Machinery. https://doi.org/10.1145/2468356.2468820
   Google Scholar
• Sagnier, C., Loup-Escande, E., & Valléry, G. (2020). Effects of gender and prior experience in immersive user experience with virtual reality. In T. Ahram & C. Falcão (Eds.), Advances in usability and user experience (Vol. 972, pp. 305–314). Springer International Publishing.
   Google Scholar
• Schmitt, B. (1999). Experiential marketing. Journal of Marketing Management, 15(1–3), 53–67. https://doi.org/10.1362/026725799784870496
   Google Scholar
• Schneidermeier, T., Hertlein, F., & Wolff, C. (2014). Changing paradigm – Changing experience? Comparative usability evaluation of windows 7 and windows 8. In A. Marcus (Ed.), Design, user experience, and usability. Theories, methods, and tools for designing the user experience (Vol. 8517, pp. 371–382). Springer International Publishing.
   Google Scholar
• Schrepp, M., Hinderks, A., & Thomaschewski, J. (2017). Design and evaluation of a short version of the user experience questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence, 4(6), 103. https://doi.org/10.9781/ijimai.2017.09.001
   Web of Science ®Google Scholar
• Shteingart, H., Neiman, T., & Loewenstein, Y. (2013). The role of first impression in operant learning. Journal of Experimental Psychology: General, 142(2), 476–488. https://doi.org/10.1037/a0029550
   PubMed Web of Science ®Google Scholar
• Vermeeren, A. P. O. S., Law, E. L.-C., Roto, V., Obrist, M., Hoonhout, J., & Väänänen-Vainio-Mattila, K. (2010). User experience evaluation methods: Current state and development needs (pp. 521–530). ACM.
   Google Scholar
• Williams, B., Onsman, A., & Brown, T. (2010). Exploratory factor analysis: A five-step guide for novices. Australasian Journal of Paramedicine, 8(3), 1–13. https://doi.org/10.33151/ajp.8.3.93
   Google Scholar
• Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media.
   Google Scholar
• Zeithaml, V. A., Parasuraman, A., Berry, L. L., & Berry, L. L. (1990). Delivering quality service: Balancing customer perceptions and expectations. Simon and Schuster.
   Google Scholar