Challenges to Assessing Usability in the Wild: A Case Study

REFERENCES

• Agarwal, R., & Prasad, J. (1997). The role of innovation characteristics and perceived voluntariness in the acceptance of information technologies. Decision Sciences, 28, 557–582.
   Web of Science ®Google Scholar
• Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kuhl & J. Beckmann (Eds.), Action control: From cognition to behavior (pp. 11–39). New York, NY: Springer Verlag.
   Google Scholar
• Baddeley, A. D., & Della Sala, S. (1996). Working memory and executive control. Philosophical Transactions of the Royal Society of London, 351, 1397–1404.
   PubMed Web of Science ®Google Scholar
• Bargas-Avila, J., Oberholzer, G., Schmutz, P., De Vito, M., & Opwis, K. (2007). Usable error message presentation in the World Wide Web: Do not show errors right away. Interacting with Computers, 19, 330–341.
   Web of Science ®Google Scholar
• Bevan, N. (2009). What is the difference between the purpose of usability and user experience evaluation methods? In Proceedings of the 12th IFIP TC13 Conference on Human-Computer Interaction, INTERACT’09. Uppsala, Sweden. Retrieved from http://www.nigelbevan.com/papers/What_is_the_difference_between_usability_and_user_experience_evaluation_methods
   Google Scholar
• Blythe, M. A., Overbeeke, K., Monk, A. F., & Wright, P. C. (Eds.). (2004). Funology: From usability to enjoyment. Dordrecht, the Netherlands: Kluwer Academic.
   Google Scholar
• Brown, S. A., Massey, A. P., Montoya-Weiss, M. M., & Burkman, J. R. (2002). Do I really have to? User acceptance of mandated technology. European Journal of Information Systems, 11, 283–295.
   Web of Science ®Google Scholar
• Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233–246.
   Web of Science ®Google Scholar
• Chattratichart, J., & Brodie, J. (2004). Applying user testing data to UEM performance metrics. Proceedings of CHI 2004, 1119–1122.
   Google Scholar
• Chattratichart, J., & Lindgaard, G. (2008). A comparative evaluation of heuristic-based usability inspection methods. Proceedings of CHI 2008, 2213–2220.
   Google Scholar
• Chuttur, M. Y. (2009). Overview of the technology acceptance model: Origins, developments and future directions. Working Papers on Information Systems, 9(37), 1–22.
   Google Scholar
• Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. London, UK: Harper & Row.
   Google Scholar
• Davis, F. D., Jr. (1985). A technology acceptance model for empirically testing new end-user information systems: Theory and results (Unpublished doctoral thesis). Cambridge, MA: MIT.
   Google Scholar
• Davis, F., Bagozzi, R., & Warshaw, P. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35, 982–1003.
   Web of Science ®Google Scholar
• Davis, F., Bagozzi, R., & Warshaw, P. (1992). Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22, 1111–1132.
   Web of Science ®Google Scholar
• Dekker, S. W. A., & Laursen, T. (2007). From punitive action to confidential reporting. Patient Safety Quality Healthcare, 5, 50–56.
   Google Scholar
• Desmet, P., & Hekkert, P. (2007). Framework of product experience. International Journal of Design, 1, 1–12.
   Web of Science ®Google Scholar
• Endsley, M. R. (1995a). Measurement of situation awareness in dynamic systems. Human Factors, 37, 65–84.
   Web of Science ®Google Scholar
• Endsley, M. R. (1995b). Towards a theory of situation awareness in dynamic systems. Human Factors, 37, 32–64.
   Web of Science ®Google Scholar
• Ferguson, G. A., & Takane, Y. (1999). Statistical analysis in psychology and education (6th ed.). Toronto, Canada: McGraw-Hill.
   Google Scholar
• Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.
   Google Scholar
• Hale, A., Guldenmund, F., & Gossens, I. (2006). Auditing resilience in risk control and safety management systems. In E. Hollnagel, D. D. Woods, & N. Leveson (Eds.), Resilience engineering concepts and precepts (pp. 289–314). Aldershot, UK: Ashgate.
   Google Scholar
• Hartmann, J., Sutcliffe, A., & De Angeli, A. (2008). Towards a theory of user judgment of aesthetics and user interface quality. Association for Computing Machinery Transactions on Computer-Human Interaction, 15, 1–30.
   Web of Science ®Google Scholar
• Hartwick, J., & Barki, H. (1994). Explaining the role of user participation in information system use. Management Science, 40, 440–465.
   Web of Science ®Google Scholar
• Hassenzahl, M. (2003). The thing and I: Understanding the relationship between user and product. In M. Blythe, C. Overbeeke, A. F. Monk, & P. C. Wright (Eds.), Funology: From usability to enjoyment (pp. 31–42). Dordrecht, the Netherlands: Kluwer.
   Google Scholar
• Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products. Human–Computer Interaction, 19, 319–349.
   Web of Science ®Google Scholar
• Hassenzahl, M. (2008). Aesthetics in interactive products: Correlates and consequences of beauty. In H. N. J. Schifferstein & P. Hekkert (Eds.), Product experience (pp. 287–302). San Diego, CA: Elsevier Science.
   Google Scholar
• Hassenzahl, M., & Monk, A. (2010). The inference of perceived usability from beauty. Human–Computer Interaction, 25, 235–260.
   Web of Science ®Google Scholar
• Hassenzahl, M., & Tractinsky, N. (2006). User experience—A research agenda. Behavior and Information Technology, 25, 91–97.
   Web of Science ®Google Scholar
• Hollnagel, E. (2006). Resilience—The challenge of the unstable. In E. Hollnagel, D. D. Woods, & N. Leveson (Eds.), Resilience engineering concepts and precepts (pp. 9–18). Aldershot, UK: Ashgate.
   Google Scholar
• Huber, S., van Wijgerden, I., de Witt, A., & Dekker, S. W. A. (2009). Learning from organizational incidents: Resilience engineering for high-risk process environments. Process Safety Progress, 28, 90–95.
   Web of Science ®Google Scholar
• ISO. (1999). ISO 13407: Human centred design processes for interactive systems.
   Google Scholar
• ISO. (2010). ISO 9241-210:2010: Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems.
   Google Scholar
• Jordan, P. W. (2001). An introduction to usability. London, UK: Taylor & Francis.
   Google Scholar
• Kurosu, M., & Kashimura, K. (1995). Apparent usability vs. inherent usability. Proceedings of CHI 1995, 292–293.
   Google Scholar
• Lavie, T., & Tractinsky, N. (2004). Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human-Computer Studies, 60, 269–298.
   Web of Science ®Google Scholar
• Law, E. L., Roto, V., Hassenzahl, M., Vermeeren, A. P. O. S., & Kort, J. (2009). Understanding, scoping and defining user experience: A survey approach. Proceedings of CHI 2009, 719–728.
   Google Scholar
• Law, E., Roto, V., Vermeeren, A. P. O. S., & Hassenzahl, M. (2008). Towards a consensual definition of user experience. Proceedings of CHI 2008, 2395–2398.
   Google Scholar
• Law, E. L., & Van Schaik, P. (2010). Modelling user experience: An agenda for research and practice. Interacting with Computers, 22, 313–322.
   Web of Science ®Google Scholar
• Lewis, C., & Wharton, C. (1997). Cognitive walkthrough. In M. Helander, T. K. Landauer, & P. Prabhu (Eds.), Handbook of human–computer interaction (pp. 717–732). New York, NY: Elsevier Science.
   Google Scholar
• Lindgaard, G. (2014). The usefulness of traditional usability evaluation methods. Interactions, 21, 80–82.
   Google Scholar
• Mars, G. (2009). East-End warehouse: A case study of ‘organisational capture’ and cultural conflicts. Culture and Organization, 15, 237–256.
   Web of Science ®Google Scholar
• Mathieson, K. (1991). Predicting user intentions: Comparing the TAM with the theory of planned behavior. Information Systems Research, 2, 173–191.
   Web of Science ®Google Scholar
• Nielsen, J. (1993). Usability engineering. San Francisco, CA: Morgan Kaufman.
   Google Scholar
• Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. New York, NY: Basic Books.
   Google Scholar
• O’Brien, H. (2011). The influence of hedonic and utilitarian motivations on user engagement: The case of online shopping experiences. Interacting with Computers, 22, 344–352.
   Web of Science ®Google Scholar
• O’Brien, H. L., & Toms, E.G. (2010). The development and evaluation of a survey to measure user engagement in e-commerce environments. Journal of the American Society for Information Science & Technology, 61, 50–69.
   Web of Science ®Google Scholar
• Overby, J., & Lee, E-J. (2006). The effects of utilitarian and hedonic online shopping value on consumer preference and intentions. Journal of Business Research, 59, 1160–1166.
   Web of Science ®Google Scholar
• Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge, UK: Cambridge University Press.
   Google Scholar
• Petersen, M. G., Iversen, O. S., Krogh, P. G., & Ludvigsen, M. (2004). Aesthetic interaction: A pragmatist’s aesthetics of interactive systems. Proceedings of Designing Interactive Systems: Processes, Practices, Methods, and Techniques, 269–276.
   Google Scholar
• Picard, R. W. (1998). Affective computing. Cambridge, MA: MIT Press.
   Google Scholar
• Rasmussen, J. (1979). On the structure of knowledge—A morphology of mental models in a man-machine context (Report No. M-2192). Risø National Research Laboratories, Denmark.
   Google Scholar
• Rittel, H., & Webber, M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.
   Web of Science ®Google Scholar
• Rittel, H., & Webber, M. (1984). Dilemmas in a general theory of planning. In N. Cross (Ed.), Developments in design methodology (pp. 135–144). Chichester, UK: Wiley & Sons.
   Google Scholar
• Roth, S. P., Schmutz, P., Pauwels, S. L., Bargas-Avila, J. A., & Opwis, K. (2010). Mental models for web objects: Where do users expect to find the most frequent objects in online shops, news portals, and company web pages? Interacting with Computers, 22, 140–152.
   Web of Science ®Google Scholar
• Sachau, D. A. (2007). Resurrecting the motivation-hygiene theory: Herzberg and the positive psychology movement. Human Resource Development Review, 6, 373–393.
   Google Scholar
• Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.
   Web of Science ®Google Scholar
• Taylor, S., & Todd, P. A. (1995). Understanding information technology usage: A test of competing models. Information Systems Research, 6, 144–176.
   Web of Science ®Google Scholar
• Tractinsky, N. (1997). Aesthetics and apparent usability: Empirically assessing cultural and methodological issues. Proceedings of CHI 1997, 115–122.
   Google Scholar
• Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13, 127–145.
   Web of Science ®Google Scholar
• van Schaik, P., Hassenzahl, M., & Ling, J. (2012). User-experience from an inference perspective. Association for Computing Machinery Transactions on Computer-Human Interaction, 19, 11:1–11: 25.
   Google Scholar
• Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information Systems Research, 11, 342–365.
   Web of Science ®Google Scholar
• Venkatesh, V., & Davis, F. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46, 186–202.
   Web of Science ®Google Scholar
• Weiser, M., & Seely-Brown, J. (1991). Designing calm technology. PowerGrid Journal, 1.01, http://powergrid.electriciti.com/1.01
   Google Scholar
• Wharton, C., Rieman, J., Lewis, C., & Polson, P. (1994). The cognitive walkthrough method: A practitioner’s guide. In J. Nielsen & R. Mack (Eds.), Usability inspection methods (pp. 105–141). New York, NY: Wiley & Sons.
   Google Scholar
• Woods, D. D. (2006). Essential characteristics of resilience. In E. Hollnagel, D. D. Woods, & N. Leveson (Eds.), Resilience engineering concepts and precepts (pp. 21–34). Aldershot, UK: Ashgate.
   Google Scholar
• Wreathall, J. (2006). Properties of resilient organizations: An initial view. In E. Hollnagel, D. D. Woods, & N. Leveson (Eds.), Resilience engineering concepts and precepts (pp. 275–286). Aldershot, UK: Ashgate.
   Google Scholar