Usability: Lessons Learned … and Yet to Be Learned

Abstract

The philosopher of science J. W. Grove (1989) once wrote, “There is, of course, nothing strange or scandalous about divisions of opinion among scientists. This is a condition for scientific progress” (p. 133). Over the past 30 years, usability, both as a practice and as an emerging science, has had its share of controversies. It has inherited some from its early roots in experimental psychology, measurement, and statistics. Others have emerged as the field of usability has matured and extended into user-centered design and user experience. In many ways, a field of inquiry is shaped by its controversies. This article reviews some of the persistent controversies in the field of usability, starting with their history, then assessing their current status from the perspective of a pragmatic practitioner. Put another way: Over the past three decades, what are some of the key lessons we have learned, and what remains to be learned? Some of the key lessons learned are:

• When discussing usability, it is important to distinguish between the goals and practices of summative and formative usability.

• There is compelling rational and empirical support for the practice of iterative formative usability testing—it appears to be effective in improving both objective and perceived usability.

• When conducting usability studies, practitioners should use one of the currently available standardized usability questionnaires.

• Because “magic number” rules of thumb for sample size requirements for usability tests are optimal only under very specific conditions, practitioners should use the tools that are available to guide sample size estimation rather than relying on “magic numbers.”

Additional information

Notes on contributors

James R. Lewis

James R. Lewis has been a usability practitioner at IBM since 1981. His books include Practical Speech User Interface Design (2011) and Quantifying the User Experience (2012, with Jeff Sauro). He serves on the editorial boards of the International Journal of Human-Computer Interaction and the Journal of Usability Studies, is an IBM Master Inventor with more than 80 U.S. patents, and is currently president of the Association for Voice Interaction Design.