The Relationship Between System Effectiveness and Subjective Usability Scores Using the System Usability Scale

Abstract

This article examines the relationship between users’ subjective usability assessments, as measured using the System Usability Scale (SUS), and the ISO metric of effectiveness, using task success as the measure. The article reports the results of two studies designed to explore the relationship between SUS scores and user success rates for a variety of interfaces. The first study was a field study, where stereotypical usability assessments on a variety of products and services were performed. The second study was a well-controlled laboratory study where the level of success that users were able to achieve was controlled. For both studies, the relationship between SUS scores and their attendant performance were examined at both the individual level and the average system level. Although the correlations are far from perfect, there are reliable and reasonably strong positive correlations between subjective usability measures and task success rates, for both the laboratory and field studies at both the individual and system level.

Additional information

Notes on contributors

Philip Kortum

Philip Kortum is an assistant professor at Rice University in Houston, Texas. His primary interests are in the research and development of highly usable systems in the voting and mobile computing domains and in the characterization of measures of usability and usable systems.

S. Camille Peres

S. Camille Peres is an assistant professor at Texas A&M University and co-director of the Ergonomics Center. Her research is generally focused on discovering and investigating the biomechanical risks associated with software design and input methodologies and effective design methods for auditory displays.