Measuring Perceived Usability: The SUS, UMUX-LITE, and AltUsability

Abstract

The purpose of this research was to investigate various measurements of perceived usability, in particular, to assess (a) whether a regression formula developed previously to bring Usability Metric for User Experience LITE (UMUX-LITE) scores into correspondence with System Usability Scale (SUS) scores would continue to do so accurately with an independent set of data; (b) whether additional items covering concepts such as findability, reliability, responsiveness, perceived use by others, effectiveness, and visual appeal would be redundant with the construct of perceived usability or would align with other potential constructs; and (c) the dimensionality of the SUS as a function of self-reported frequency of use and expertise. Given the broad use of and emerging interpretative norms for the SUS, it was encouraging that the regression equation for the UMUX-LITE worked well with this independent set of data, although there is still a need to investigate its efficacy with a broader set of products and methods. Results from a series of principal components analyses indicated that most of the additional concepts, such as findability, familiarity, efficiency, control, and visual appeal covered the same statistical ground as the other more standard metrics for perceived usability. Two of the other items (Reliable and Responsive) made up a reliable construct named System Quality. None of the structural analyses of the SUS as a function of frequency of use or self-reported expertise produced the expected components, indicating the need for additional research in this area and a need to be cautious when using the Usable and Learnable components described in previous research.

ACKNOWLEDGEMENTS

Some portions of this article originally appeared in the Proceedings of HCII 2015, Investigating the Correspondence Between UMUX-LITE and SUS Scores.

Additional information

Notes on contributors

James R. Lewis

James R. Lewis is a Senior Human Factors Engineer (at IBM since 1981), currently focusing on the design/evaluation of speech applications. He has published influential papers in the areas of usability testing and measurement. His books include Practical Speech User Interface Design and (with Jeff Sauro) Quantifying the User Experience.

Brian S. Utesch

Brian S. Utesch is a Senior Technical Staff Member and User Research Practice Lead in the IBM Enterprise Social Solutions group. Brian and his team are responsible for understanding the voice of the customer across the social solutions product portfolio.

Deborah E. Maher

Deborah E. Maher is a User Research practitioner in the IBM Enterprise Social Solutions group. She works closely with customers to capture behaviors, expectations and overall ease of use feedback within products such as IBM Connections. (In her spare time she enjoys running around after her two young children.)