Must evaluation methods be about usability? Devising and assessing the utility inspection method

Abstract

Whereas research in usability evaluation abounds, few evaluation approaches focus on utility. We present the utility inspection method (UIM), which prompts evaluators about the utility of the system they evaluate. The UIM asks whether a system uses global platforms, provides support infrastructure, is robust, gives access to rich content, allows customisation, offers symbolic value and supports companionship among users and between users and developers. We compare 47 participants’ use of UIM and heuristic evaluation (HE). The UIM helps identify more than three times as many problems as HE about the context of activities; HE helps identify 2.5 times as many problems as UIM about the interface. Usability experts consider the problems found with UIM more severe and more complex to solve compared to those found with HE. We argue that UIM complements existing usability evaluation methods and discuss future research on utility inspection.

Keywords:

• utility
• usability
• usability inspection
• usefulness
• inspection methods
• heuristic evaluation

View correction statement:

Erratum

Acknowledgements

We thank the student evaluators for participating in this study and the usability experts for their effort in making this work possible.