Towards the experimental study of usability

Abstract

Usability is presented as a concept which can limit the degree to which a user can realize the potential utility of a computer system. A field study is presented to illustrate the manner in which usability problems inhibit usage. The study examined a banking system which provided staff with 36 ways of extracting information from a customer's account. The usage log shows that four ‘codes’ accounted for 75% of usage and many codes’, although designed specifically for known banking tasks, were virtually unused. An investigation was undertaken in 15 branches to identify what happened when staff were confronted by tasks for which unused facilities had been designed. The results showed that staff were able to use a small set of well-known facilities for most purposes, albeit inefficiently and sometimes ineffectively. The strategy adopted was to avoid searching unknown facilities except as a last resort.

From this and other field studies a framework is presented to summarize the variables affecting the usability of a system. It portrays the user making a series of implicit cost-benefit assessments as he undertakes tasks, attempting to minimize search effort ‘costs’ rather than trying to optimise ‘benefits’. The paper ends by examining the methodological implications of this framework. It is suggested that many of the variables which influence usability are excluded from normal experimental paradigms with the result that usability issues are often not evident in the results of experimental studies. It is advocated that more attention be paid to realistic simulation studies and to field experiments in order to submit the concept of usability to thorough scientific scrutiny.