Effects of Importance and Detectability of Usability Problems on Sample Size Requirements

Abstract

Existing models of usability evaluation estimate sample size requirements based on goal discovery rate and detectability of potential problems without consideration of the importance of the problem and an index of usability, which allows the comparison of the effectiveness of usability of information technology products and services across different domains. This article presents a methodology which accounts for the aforementioned limitations in usability evaluation. A case study is presented, which illustrates the utility of the proposed methodology and indicates that the current methods underestimate the number of participants in usability evaluation studies.