The theoretical foundation and validity of a component-based usability questionnaire

Abstract

Although software engineers extensively use a component-based software engineering (CBSE) approach, existing usability questionnaires only support a holistic evaluation approach, which focuses on the usability of the system as a whole. Therefore, this paper discusses a component-specific questionnaire for measuring the perceived ease-of-use of individual interaction components. A theoretical framework is presented for this compositional evaluation approach, which builds on Taylor's layered protocol theory. The application and validity of the component-specific measure is evaluated by re-examining the results of four experiments. Here, participants were asked to use the questionnaire to evaluate a total of nine interaction components used in a mobile phone, a room thermostat, a web-enabled TV set and a calculator. The applicability of the questionnaire is discussed in the setting of a new usability study of an MP3 player. The findings suggest that at least part of the perceived usability of a product can be evaluated on a component-based level.

Keywords:

• usability evaluation
• questionnaire
• component-based usability evaluation
• technology acceptance model (TAM)

Acknowledgements

We thank the anonymous reviewers, Kate Hone, Audrey Bink and Nayna Patel for their comments and advice that helped us to improve the paper.

Notes

1. The introduction text, task instruction, the MP3 player, music files, questionnaires, and the results of the test can be found at http://mmi.tudelft.nl/∼willem-paul/mp3player/Intro.htm

2. One undergraduate student also worked for the university.

3. If the MP3 data is incorporated into the norm data set, the mean of sample set A becomes 4.58 (SD = 0.77), and 5.94 (SD = 0.57) for sample set B. The 2.13 mean rating of the volume control however is an extreme outlier (<Q₁ −3 × IQR), which would drive up the break-even point to 5.36. Ignoring this extreme outlier, results in a mean for sample set A of 4.71 (SD = 0.52) and break-even point of 5.29 again.