Studying Usability In Sitro: Simulating Real World Phenomena in Controlled Environments

Abstract

Increased complexity of organizations and emerging technologies poses new and difficult challenges for the evaluation of software systems. Several years of research have proven that usability evaluations are invaluable tools for ensuring the quality of software technologies, but the increased complexity of technology requires new ways of understanding and evaluating the quality of software systems. This article explores limitations, challenges, and opportunities for studying mobile technologies “in use, in situ;” in laboratories (in vitro); and in controlled high-fidelity simulations of the real world. The latter condition is called in sitro. This report comes from 2 different case studies of evaluating the usability of mobile systems within these 3 different conditions. Results show that it is possible to recreate and simulate significant elements of intended future use situations in laboratory settings and thereby increase the level of realism and maintain a high level of control. In fact, the in sitro condition was able to identify most of the same usability problems as found in the other conditions. However, the in situ evaluation proved to provide a level of realism that is difficult to achieve in laboratory environments.