A foundational observation method for studying design situations

Abstract

Observational studies of designers play an important role in engineering design research, yet there is currently no accepted standard approach for comparing, combining, or contrasting studies. Consequentially, reuse, reanalysis, replication, and aggregation of data are limited and the potential impact of individual studies is severely constrained. This paper begins to address this issue by introducing and developing a foundational method for observational design research to improve replicability, reuse, and overall comparability of empirical studies. A three-step foundational method is proposed that covers capture, coding, and analysis. The capture step defines overall and situational context as well as multiple capture streams, generating a broad data-set that can be examined from multiple perspectives. The coding step employs a multi-level approach that seeks to minimise workload while describing both detailed and high-level information. The analysis step builds on the multi-level approach to provide for a flexible yet standardised examination of the data-set. The overall approach is introduced theoretically and illustrated using a comparison of an industrial study and an experimental study. Finally, it is argued that the proposed method promotes rigour, reliability, and standardisation; and could provide one means for improving comparison and aggregation, ultimately increasing impact in academia and practice.

Keywords:

• descriptive models of the design process
• empirical study
• ethnography

Acknowledgements

The authors gratefully express their thanks for the advice and support of all concerned. We would also like to express our thanks to the reviewers for their insightful input.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The work reported in this paper has been undertaken as part of the EPSRC Innovative Manufacturing Research Centre at the University of Bath [grant number GR/R67507/0] and has been supported by a number of industrial companies.