Towards Precedence that Justifies the Knowledge Claims of Design Methods

Abstract

This paper analyses the relation between the precedence of a design method and the justification of the method. A design method is assumed to advance two knowledge claims concerning its domain of application: that it is an effective means for designing and that it is an efficient means for doing so. It is argued that precedence of a method can justify these knowledge claims under two conditions. The first is that precedence, in addition to descriptions of successful design projects realized with the method, also comprises: (1) negative precedence of unsuccessful design projects; and (2) comparative precedence of design projects carried out with other methods rival to the method concerned. The second condition is that the two knowledge claims of a method can be justified by precedence about only exemplar design tasks that represent the full application domain of the method.

Keywords:

• precedence
• design methods
• justification

1. Introduction

Successful design projects play in design methodology at least three roles: first they are analysed for deriving new methods to design with (or tools, or approaches, etc.), second they are used for illustrating how design methods work, and third they are presented for justifying the methods. These roles are visible in, for instance, the literature presenting the newer design methods. Successful design projects are analysed and then used for abstracting the methods from (e.g. Verganti, 2009), or methods are described in parallel with giving explanatory successful cases (e.g. Brown, 2009; Dorst, 2015; Hekkert and Van Dijk, 2011; Plattner et al, 2009). And in this literature the success cases are presented for giving the promise: ‘adopt this method, and similar design successes are in your reach’. The first two roles of successful design projects make research-methodological sense: descriptions of design projects form a central starting point in design research; and it is explanatorily beneficial to illustrate methods with examples. The third role is also useful for displaying the successes design methods can bring. Yet this role is research-methodologically problematic since it can be argued that success cases hardly justify methods.

This paper considers the role of design projects in justifying design methods. These methods are taken as advancing two knowledge claims, namely that they are effective means for doing design projects in their domain of application, and that they are efficient means for doing so. Design projects successfully carried out with design methods are, in design research, taken as the precedence of these methods, and the aim of this paper is to determine how this precedence can be related to the justification of the two knowledge claims of methods. The upshot is that for establishing this relation in a research-methodologically sound way, one should firstly take precedence in a broader sense than merely successful design projects, and secondly, limit justification of methods to their performance on exemplar design tasks representative for the methods’ application domains.

The analysis is general and intended to hold for all design methods regardless of the specific application domains they are meant for. A drawback of this generality is the abstractness of the analysis.

In Section 2 it is argued that standard precedence cannot justify design methods. In Section 3 the notion of precedence is expanded to include both negative and comparative precedence. In Section 4 it is shown that precedence may justify methods when this justification is based on their performances on exemplar design tasks. Section 5 gives research-methodological backgrounds. And Section 6 discusses the presentation of design methods in this approach.

2. Effectiveness and efficiency

For exploring how precedence can justify the effectiveness and efficiency of a method, it is useful to model the knowledge represented by these two claims. In keeping the analysis general, some informal notation is introduced.

Let D refer to a designer, M to a design method and N_M to methods that are taken as rivals of M (boldfaced symbols refer to sets, so N_M may refer to a number of rival methods N, N’, N’’, …). Let E_M refer to the design expertise a designer is supposed to have for applying method M, and let R_M refer to the maximum resources the designer is supposed to use for carrying out a project with M. Let, finally, T_M refer to the application domain of design tasks for which the method M is meant.

It is here assumed that proponents of a design method M define N_M, E_M, R_M, and T_M. If, for instance, M is a method for innovation by design thinking, the rivals N_M could be other design-thinking approaches. The expertise E_M of a designer could be the type of education the designer has had and the number of years of his or her professional experience. The resources R_M for a project may be the time and money that may be spent on a design project, and the tasks T_M may be designing a specific type of products, say, energy infrastructures or computer games.

The knowledge claims of effectiveness and efficiency can then be modelled as:

Effectiveness of design method M:

A designer with expertise E_M can resolve a design task in T_M with design method M with a specific success rate S_M, using resources R_M or less.

Efficiency of design method M:

If a designer with expertise E_M resolves a design task in T_M with a design method N in N_M rival to M, then the designer uses regularly more resources than R_M.

The success rate S_M part of the formulation of effectiveness captures to what extent a method M allows for failures. This rate may be high, meaning that designers regularly resolve tasks in T_M with M, or it can be low, as when the method gives merely solutions with a specific probability (the analysis remains qualitative, so talk about quantitative probabilities is avoided). In the formulation of efficiency it is assumed that a method M can be called more efficient than a rival N if applying N regularly requires more resources than M. The possibility that a method M is only sometimes more efficient than N is not included, for that possibility implies that using method M regularly requires more resources than method N and that N is thus more efficient than M.

The formulation of effectiveness does not rule out that designers with less or different expertise than E_M can also (occasionally) resolve a task in T_M with the method M, or that designers with more expertise than E_M may resolve tasks in T_M with M with higher success rates.

Precedence consisting of successful design projects that are carried out with a method M gives inductive support for the effectiveness of M, but hardly justifies this claim. It establishes that there exist some tasks in T_M for which the method M is effective, but does not give much information about the other tasks in T_M. And precedence leaves open whether M is a method with a high or low success rate S_M; it merely shows that there were occasions at which applying the method M led to successful projects, which still may mean that M does not often lead to success.

Precedence also does not justify the efficiency of a method M; evidence that there were occasions at which M led to successful design projects does not rule out that rival methods N in N_M can do so with resources less than R_M. Only in the special case that T_M contains tasks no other method than M can address, then precedence of M lends support to the efficiency of M relative to other methods (an example would be the frame creation design method of Dorst [2015] that is presented as being able to resolve ‘networked problems’ other approaches cannot deal with).

3. Precedence

For preserving the relation between the precedence and the justification of a design method M, one can consider including as precedence projects that failed when carried out with M, as well as projects in the domain T_M of the method M that were carried out with other methods N in N_M that are rivals to M. Let positive precedence be the usual type of precedence consisting of design projects successfully carried out with M. Two new types of precedence can then be negative precedence being projects that failed with M, and comparative precedence consisting of projects that are carried out with methods N rival to M:

Positive precedence of design method M:

•	Designer D’ with expertise E’ equal to EM resolved design task T’ in TM with design method M, using resources R’ equal to or less than RM.

Negative precedence of design method M:

•	Designer D’’ with expertise E’’ equal to EM did not manage to resolve design task T’’ in TM with design method M, using resources R’’ equal to or less than RM.

Comparative precedence of design method M (two subtypes):

•	Designer D’’’ with expertise E’’’ equal to EM did not manage to resolve design task T’’’ in TM with a design method N in NM rival to M, using resources R’’’ equal to or less than RM.
•	Designer D’’’’ with expertise E’’’’ equal to EM resolved design task T’’’’ in TM with a design method N in NM rival to M, using resources R’’’’ equal to or less than RM.

The first subtype of comparative precedence includes projects that did not lead to results at all when carried with a rival method N.

Negative precedence supports conclusions about the success rate S_M of a design method M, since if such precedence exists, one has evidence that this success rate S_M cannot be high. Negative precedence may moreover reveal ways in which a method M can be improved, by suggesting what additional expertise or resources designers should have for applying the method. Hence, it may point to ways to reformulate the method M for arriving at one that does have a high(er) success rate.

Comparative precedence gives information about how a design method M fares relative to its rivals N_M. It supports the efficiency of M if designers using those rival methods, regularly need more resources than R_M when carrying out design tasks in the application domain T_M of M. Reversely, if those rival methods do regularly lead to successful outcomes using less recourses than R_M, one has reason to limit the domain T_M at which method M is claimed to be efficient.

With negative and comparative precedence included, the relationship between precedence and justification is still not established. For having justification of the effectiveness of a design method M one should minimally have that its positive and negative precedence in some way exhausts the whole application domain T_M of the method M. And even with abundances of positive precedence supporting that the success rate S_M is high, subsequent experiences may generate large amounts of negative precedence suggesting a lower rate. The problem one encounters here is that of induction: logically and practically a series of predominantly successful design projects may still be succeeded by a series of mostly unsuccessful projects, as tossing ‘heads’ five times with a coin may be succeeded with tossing ‘tails’ four times

The justification of the efficiency of a method M leads to similar problems, even when efficiency is only claimed relative to one rival method N. Again the comparative evidence should exhaust in some way the application domain T_M of the method M, and again the problem of induction turns up.

Adding the two new types of precedence does have the advantage of relating precedence more broadly to the knowledge claims inherent to design methods: negative precedence connects with the claim of effectiveness, specifically to the success rate S_M, and comparative precedence gives a connection with the claim of efficiency. Still precedence of a method in this broader sense does not fully justify the method.

4. Justification

The observed gap between the support that precedence gives to design methods and what is needed for their justification, is in part due to the assumption that justification of design methods means full verification. By this assumption justification means evaluating effectiveness and efficiency of a method M for all the design tasks in its application domain T_M. Hence, for bridging the gap one may try relaxing this assumption. This can be done on the basis of recent design research on method justification, called the validation of design methods (e.g. Blessing and Chakrabarti, 2009; Frey and Dym, 2006; Frey and Li, 2006; Seepersad et al, 2006). This gives at least four options for justifying a design method M through considering only some design tasks in its application domain T_M.

Three options can be found in Frey and Dym (2006) who explore possibilities to validate design methods similar to how medical treatments are evaluated. First, using the analogy to medical in vitro experiments on animal models of medical disorders, design methods could be validated in controlled lab circumstances. In design research this would involve focusing on design tasks for which the possible design actions and their results can be modelled upfront, say by means of computer simulations. The effects of applying a design method M to one of these tasks can then be monitored and assessed in an unambiguous way (an example of such a model design task is given in Frey and Li, 2006). A second option draws on the analogy with clinical trials and involves defining design tasks that can act as benchmarks for the validation of methods. Designers can apply different methods M, M’, … to these benchmark tasks, and the results of the different projects can then be compared. The third option Frey and Dym discuss is doing field research on design methods, now in analogy to field research on the effects of medical treatments. This option entails determining the overall success of firms that have adopted a specific design method and comparing this success between firms using different methods M, M’, … . This third option means that only those design tasks are considered that firms are actually carrying out.

A fourth option can be found in Seepersad et al (2006). They propose to validate the effectiveness of design methods by means of four steps making up four quadrants of what they call the validation square. First, the structural validity of a design method M is to be established by showing that the method is logically rigorous, internally consistent, mathematically correct, and applicable to its intended domain T_M. Second, example design tasks should be defined for verifying its performance, as well as an argument be given why these example tasks are appropriate for, e.g. representing the full application domain T_M. Third, it should be demonstrated that the method gives successful results when applied to the example problems and that these results are due to using the method. Fourth, an argument should be given that the usefulness of the method for the example tasks may be generalized to the full application domain T_M.

There are differences between these four options. The three that Frey and Dym are providing assess and compare design methods and thus validate effectiveness and efficiency. The fourth by Seepersad et al focuses on single methods and therefore seems to be about effectiveness only. What the options share is that validation of a design method M does not mean giving evidence that the method M successfully resolves all the design tasks in its application domain T_M; the evidence required is that designers can do so for a few design tasks representative to this domain T_M, ranging from lab design tasks to example design tasks.

Let us call these design tasks that are representative to the application domain T_M of a design method M the exemplar design tasks of the method M, and let TX_M be the set containing these exemplars. Let us adopt the described validation approach followed with the four options, and take justification of a design method M as showing that M is effective and efficient on only its exemplar design tasks. Positive and negative precedence about the exemplars of the method M can then sufficiently cover the exemplar domain TX_M, and hence justify that M is effective with a certain success rate S_M. Comparative precedence about the exemplars can similarly justify efficiency of the method M. A straightforward relation between precedence of a method M on its exemplar design tasks and the justification of the method is then achieved.

5. Research methodology

The approach to justifying a design method M by showing that M is effective and efficient for its exemplar design tasks TX_M can research-methodologically be understood as a move from justification by verification of the design method M’s knowledge claims to justification by falsification (Vermaas, 2014). Verification indeed means showing for all tasks in its domain T_M that the method M is effective with a certain success rate S_M and more efficient than its rivals methods in N_M. The sketched approach limits the (initial) focus on the full application domain T_M of the method M to a focus on the smaller domain TX_M of M’s exemplars. Hence, the exemplar domain TX_M of the method M declares what design tasks can be used, in the first instance, for testing M, also meaning that the method can already be rejected if effectiveness and efficiency is proved not to hold for the exemplars. This possibility of a quick rejection of a design method is similar to falsification in the Popperian sense, although in falsification it is also the evaluator who may choose the design tasks relative to which he or she wants to put the method M to the test.

The approach to, in addition to effectiveness, considers also the justification of the claim of efficiency of a design method M can research-methodologically be understood as a second move from justification by Popperian falsification to justification by (sophisticated) falsification as described by Lakatos (1978). In falsification as associated with Popper (1968), a scientific theory, or design method in our case, is put to the test with the aim of either corroborating or rejecting it. If testing the theory, or method, is considered in isolation, and if the outcome is negative then the theory or model has to be thrown away. In philosophy of science Lakatos argued that such rejections in isolation do typically not occur in science. Scientists do not take single counterevidence as a reason to drop a theory; that theory may be, for instance, the only theory scientists have, and provides at least some means to understand and predict phenomena. In science such theories are rather kept, and the counterevidence is set aside as anomalies to be dealt with later (or not at all). Yet, if rival theories are available and scientists have the opportunity to switch to an alternative second theory when the first is falsified, then theories are indeed rejected or, better put, replaced by their rivals. By including efficiency in the justification of design methods, rivals are explicitly introduced in design research. Hence, when effectiveness and efficiency of a method M is falsified for a few (exemplar) design tasks, design researchers are in the position to actually reject the method M and replace it with one of its rival methods, N in N_M.

This prospect may give a new impetus to design research (Vermaas, 2014). In an analogy to Lakatos’ scientific research programmes consisting of series of competing scientific theories that in time will replace each other, one can imagine the emergence of design research programmes of series of competing design methods that will, in time, replace one other. Engineering design methods (e.g. Gero, 1990; Pahl et al, 2007; Suh, 1990) may form such a design research programme, and so may design-thinking methods (e.g. Brown, 2009; Plattner et al, 2009; Verganti, 2009).

6. Presenting design methods

In this paper the roles of precedence in presentations of design methods have been discussed, focusing on the role of justifying the methods concerned. For letting precedence play this latter role in a research-methodologically sound way, two new types of precedence were defined – negative and comparative precedence – in addition to the standard positive precedence. If justification is understood as testing that a method is effective and efficient with respect to exemplar design tasks that represent the method’s full application domain, precedence of a method indeed may justify the method.

When adopting this approach, a presentation of a new or existing design method M should include the following: First, the design tasks making up the application domain T_M of the method M should be defined, and the exemplar tasks TX_M representative of this application domain be given. Second, it should be stated what expertise E_M designers are supposed to have to apply the method, and with what rate S_M and with what resources R_M such designers can successfully realize a task in the application domain T_M. Third, the rival design methods N_M should be given relative to which the design method M is supposed to be an improvement in terms of efficiency. Finally, an effort should be made to present positive, negative, and comparative precedence of the design method M consisting of design projects in which the method M and its rivals N_M are applied to the exemplar design tasks TX_M of M. This precedence can be taken as justifying the design method M, and the rival methods N_M define the design research programme M is part of.

This list of deliverables is a mouthful. Yet it would put the presentation of design methods on a firmer footing. The precedence of design methods would justify their knowledge claims. And this precedence would allow conclusions to be drawn about design methods making up a design research programme. For instance, it can then be argued that a new design method M adds the possibility of taking up new design tasks, or of carrying out existing design tasks more efficiently than established design methods N in a design research programme. And it can be argued that a new design method M may replace an established method N by being a more efficient means for the design tasks that belong to the application domain T_N of the established method N. Or, worse, research may yield that the new design method M is not a valuable addition to the research programme at all. Specifically, this comparison of design methods is not yet a regular part of design research; using precedence for the justification and comparison of design methods would advance the field considerably.

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Notes on contributors

Pieter E. Vermaas

Pieter E. Vermaas is Associate Professor at the Philosophy Department of the Delft University of Technology. His research on design focuses on the structure of design methods, success criteria for design steps and tools, research methodology for design research, and validation of design methods and tools. The aim of this research is to arrive at more scientifically conducted design research that takes into account, and is based on, the actual practices of design and design research. Pieter Vermaas co-edits the Design Science journal and is editor-in-chief of the book series Philosophy of Engineering and Technology and of Design Research Foundations.