A Design Thinking Rationality Framework: Framing and Solving Design Problems in Early Concept Generation

Abstract

The concept of “Design Thinking” opens up debate regarding the prevalent human–computer interaction design practice. This article focuses specifically on the cognitive processes of designers during their early design activities. Two groups of designers—experts and novices—were asked to develop a fictitious vacuum cleaner. We then examined the different ways in which these groups manage their design thinking processes and how the groups choose design concepts. The empirical study revealed that expert designers are effective at framing design problems. They make quick decisions (through the use of the affect heuristic) but are more wedded to their own previously developed design concepts, which they do not change in subsequent design stages. In contrast, novice designers are less skilled in framing new design problems but better able to renounce their initial design concepts. These diverse design thinking approaches are linked to potential problems. We then discuss how to address these concerns in conjunction with empathy for the artifact (i.e., artifact empathy via the mediated self) or user (i.e., user empathy via the simulated self), problem framing with second-order semantic connotations, and irrationality when analyzing design solutions. Finally, we propose a design thinking rationality framework that can establish a designer's view of design activities and thereby assist designers educated in both creative and rational design decisions.

1. INTRODUCTION

Nothing can be created out of nothing (Lucretius, 94–55 BC: De Rerum Natura)

Design is challenging. To ensure better design quality, the role and nature of both design practice and the design process have been increasingly emphasized in human–computer interaction (HCI) studies (e.g., Atwood, McCain, & Williams, 2002; Bartneck, 2007; Fallman, 2003, 2007; Löwgren & Stolterman, 2004; Zimmerman, Forlizzi, & Evenson, 2010). These studies agree that design cannot be analyzed entirely from a scientific perspective; however, it appears that a prominent HCI perception determines that design is, at best, user-oriented research rather than design-oriented research (e.g., Fallman, 2007).

The user-oriented design paradigm has been openly criticized by prominent design educators (e.g., David Kelley of the “d.school” at Stanford University) and design innovators (e.g., Brown, 2009; Cross, 2001, 2011; Martin, 2009), who believe that creative (or, at the very least, designer inspired) design that attracts more users eventually contributes to the HCI ideals (e.g., being pleasant to use). In this regard, the user-oriented attitude in design does not seem to fully contribute to a proper or elaborate understanding of what design itself is, nor does it provide insight into the active role of the designer in the design activity. However, both paradigms appear to share a common belief: Both ask that the designer satisfy the user by providing product/service utility, sensation, aspiration levels and experiences, but in a rather different tone.

The user-oriented design approach does not disregard the designer's creativity. It envisions design work as divided into three distinct stages. First, the designer analyses a design scope through user studies (i.e., problem framing), then generates and synthesizes a solution using their creative ideas (i.e., problem solving), and finally evaluates the outcome with the participation of the user (i.e., evaluation; e.g. Alexander, 1964; Broadbent, 1973). However, this structural design approach has led to well-documented failures (e.g., Lawson, 1997), which reveal that designers often use less structured design methods such as brainstorming, bodystorming, checklists, and agile design methods (e.g., the double diamond design process model; Design Council, 2005). As a compromise to the distinctive linear design process, the designer tends to employ a more iterative design strategy in their analysis, synthesis, and evaluation. Therefore, he or she can choose to synthesise design solutions, thereby creating plausible new design scopes. However, precisely how this “up-and-down” design process works for the designer is yet to be fully understood, and the question remains as to which elements trigger this cognitive process, particularly during the early concept generation phase (Cross, 2001; Guindon, 1990; Kim, 2011; Suwa, Purcell, & Gero, 1998).

Further discussion regarding the designer's thinking process is thus needed. Psychologists (e.g., Cathcart & Gumpert, 1986) assert that humans use mental simulations to learn or create new thoughts—a process linked to the designer's mediated self. This mediated self can perceive or reason various qualities of the artifact or the world, such as that the artifact might be interesting, boring, pleasant, easy to use, and so forth. However, the theory of mind (e.g., Premack & Woodruff, 1978; Shamay-Tsoory, Aharon-Peretz, & Perry, 2009) suggests that our mental simulations not only serve to understand the artifact or world but also the worlds of others, which provides humans with many sophisticated traits including empathy. A designer's simulated self, which tries to account for others' attitudes and behaviors, can then act as a proxy for the user. In this regard, C. R. Rogers's (1989) early definition of empathy is worth noting: “It (empathy) is an accurate, empathetic understanding of the user's world as seen from you. To sense the user's private world as if it were your own, but without losing the ‘as if’ quality—this is empathy” (p. 226). In fact, that which we normally refer to as the designer's mediated self is actually a representation that allows for much deeper and wider design thinking. However, in some situations, the designer's mediated self needs to be weaker in order to become more sensitized to the user; the simulated self (Markman, Klein, & Suhr, 2009) should understand how the user might perceive the artifact or world that the designer is creating.

To illustrate this, one might imagine a designer creates a mug for the elderly. The internal process involved in design thinking entails a part of the designer's brain (i.e., the mediated self) predicting how the mug will work for the user. If there is no available user testing or evaluation, some proxy for the elderly (i.e., an empathetic self to the elderly) should serve the designer in mimicking an elderly person's behavior. In this case, every time the designer executes design thinking, they must instruct their “simulated self” to behave like the elderly person and clarify the predicted outcomes. This process illustrates that the two versions of self (the simulated self and mediated self) would create a setup for the design thinking process. Of interest, the simulated self is mainly limited to the designer's own knowledge or understanding of the user, based on experience. It might be assumed that someone with more experience designing for the elderly would quickly form such a simulated self. This would enable them to objectify their designed artifact in a more rational way, as it would distance them from the artifact and force them to jettison certain attitudes and behaviors from the mediated self.

The conception of the two versions of self in the designer's mind has certain advantages. First, it can explain why many designers still prefer less formal design methods. Second, it can clarify why designers remain doubtful of users, though advocates of User-Centered Design have long publicized its significant benefits. Finally, this viewpoint determines when the designer's cognitive process is focused on the designer's mediated self (i.e., how to deliver their design intention through the artifact), and when the simulated self instructs the user's cognitive process of the artifact (i.e., how easily the user can understand the artifact). Given that the early stages of the design activity are cognitively intense, we should first conduct a thorough study to better understand designers' internal cognitive processes. These processes include perception, problem solving, reasoning, and thinking about the design (e.g., Coley, Houseman, & Roy, 2007; Eastman, 2001; Goldschmidt, 1994; Hallihan, Cheong, & Shu, 2012; Suwa & Tversky, 1997). This research is the central premise of the study.

Our first research aim is to examine how designers set up a design problem and discover solutions during the early design process. Of course, these design solutions do not necessarily optimize the design outcome, and if this is the case, designers employ a particular heuristic (Yilmaz & Seifert, 2011). The evaluation of this heuristic is the second aim of this study. In this way, one can see how the two different versions of self (i.e., simulated self and mediated self) operate throughout the design thinking process. Our primary research questions in this study are (a) what form of design thinking does the designer employ, and (b) how do particular heuristics affect the designer's design decisions? We can use the answers to these questions in building a framework that can stimulate effective design thinking and regulate ineffective design solutions.

This article comprises five sections. Section 2 provides a theoretical background and literature review related to the designer's early concept generation and design decision making. This assists in framing our research aims and objectives. Section 3 describes the empirical study conducted to test and address our core research questions. Section 4 interprets the study's quantitative and qualitative results. Finally, Section 5 suggests a design thinking rationality framework, which includes several broad design principles arising from this study, and discusses its potential contributions to both HCI and design researchers.

2. HOW DO WE DESIGN?

Simon's (1969) thesis—The Sciences of the Artificial—claims that design activities are a type of problem solving and therefore applies a cognitive analysis to design “problem-solving” tasks (Eastman, 2001; Gero, 2006; Goldschmidt, 1994; Suwa & Tversky, 1997). Although this engineering perspective has dominated HCI studies, many HCI practitioners now question the appropriateness of such a “problem-solving” paradigm for today's designers. For instance, Visser (2006) argued that (a) designers often encounter solutions opportunistically and locally, rather than as a result of a systematic search for optimal solutions; (b) during these encounters, designers' cognition constantly develops, and they evaluate several occurrences of the potential design concepts using their own mental categories of the design elements; and (c) a combination of opportunistic, local, “top-down,” and “bottom-up” solutions occurs. These are just some of the forms in which opportunism can reveal itself in design.

Indeed, the designer employs various levels of design elements, through which he or she is able to create design solutions for a “good form” of one or several concepts. It is noteworthy that these design solutions do not arise solely from user studies but rather distance the designer from the user, creating a sense of emotional amplification such as astonishment and surprise for the user. This “I-for-myself” design philosophy also exists within inspirational design tradition, whereby the designer creates what they like without any further disruptions to their design intention. Conversely, “user-centered design,” “participatory design,” and “co-design” advocate the way in which others (users) incorporate my (designer's) perceptions of them (users) into my (designer's) design outcomes. Indeed, a frequent shift between the two different design identities (I-for-myself vs. I-for-the-others) might suggest a key factor that accounts for the designer's cognition in the design thinking process. In practice, some designers intentionally distance themselves from the user-oriented approach. Instead of providing a self-evident, familiar, and obvious design quality for others (users), they instead create a sense of astonishment and curiosity about the design values. This is particularly true in the early stages of the design process (Brown, 2008). As a result, the designer is able to adopt a new mind set in order to develop a different perception of the design (Cross, 2001; Dorst, 2011; Lawson, 2005; Lawson & Dorst, 2009; Nelson & Stolterman, 2003).

The emergence of contemporary HCI and design studies has grown from the need to formally address the increasing complexity of design itself (e.g., Carroll, 2003; Rogers, 2004). An initial approach to the issue of design and design-oriented HCI is thus to emphasize that design is a matter of creation rather than an object of evaluation (Fallman, 2003). This suggests that the design process involves a certain mystical and aesthetic element and is by no means a fully rational and explicable process, as some scientific accounts would have us believe. This approach emphasizes the notion that the designer should be insightful and able to generate creative design qualities but also may be unable to explain, or uninterested in explaining, how these qualities came about. Hence, advocates of designer-oriented HCI claim that a design should be judged according to the designer's personal values, taste, quality, and aesthetics—an evaluation that is often labeled as a “romantic” or “artistic” account of design (Ball & Ormerod, 1995; Guindon, 1990; Visser, 1990). Oscar Wilde's classic philosophy of art (Ellmann, 1969) strongly purports this stance, asserting the centrality of aesthetic imagination in design.

In contrast to the “artistic” account of design, the “scientific” account envisioned by Simon (1969) has been conceived as a scientific or engineering endeavor borrowing many methodologies and terminologies from other science disciplines and drawing on a rational philosophical base (Fallman, 2003). According to this view, design is fundamentally a process of problem solving, rooted in the creation of new, useful design elements. These problem-solving methods are commonly strung together to form the cycle of user research, ideation, prototyping, iteration, and the refinement process. However, this scientific design approach inadvertently deemphasizes the active role of the designer, particularly the insightful or creative aspect of the designer's contribution. At best, it considers the designer as a rational thinker, striving toward a disembodied design process built on structured methods and externalized guidelines. That is, the scientific conception of design envisions design work as a problem-solving task, a solution synthesis, and outcome evaluations. A significant drawback of the scientific approach thus lies in its major axiom that “the designer is rational.” This view has long been criticized by many decision theorists (e.g., Tversky & Kahneman, 1974). Numerous case studies of actual design projects show that designers are not rational, nor would it be possible to educate them in such a predefined scientific manner (e.g., Stolterman, 2001). The following literature review provides a basis for understanding how the two competing accounts—“artistic” and “scientific”—provide different but equally important ways of conceptualizing design thinking during the generation of a design concept and while making early design decisions.

2.1. Design Thinking in Early Concept Generation

Sketching is the most common tool that designers use in the development of early design concepts as a means of creating and presenting design elements. There are two markedly different approaches to sketching, namely the “bottom-up” and “top-down” approaches (e.g., Ball & Ormerod, 1995; Cross, 2004; Guindon, 1990; Visser, 1990, 2009). The bottom-up approach consists of a designer examining various design elements, considering the degree of commonality and variability among them and then establishing a mental category to abstract the concept. For example, when creating a vacuum cleaner design (Kim, 2011), the designer first mentally searches for various designs or elements (e.g., aesthetics, the form or appearance of existing vacuum cleaners). He or she then assesses whether the design elements can be classified into the same category. This specific-to-general cognitive process helps and/or constrains the designer in deciding whether to evolve the design elements from the inferred mental category by chiefly focusing on the specific design properties that have provided useful distinctions in the past. In contrast, the top-down strategy (i.e., general-to-specific) consists of a selective search for constancies within numerous design iterations and selectively combining new design elements and existing mental categories. For example, in the 1860s, Ives McGaffey (the inventor of the first powered vacuum cleaner), realized that the nature of a “whirlwind” could be classified with the mechanical solutions able to replicate its movement. Here, the “form” factor in his vacuum cleaner was matched with a natural occurrence, and his mental operation suggested mechanical solutions to his design problem.

Design cognition resulting from the bottom-up or top-down approach is not a one-off process in the early generation of a concept (Anderson & Betz, 2001; Erickson & Kruschke, 1998; Holyoak & Nisbett, 1988). Quite often, top-down design cognition is quick to identify a higher order product value (e.g., elegance and pleasure) that facilitates its subsequent encounter with lower order design elements (e.g., color and/or forms to represent elegance or pleasure). In contrast, bottom-up design cognition first sees the lower order design elements that might facilitate their subsequent synthesis into a higher order design. In this case, vivid design elements or mental categories would strongly influence the designer's decisions when developing a prototype. As Gentner, Rattermann, and Forbus (1993) argued, if higher order design values are brought into the design thinking process first, they tend to focus on a sense of design identity rather than design decisions.

It is thus important to question how effectively the designer is able to retrieve design information. As briefly discussed earlier, designers do not employ a single strategy but generally use both bottom-up and top-down strategies when sketching inferences (Anderson & Betz, 2001; Erickson & Kruschke, 1998; Holyoak & Nisbett, 1988). Here, they employ both exemplar information (based on experience or an opportunistic encounter) and abstract information (based on what they already know or have previously inferred) in order to make appropriate sketches. In particular, highly experienced designers might “affectively” use their existing knowledge to blindly justify what they think is best (i.e., unselfconscious processes of craft design; Alexander, 1964; Jones, 1970/1992).

Some design studies discuss such expertise in design thinking as more opportunistic. Both Bassok (2003) and Visser (2006) observed that the designer opportunistically encounters new design concepts while hopping between domains in which design elements bear no surface similarity (i.e., an interdomain analogy). The benefits from the deliberate “distancing effect” for the designer can be seen in opportunistic design thinking (Dahl & Moreau, 2002; Kalogerakis, Lüthje, & Herstatt, 2010). Its primary quality allows the designer to easily shadow past design concepts or, at the very least, to avoid a fixation on previous design inferences (e.g., Cardoso & Badke-Schaub, 2011; Jansson & Smith, 1991; Kerne, Smith, Koh, Choi, & Graeber, 2008). However, how the defamiliarization with existing design elements is triggered and what this process achieves have not yet been empirically demonstrated; this presents a challenge that is important to this study.

This “opportunistic” design thinking can be mediated by the frequent use of analogies by the designer. In developing their design thinking, designers must be able to represent the differences between “the desired” and “the real.” However, for many current designers, it is difficult to establish the difference between these two states, as the problem may have been specified at a concrete level (e.g., imagine that a vacuum cleaner for Nike is debriefed in a design project). However, at the same time, the design goal will generally be specified at an abstract level (e.g., how the vacuum cleaner presents the quality and value of the Nike brand). In this case, the designer would prefer to see analogies arising from similar design content and tends to believe, incorrectly, that a similar surface structure implies a comparable deeper structure (e.g., the logo of Nike resembles a boomerang, so the shape of the vacuum cleaner must look like a boomerang as well). This might lead a designer toward poor design thinking. Of interest, those who see similarities by first looking at the surface structure tend to stick to early design decisions that they then elaborate at all costs (e.g., Gentner et al., 1993). Semiotics and poststructuralism in new artistic movements also identified this concern as a relationship between the “signifier” and the “signified.” Roland Barthes (1975) claimed that second-order signs or connotations would enrich the interpretation of the “signified,” rather than those of the first order. For example, a picture of a full, dark bottle is a signifier that relates to a specific signified: a fermented alcoholic beverage (i.e., the first-order connotation). However, those who first see similarities by looking at the deeper structure might relate it to a new signified: for example, the idea of a healthy, robust, relaxing experience (i.e., the second-order connotations). This raises interesting questions: what design thinking designers use, whether there were differences between expert and novice designers, and what the subsequent outcomes were during early concept generation (i.e., sketches).

2.2. Design Decision Making in Early Concept Generation

Whether they are rough or fine, sketches are ideal means of capturing and understanding the nature of design work (e.g., Cross, 2001, 2011; Kavakli & Gero, 2001, 2002). In this article, sketching is not regarded as a simple design tool available to all designers but rather as a means of thinking about and generating further design concepts. Sketching, then, can be seen as a useful way in which the form, appearance, and characteristics of artifacts, which are as yet intangible, may be transferred from the designer's mind to the user. Here, the designer's cognition serves to develop design concepts for the best sketches.

As demonstrated in the vacuum cleaner case, a key feature of design expertise is “problem framing,” or structuring and formulating the design problem in a creative and novel way (e.g., Cross, 2001; Schön, 1983). However, given that many design studies (e.g., Cross, 2004) maintain that a thorough improvement can result from creative searches on new design solution spaces, the number of empirical studies about the designer's cognitive process in “problem framing” are rather limited—several exceptions include the studies of Dorst (2011) and Bjorklund (2013). The difficulty here is that the designer's repertoire of design inferences inclines toward particular features of a situation, such that the designer's bias toward a certain design might blind him or her to the irrational processes that are omnipresent in his or her design practices (Gentner, Rattermann & Forbus, 1993). Within this design practice, design sources can inform designers about potential design concepts, as previously discussed; however, they are not able to exhaustively explore all potentially useful information.

Facing such infinite knowledge sources might lead the designer to experience overwhelming design complexity and information overload. In this case, their bounded rationality (Simon, 1969) would inevitably limit their design decision making. A less experienced designer might suffer from “design paralysis” when confronted with such endless opportunities. If there are too many potential design elements to be considered in these “wicked” design situations (Rittel & Webber, 1974; Schön, 1983), it becomes much more difficult for designers to systematically evaluate design solutions. It is interesting to note how designers decide on design elements according to their design objective, as well as how their existing design knowledge influences their design solutions.

In this case, a designer's mental categories of design elements are critical to eliciting the design information he or she uses to reach the desired state, including, for example, the forms, functions, semantic words, values, aesthetic harmony, and so forth. Many behavioral studies on design (e.g., Almendra & Christiaans, 2009; Hallihan et al., 2012; Kim, Ryu, & Kim, 2013) demonstrate that the choice of relevant mental categories tends to violate the logic proposed by the scientific design models, thus implying that the designer might locally adjust their own design using design heuristics (Yilmaz & Seifert, 2011). There might also be further gaps when deciding on what heuristics to use, depending on the designer's level of expertise (i.e., experts vs. novices). In this context, the research questions concerned in the present study are (a) what design thinking do designers use and (b) how particular heuristics affect their design decisions. Our research framework was then designed to portray a designer's attitude toward a particular way of “design thinking” or “decision rationality” during early concept generation (Figure 1).

Applying Bakhtin's (1984, 1986) claim to these notions, design activity itself can be viewed as a site of dialogue between artifacts and designers (I-for-myself), and between users and designers (I-for-the-others). The user-oriented design has long emphasized the dialogue between designers and users, which suggests that the designer should effectively take a third-person view (e.g., Wright & McCarthy, 2008), rather than the dialogue between artifacts and designers, which implies a designer's first-person view of an artifact. In this respect, perhaps, too much empathy for the user might lead to less inspired or less innovative design outcomes (i.e., gulf of design thinking); however, too little empathy might lead to the creation of a “so-what?” artifact (i.e., gulf of decision rationality).

FIGURE 1. The research framework in the present study.

Of course, the hypotheses following from the accounts described here may be too strong to be tested by a single empirical study. However, at the very least we hope to reveal how design thinking and decision rationality might be associated with, or separated from, the perspectives of problem framing and problem solving. This might suggest a finer design methodology to best serve design-oriented HCI.

3. AN EMPIRICAL STUDY

Section 2 made several references to design processes, which are associated with design cognition, ranging from design decision making to the creation of design elements. The empirical study begins by using a research framework (illustrated in Figure 1) to describe the key design thinking and design rationality patterns that take place during early concept generation. We examined the cognition of designers within two groups (experts vs. novices) as they retrieve design information from their memory or experience, including the way in which they generate potential designs and the design heuristics they employ.

To empirically substantiate this claim, one would ideally study design development in real design situations with a fully controlled experimental approach. There are relatively few articles in the HCI literature in which numerous product designers are randomly assigned to different design projects for comparison purposes. Practically, this is because designers are difficult to access and are generally unable to offer the time required to participate in a controlled intervention of this kind. That said, we present a study in which 16 designers (eight experts and eight novices) were brought into a design session and asked to join an imaginary design project: “Designing a Nike vacuum cleaner.” The designers were asked to think aloud about what they were thinking while designing. It is difficult to generalize from a single design exercise using this verbal protocol method, and this approximation differs from real design contexts in certain critical aspects; however, we suggest that the exercise provides some empirical support for the plausibility of the design cognition process presented in Section 2.

The verbal protocol method was chosen, as it is the only technique we are aware of that also takes a qualitative stance on the cognitive design process. Furthermore, a thematic analysis was applied to interpret the transcripts of the verbal protocols. Through thematic analysis, we identified a limited number of themes (codes) that adequately reflected the verbal data of the transcripts. Two researchers familiar with design studies independently coded the data and suggested two classes (“Retrieval of design information” and “Design choice process”) of the thematic codes regarding how the participants appropriated the design.

3.1. Participants

Sixteen volunteers were recruited: eight experts (referred to as E1–E8) and eight novice designers (referred to as N1–N8). The participants were all of French origin; 10 were male and six were female. Eight expert designers had been working with diverse design projects in an agency or freelancer for at least 4 years, averaging 8.25 years (Figure 2). All experts had postgraduate degrees in industrial design, graphic design, or transportation design. By comparison, all novice student designers were recruited from the three French design schools: Arts et Métiers ParisTech, École nationale supérieure de creation industrielle, and Université de Technologie de Compiègne; half of the novices were undergraduates, and the other half were master's or Ph.D. students. All had joined at least one design project for a product development internship at different design agencies.

3.2. Materials and Apparatus

The experiment was conducted in the participants' own studios in order to not change their natural working environment and to minimize external stimuli. The design studio was equipped with two video cameras (one for close-ups of the sketches and the other for body gestures), and any inspirational sources for design (e.g., magazines, pictures, or the Internet) were removed. This was deliberate in order to ensure that the participants were not influenced by external stimuli that might affect their design cognition. The participants were also told to generate design sketches using only traditional sketching tools (i.e., pens, paper, and an eraser). The experimental setting is shown in Figure 3.

FIGURE 2. The profiles of the participants.

FIGURE 3. Experimental setting in a participant's studio (Kim, 2011).

Prior to the main session, a pilot study with four novice designers was conducted, and the design task (an imaginative design brief: “Designing a Nike vacuum cleaner”) was identified. Full details are given in the appendix. One of the critical conditions explained to the participants was that the design outcome should not be too function oriented, but rather an inspiring design exercise, and that it should also have a reasonable level of complexity. The participants were each given 1 hr to generate a first design idea sketch (Kim, 2011).

3.3. Procedure

The empirical study comprised three stages: warm-up exercises (≤ 15 min), design generation (≤ 60 min), and semidirected interviews(≤ 15 min), which followed the design session.

First, all participating designers were introduced to the experimental procedure. They then used and tested the concurrent verbalization method by conducting a small design task—“Designing a nutcracker”—for approximately 15 min.

Second, at the design generation stage, the participants were asked to design a Nike vacuum cleaner and were then told to complete the task within 1 hr. During the design session, the participants were encouraged to express their thoughts out loud.

Finally, at the end of the experiment, we conducted a semidirected interview to compensate for the deficiencies of the concurrent verbalization method (Coley et al., 2007). Following the suggestions of Taylor and Lindlof (2002), the interviewer utilized the sketches generated by the designers as reflective cues and repeated open-ended questions, for example, “What did you think when you received the design brief?”; “Tell me more about your thinking on that point”; and “What is that?”

3.4. Coding Process and Data Analysis

The video recordings of the verbal protocols were transcribed and analyzed both qualitatively and quantitatively. Data analysis was carried out by two independent researchers who had studied design science for more than 5 years. The researchers transcribed the verbal protocols and developed coding schemes. A “bottom-up” thematic coding scheme (Merriam, 2009; Silverman, 1998) was chosen rather than developing a predetermined coding scheme prior to the coding process. The latter approach helps coders mitigate their biases and allows various statistical analyses, but the predetermined coding scheme may limit the potential differences between the novice and expert designers in our specific context. Instead, we asked the two coders to manually review all the verbal protocols, and they agreed on two distinct classes based on the human information-processing paradigm: Retrieval of design information and Design choice process.

The two coders also agreed on a detailed coding scheme that comprised nine categories, and organized them into the two aforementioned classes (Figure 4). The thematic coding scheme of Retrieval of design information (Class I) included Functional solutions, Form descriptions, Semantic descriptions, and Analogy. Design choice process (Class II) contained five subcategories: Affirmative decision of artifact, Decision to compare options, Decision to go for new design alternatives, Affective evaluation, and Decision to postpone.

The reliability of the coding scheme was calculated using the percentage of intercoder agreement. In particular, the lowest intercoder agreement was determined for the category Affirmative decision of artifact. For instance, as shown in Figure 4, the idea that “my Nike vacuum cleaner should be a backpack type for the young” was relatively difficult to code. However, our two coders still generated a high Cohen's Kappa value at 0.84, even for this category. Other Kappa values were higher than this minimum, ranging from 0.84 to 0.90.

4. RESULTS

Both expert and novice designers created nicely coated designs, some of which were rather unique. Figure 5 provides the last sheets of the sketches produced by the eight experts and novices, which show their final design outcomes. Before examining the results of verbal protocols, one might question whether the process of concurrent verbalization while designing can actually reveal the designer's mental processes. In particular, the sketching activity itself might be inhibited by the pressure of thinking aloud (Coley et al., 2007).

As Figure 6 shows, the expert designers were able to verbalize their thoughts for approximately 83% (~50 min) of the design session, and they remained silent for the remainder of the experimental session. The novice designers managed to verbalize their thoughts for approximately 70% (~42 min) of their design practice. Neither group had much difficulty with the concurrent verbalization and spent around half of their time verbalizing their thoughts while drawing sketches (65.71% for the experts, and 52.83% for the novices). However, the expert designers were significantly more likely to verbalize their thoughts than the novice designers, F(3, 28) = 10.34, p < .05. This reveals that the expert designers were more confident with such verbalization, and their cognitive burden to do this was much lower than that of the novices.

FIGURE 4. Categories produced through the bottom-up thematic approach.

FIGURE 5. Sketches by the eight experts and eight novices.

This different verbalization pattern for experts and novices can be interpreted using the working memory model (Baddeley, 2000; Baddeley, Hitch, & Allen, 2009). The working memory consists of the central executive system and three slave systems: the phonological loop, the visuo-spatial sketchpad, and the episodic buffer. Baddeley suggested that the phonological loop and the visuo-spatial sketchpad are independent buffers and handle verbal information and visuo-spatial information respectively. Our results showed that more experienced designers were capable of verbalizing while also developing sketches. Hence, the difference between the novice and expert designers under concurrent verbalization during the sketching activity arose from a different capacity of working memory depending on their level of expertise. Bilda and Gero (2007) stated that expert designers can effectively organize their cognitive resources better than novices and often use their tacit knowledge and the preexisting chunks of spatial models from long-term memory.

FIGURE 6. Modality of gathering information while engaged in design activity.

4.1. Class I: Retrieval of Design Information

Whereas our research approach was qualitative in its nature, we also attempted quantitative analysis of the design protocols collected (see Figures 7 and 8 for examples of the verbal protocol codes and sketches).

First, it is interesting to examine the total amount of coded occurrences produced by our participants. The eight experts produced 64 to 115 codes, whereas the eight novices produced 54 to 74 occurrences in total, which is somewhat lower than the experts. This may indicate that the novice designers were less familiar with the process of verbalization while sketching, which is in line with the results shown in Figure 9 (i.e., the time spent verbalizing was 83.31% for experts and 69.65% for novices). However, the number of retrieval actions from memory/previous design experience is quite similar between experts and novices (see Figure 9). The sum of the numbers of codes of each category produced by the eight experts and eight novices are compared. According to the frequencies of retrieval of design information (Class I), both novices and experts show a similar number of occurrences: 391 cases for the experts versus 321 cases for the novices.

FIGURE 7. An example of verbal protocol codes and sketches (Expert 5).

FIGURE 8. An example of verbal protocol codes and sketches (Novice 2).

As shown in Figure 9, the most frequent design information used is for functional solutions, irrespective of the designer's level of expertise (42.97% for the experts vs. 40.50% for the novices). This indicates that, to design a Nike vacuum cleaner, around half of the design retrieval time is dedicated to understanding how to make the design work like a vacuum cleaner, for example, how to improve its suction power, its versatility, and so on. For instance,

“Can I design a large module next to one person to reach the floor with his arm or just [create] a small module?” (Expert 1 [E1])

Of course, it is inevitable that the most basic functions should be met by their design, and the functional requirements are crucial to design problem solving. Of interest, the second-highest portion of cognitive time is given to the form and semantic descriptions, though there were some gaps between forms and semantics crossed by the experts and the novices (85 and 72 cases in experts vs. 90 and 40 cases in novices, respectively). A chi-square test revealed that these amounts are significantly different (p < .05). Critically, the expert designers spend more time on both forms and semantics than the novices, and this might reveal how a certain form is associated with semantic descriptions by the expert. For example, in our verbal protocol, Expert 4 utilized semantic words such as dynamic and active with dissymmetry form and tense line. Expert 3 also described a semantic word soft alongside curved shape, pastel colors, and smooth texture. This particular pattern of the experts design cognition is often described in terms of aesthetic dimensions or harmony rules of design (Bouchard et al., 2009). Mougenot, Bouchard, Aoussat, and Westerman (2008) also found a similar pattern among the professional car designers when they retrieved inspirational visual sources and annotated them. Likewise, the use of higher level (i.e., second-order) semantic features can be a particular part of their design expertise, which demands a greater cognitive load than the use of low-level attributes in design (Pasman, 2003).

FIGURE 9. Occurrences of retrieval of design information produced by the eight experts and eight novices.

In contrast, novice designers tend to develop more forms to meet their semantic descriptions, and it seems that they feel less confident in what they are sketching to reveal their semantic descriptions. For instance, at the beginning of the design session, Novice 4 (N4) suggested that he was “not quite sure, but I think the Nike logo looks very dynamic and masculine [00:03:04]. I'll play with the Nike logo to make a more dynamic and masculine vacuum cleaner.”

In actuality, he started to draw the Nike logo first. During the rest of design activity, which accounted for approximately 28 min of the session, he tried to vary the form in many different ways and considered its corresponding functional value as a vacuum cleaner at the same time. However, as Figure 3 [N4] shows, his final outcome was hardly evolved from the Nike logo, which revealed that he was not entirely sure that what he was making was equivalent to what he meant to make.

Again, the largest category of retrieval of design elements is functional solutions associated with several parts of the vacuum cleaner's mechanics, units, usage, and principal operation, including the ventilating grill, caster, hose and pipe, bag compartment, brush, and so on. For example,

I'll try a backpack type of vacuum cleaner. This model should provide enough power, but it should not be too heavy to carry on his shoulder. Also, a potable battery-powered model is still problematic, because its wrong orientation of air evacuation can cause a user to get scalded. (N7)

There were no substantial differences in the order of developing which the functional solutions, such as type of wheel or suction motor. For example, five experts and all novices began by sketching the most popular type of vacuum cleaner, a canister (except for two backpack styles [E1 & E5] and one upright style [E7]); however, 12 of 16 designers changed their initial design concepts at least once when new semantic or functional terms came to mind, such as a backpack, a stick, or a handheld cleaner, except for E6, N1, N3, and N4.

The most common semantic words the participants mentioned in sketching included sporty, dynamic, rapid, rigid, light, young, and masculine. Of course, these semantic words are highly related to both the Nike brand and the product, that is, the vacuum cleaner. However, the most frequently mentioned semantic words concerned the brand Nike. For instance, all the participants uttered the words sporty, dynamic, rapid, and rigid at least once, which all seem to be anchored toward the sport brand Nike. Furthermore, there was a substantial gap between the experts and the novices in choosing the words for Nike vacuum cleaner, as shown in Figure 10.

This clearly implies that, to some extent, the novice designers were demonstrative of finding the low-level similarity. That is, Nike seems to have been the foremost concept they used to begin developing their sketches. In contrast, a more abstract level and a variety of semantic words (e.g., intelligent [E4 & E7], energetic [E4], fluid [E1], elegant [E2], aggressive [E4 & E6], futuristic [E4 & E5], organic [E2], seductive [E8]) were used by the expert designers, thereby resulting in rather different types of form descriptions later on. Again, the semantic words uttered seemed to be highly associated with the forms they were creating in their design cognition, especially for the expert designers. For example, one of the experts (E3) said, “It is necessary to study Nike shoe design too; however, it [the vacuum cleaner] should not look like the Nike shoes, [but] rather evoke the Nike style and its impression of ‘rapidity.”’

As shown by the preceding examples, the designers demonstrated a high level of association between the semantic words and the form development for the semantic words. This implies that the relationship between the semantic words and the related forms is one of the critical qualities of design problem framing, and frequent use of analogies by our designers confirms this. Contrary to what is considered to be “good design practice,” those who see the similarity from the surface or the first-order connotation rather than the structural similarity readily stick to a content-based analogy that they can elaborate. As Figure 11 shows, for instance, a mental image of a Segway was used by N5 to present a more “rapid” and “sporty” design, which were the semantic descriptions most commonly used to describe the sport brand Nike. In contrast, when E4 elaborated his design, he was thinking not only of something “rapid” but also of the intelligent human nose (which “sniffs” or “sucks”), which has a second-order connotation related to a new significance: the idea of its strength, fit, and moving experience, which seems to be a totally new approach of framing the problem.

Almost at the same time, the designers tended to develop the intradomain and interdomain analogy referring to, for example, sports (using a harness, scooter, or flippers; lifting weights; cycling; dancing, etc.), biomorphism (animals: shark fin, humans: mouth, and vegetables), shoes and luggage (e.g., backpacks and accessories), and others. Of interest, these form descriptions also draw heavily on semantic and conceptual associations with the Nike brand (e.g., shoes for sports). Other sectors (e.g., industrial products, household electrical appliances, air conditioning, robots, containers, and real estate) were also spoken, but the way they are conceived is quite different. For instance, an intradomain analogy “droplet” was employed in both an expert designer's (E4) and a novice designer's (N7) protocol. Expert 4 said that “a droplet form makes me think of its speedy movement”; in comparison, the novice designer (N7) commented that “a form of droplet evokes a sport and aerodynamics, so I will try to cram all components inside [the vacuum cleaner], including motors, storage, and so on” (Figure 12).

FIGURE 10. The number of designers who mentioned semantic words: eight experts and eight novices.

FIGURE 11. Different uses of semantic descriptions and analogies produced by Expert 4 and Novice 5.

The initial three semantic words used by the designers and their corresponding forms enjoyed the special status of being considered the default design options. As shown in Figure 13, many expert designers tended to resort to their first three design semantic descriptions, which might lessen the advantage of developing new design concepts from the abstract/structural similarity. In short, although they might have more opportunity to develop creative design concepts from the second-order semantic descriptions, they had spent too much time on these and were reluctant to give them up quickly (M = 9.06 min for the experts vs. 5.44 min for the novices). An independent samples t test showed that there was a significant difference between experts and novices, t(14) = 2.35, p < .05.

FIGURE 12. Different uses of the analogy “droplet” between a novice (N7) and an expert (E4).

FIGURE 13. Initial design development with the first three semantic descriptions.

The results of Class I seem to show the different types of design cognition by the expert and novice designers. Consequently, different design fixations were incubated. The fact that the novice designer thinks of more first-order semantic descriptions rather than more abstract second-order connotations seems to make them develop a greater number of design options. In contrast, the expert designers who have abstract semantic descriptions tend to develop more creative design outcomes, but they take more time to develop them and therefore produce fewer design options. Of course, it cannot be said that all of the design alternatives produced by the novice designers are good. However, the more a designer focuses on one design option, the less likely he or she is to let go of that design option. This finding shows that the novice designers would spend more time in creating form descriptions compared to the expert designers. In contrast, the expert designer might spend more time activating design concepts with new semantic descriptions. Hence, from a design thinking perspective, it is more likely that the expert designers would be skewed toward semantic descriptions before they developed form descriptions, which shows a design thinking process for the experts: problem framing.

The expert designers also seem to employ the structural similarity in sketching rather than the physical or content-based similarity that the novice designers might be ready to use. As previously discussed, design cognition to search structural similarity accompanies design expertise and inferences for the experts. Novice designers cannot easily make this happen.

4.2. Class II: Design Choice Process

As Figure 14 shows, there were marked differences between the novices and the experts in Class II occurrences, χ²(4) = 26.40, p < .01. The expert designers tended to have a rather strong affirmation of their design decisions for the artifact compared to the novices (categorized as “affirmative decision of artifact”; experts = 32.71% vs. novices = 17.30%). An opposite pattern was found in the category of Decision to postpone (novices = 16.21% vs. experts = 9.40%). This type of affirmative design choice process seems to trigger the difference in Decision to go for new design alternatives, suggesting that the novice designers (27.03%) are less sure of their ideation so they look for new design solutions, which parallels the findings of Class I in Section 4.1. This was clear in the case of novices. For subject N1: “So, I will place a transparent tube here … but its form … is not defined yet.” After a moment of silence, N1 tried to find other alternatives: “Is there another solution for this grip? I don't know how I can express [it], but I am not feeling good about this form.”

This particular pattern among the novices might result from the fact that they do not feel that they have the ability to make decisions quickly based on their (limited) design experience and expertise; furthermore, they might have a lack of confidence in their design thinking. These findings are in line with the study of Hallihan et al. (2012).

The different ways of affective evaluation, such as using the expression “wow,” support the earlier interpretation, that is, quick decision by affective evaluation. Even though our verbalization protocol was limited to quantifying the affective aspects of design in relation to their processing, the results from the vocal expressions of the designers and corresponding video clips allowed us to identify some insightful empirical evidence; however, these findings were not statistically significant. The fact that the experts more frequently employ affective evaluation (76 cases, 28.57%) rather than the novice designers (37 cases, 20.00%), reveals the affirmative attitude of the expert designers and the perceived lack of confidence of the novice designers:

FIGURE 14. Occurrences of “design choice process” (Class II) produced by the eight experts and eight novices.

“This form of body is very appealing, I like this super Nike vacuum cleaner. Cool!” (E4)

“Finally, YES! I got what I was looking for. Like this, it becomes so much better and fun!” (E6)

In particular, we observe that our experts tend to use vocal expressions in an affirmative way based on the solutions or the task itself, such as “aha,” “oh yes,” and so on, and more often experience surprise or amusement during the generative phase. In contrast, the novice designers are more careful in their use of affirmative words when they evaluate the outputs or the task itself. Therefore, instead of using verbal acknowledgment, the novices tend to use vocal expressions demonstrative of uncertainty, such as “well” and “hmm….”

Although the expert designers, perhaps those with reflective design expertise (e.g., Schön, 1983), could make effective design choices, the novice designers tend to spend more time using backward reasoning following a particular pattern, referred to as trial and error (e.g., Ahmed, Wallace, & Blessing, 2003). Not surprisingly, our protocols confirm that their common behavior is to search out new design concepts: Decision to go for new design alternatives and Decision to compare options are much higher in novices (27.03% and 19.46%, respectively) than in experts (18.42% and 10.90%, respectively).

It is important to mention here the links between the demand for retrieving design alternatives and comparing existing options and its decision choice process. Almendra and Christiaans (2009) reported that design information can create new paths of research to encounter a solution but that it also serves the purpose of evaluation and/or confirmation of the existent hypothesis. This “test–retest” capability, which the novices tend to display more frequently than the experts, might allow designers to search for a new space of design alternatives, and perhaps this attitude might be able to help them to have more opportunities to come on design problem solutions. However, it is important to note that not all information requested can be available, and sometimes it can be ignored.

In effect, the expert designers have a tendency to avoid the longer cyclical process of “trial and error” or “test–retest” more commonly adopted by novices; rather, they tend to rely on their reflective expertise, past project experience, and/or their satisficing. This is confirmed by the fact that the experts have more cases than novices of Affirmative decision of artifact and Affective evaluation. Moreover, as shown in Figure 13, the expert designers focus longer on the first inspiring set of design alternatives. The possibility of the potential confirmatory bias that the expert designer might have in the course of design practices would hinder them from having better design alternatives at the later design stages.

FIGURE 15. Sketches resulting from affective evaluation (Experts 3 and 8).

In addition, the influence of affirmation might be magnified by overconfidence with affective evaluation. For instance, Expert 3 said at the beginning of the session:

“It is necessary to study ‘Nike shoes design’ too; however, [the vacuum cleaner] should not look like Nike shoes [but], rather, evoke Nike style and its association with ‘rapidity.” [00:12:58]

However, the designer subsequently sketched an object that exactly resembled Nike shoes:

“Like shoes, Ah! OUAIS, it will be super, fun, I associate it with shoes!” [00:52:13]

Similarly, Expert 8 repeatedly said, “This form is very appealing, it winks at Nike.” However, Expert 8's design is not advanced from the Nike logo itself. As shown in Figure 15, Expert 3 appraised his shoelike sketches and Expert 8 was attached to her Nike logo-inspired vacuum cleaner. Both have only the similarity of the design content rather than the structural similarity match.

This particular confirmative pattern among the experts can be linked with their level of (over)confidence. Almendra and Christiaans (2009) reported that some designers with high self-esteem (here, expert designers) are assertive and less likely to change their design during ideation. For instance, both Experts 3 and 8 demonstrated this overconfidence in their early design concepts with affective evaluation, and no substantial changes were made between their early sketches and their final design outcome.

5. DISCUSSION AND CHALLENGES TO HCI

Our empirical study concerned the way in which a designer retrieved design information and the design choice process during early conception generation. By overarching previous scientific models of design expertise (e.g., Cross, 2004; Ericsson & Smith, 1991), we identified rather differing design cognition between experts and novices. For example, expert designers are better at problem framing (i.e., employing mediated self) and generate more second-order connotations and analogical leaps in interdomains when coming up devising with creative solutions. These findings correspond with the research of Ericsson and Smith (1991) and Cross (2004), as well as with a study on varying expertise between chess players (Chase & Simon, 1973). However, the proponents of creative design are opposed to designers' use of experience and knowledge in evaluating design choices (Gentner et al., 1993). This is largely due to experts' frequent affective recall of their own prior work as a decision-making heuristic, and this fixation limits their exploration. In contrast, novice designers appear to more easily retain a third-person view (i.e., employing the simulated self) and to move to alternate designs if they prove superior (i.e., they are more concerned with problem solving), which is in line with the study of Ahmed et al. This rather distinct stance of the novice group (i.e., being less likely to fixate or take a first-person view) seems to be of great implication in design education.

Therefore, we discuss how to embody the two selves of designers (mediated self vs. simulated self) in the designer's cognitive processes by raising the question “How do we help novice designers design more like experts and expert designers avoid affect bias and design fixation?”

5.1. The Design Thinking Rationality Framework

Our study empirically demonstrated the first- or third-person view in the designer's cognition. Figure 16 illustrates the tallies of the points of view used by both experts and novices in the study. As the third-person view has long been considered in design practices, the focal point of our discussion is the cases in which designers used the first-person view.

The results confirmed that the third-person view was most commonly articulated by our participants—every artifact and user was referred to as “it” or “they.” For example, “Nike represents an image of dynamism and a total sport, and people might like it because it means something” (N3). Of interest, the expert designer tends to project herself or himself into the design process or the artifact itself, by to some extent referring to as “I” or “we.” Seven out of the eight experts employed a first-person viewpoint at least once in their verbal protocols; five of the novices did not take any first-person view in their design inferences or thinking. E4, who has 28 years' design experience, often employed the first-person viewpoint (13 times). This is somewhat surprising, because the universal philosophy of user-centered design claims that the designer's empathy to the artifact itself (i.e., employing the mediated self) should be avoided; instead, empathy for the user must be maintained at all costs (i.e., employing the simulated self), and we expected E4 to know this. However, our results showed that many experts easily projected themselves in their own artifact. For instance,

FIGURE 16. Occurrences of the use of a first- and third-person view.

“I have my aeration, my logo; I also have necessary techniques to aspirate; and my screen will be here on the grip.” (E1)

“I have my shoes and I run my Nike vacuum.” (E3)

E4 went further and used the design artifact as an agentive empathy, such as, “I am a R2D2 [Star Wars robot]. I aspirate at the bottom. I go all around in the apartment and I aspirate everywhere. I am totally autonomous.”

Such a first-person view of the expert designers might be explained by their risk perception attitude to the design alternative. Early studies by Alhakami and Slovic (1994) and Finucane, Alhakami, Slovic, and Johnson (2000) have empirically found that the inverse relationship between perceived risk and perceived benefit of an activity was linked to the strength of positive or negative “affect” associated with that activity. This result implies that designers based their judgments of a design alternative not only on what they think about it but also on how they feel about it (i.e., “how-do-I-feel-about” heuristic). That is, if they like a particular design alternative or a type of technology, they are more likely to judge the risks or uncertainty associated with it as low and the benefits associated with it as high; if they dislike it, they tend to form the opposite opinion, judging its risk to be high and benefit to be low.

This interpretation provides some insight into why many designers still prefer a sense of design account based on the “I-for-myself” concept, rather than employing the ideal “I-for-the-others” concept, which is endorsed by user-oriented design. In particular, our empirical findings suggest that experts regularly employ affective evaluation (see Figure 14), which allows them to effectively make quick design decisions. However, this may also cause them hold too firmly to initial design concepts, rather than modifying them in the subsequent design stages. In short, a high level of artifact empathy among expert designers might hinder them from making rational design decisions for the user, as this factor is linked with a higher likelihood of design fixation, confirmation bias, and frequent affective evaluation. However, an expert's first-person view may also bring about a closer engagement with their artifact, which then enables them to interpret initial usage problems and risks associated with their design relatively quickly and address design problems in a more creative way (i.e., better problem framing; Neustaedter & Sengers, 2012). The use of second-order semantic connotations (see Figures 10 and 13) and analogical leaps in interdomains, in which design elements bear no surface similarity, appears to stimulate this creative process.

In contrast with expert designers, our novice designers exhibited a lower level of artifact empathy. This was linked to a lower level of skill in framing design problems; however, this factor also facilitates their surrender of initial design concepts in pursuit of better alternatives (i.e., better problem solving). Certainly, as Stolterman (2008) found, a rational designer works on many alternative designs in parallel and in an iterative way. This particular pattern was more evident in the novices' behavior than that of the experts. This attitude provides novice designers with an opportunity to objectify their own design artifact in a more rational way and distances them from being too emphatic with regard to the artifact (see Decision to compare options and Decision to go for new design alternatives in Figure 14).

Taken together, the empirical results presented in Section 4 suggest that both design thinking and decision rationality are differently engaged in the designer's cognitive process. It is evident that the gulf of design thinking might arise from the simulated self in the designer's mind; by contrast, the mediated self might result in a lack of decision rationality. However, both the experts and novices lack full awareness of how these two cognitive processes are triggered and what they achieve. On one hand, a high level of artifact empathy via the mediated self should be avoided for problem solving, but on the other hand, emphasizing empathy for users via the simulated self might discourage a designer's willingness or ability to surprise users with creative and insightful artifacts. A key to design thinking rationality is thus to understand which mode of designers' selves—the mediated self and simulated self—work best and how to regulate or stimulate them in relation to both design thinking and design decision rationality in early design activities. Our empirical findings, as well as literature on decision science, hint that a greater use of second-order semantic connotations and analogical leaps between domains (for problem framing)—and less use of the affect heuristic and the “how-I-feel-about-it” heuristic (for problem solving)—is more effective. Based on this, Figure 17 illustrates the design thinking rationality framework, which summarizes several broad principles arising from this study; however, certain features are left for future studies.

The next section discusses how to use the design thinking rationality framework to regulate or stimulate both design thinking and design decision rationality. That is, how do we help novices to design more like experts and assist expert designers in avoiding affect bias and design fixation?

FIGURE 17. The design thinking rationality framework.

5.2. Applying the Design Thinking Rationality Framework

Problem Solving: Regulating Affect Heuristic

From a theoretical viewpoint, design choice is typically modeled as a cognitive procedure involving an analysis of a design option's constituent features. However, some theorists have argued that choices are made more intuitively or spontaneously. Our empirical study found that relying on intuitive affective impressions (Kahneman, 1994, described this as “choosing by liking”) may be a successful heuristic if the features that mediate the designer's initial affective response correspond closely with the features that determine users' subsequent enjoyment. However, if affective evaluation is unconsciously governed by something not closely related to subsequent satisfaction in design thinking, “choosing by liking” could lead one astray. In particular, several properties linked to affect render it an imperfect basis for design thinking rationality, the most important being excessive familiarity. Hence, experts may be more likely to adhere to their prior design knowledge and/or experience in problem solving, which is known as the “Curse of Knowledge” (e.g., Camerer, Loewenstein, & Weber, 1989)—a cognitive bias that causes better informed people to find it extremely difficult to think about problems.

Concerning this intuitive and automated thinking process, Schwarz (2010) proposed that individuals may use their affective reactions to the object as a basis for judgment. For example, when asked how “likeable” an artifact is, the designer's judgment would be based on his or her own feelings toward the artifact, rather than on a review of the artifact's constituent features. In particular, in cases where design decisions may present a task that is complex and demanding, designers tend to simplify the decision task by assessing their apparent affective reactions to a designed artifact, essentially asking themselves, “How do I feel about this?” (Finucane et al., 2000; Schwarz, 2010). Our empirical study corroborated this, showing that, in an empathic relationship with artifacts in which the designer employs the mediated self (rather other than the simulated self) in a design assessment, he or she quickly equalizes what she prefers and what the user might also like. For example,

My Nike vacuum cleaner is sporty and young. This is designed for a young, sporty couple, for a family with babies, and for me…. I consider myself a young and sporty man, thus, even though this is a bit antinomic, my Nike vacuum cleaner can be like a R2D2 which is highly autonomous. This is what I am doing at home. (E4)

Consistent with other theoretical work (e.g., Almendra & Christiaans, 2009; Hallihan et al., 2012), our empirical evidence also suggests that a reliance on one's feelings is particularly likely under conditions in which one's feelings (a) are a highly relevant source of information and (b) allow one to simplify an otherwise demanding task. In particular, the first point is critical to the expert designer. Not surprisingly, this is true when the judgment explicitly pertains to the designer's feelings (e.g., liking a particular form or color factor), in which case the designer's opinion is the most diagnostic input available (refer to Figure 6, E3 and E8). Accordingly, when a judgment is too complex to make by using a piecemeal information-processing strategy, experienced designers are likely resort to the “How-do-I-feel-about-it” heuristic. Therefore, the judgment of a designed artifact, which would require integrating many of its aspects, might be more strongly influenced by general affect than specific aspects of the artifact.

Of interest, the affect heuristic serves as a mechanism by which the remembered affect associated with a design option influences subsequent choices of that design alternative (e.g., Pham, 1998; Wright, 1975). Such an attitude by a designer predisposes his or her evaluation of an object (Pratkanis, 1989), with positive attitudes invoking a favorable opinion of an object and negative attitudes creating a negative response. Pratkanis (1989) defined this “attitude heuristic” as a way for the designer to preassign design options to a favorable class or an unfavorable class, thus leading to approach or avoidance strategies appropriate to the class. For instance, N1 expressed her consistent avoidance to a particular design option: “Is there another solution for this grip? I don't know how I can express [it], but I am not feeling good about this form” (N1).

Regardless of whether the designer intends to rely on the spontaneous intuitive evaluation (e.g., affect heuristic, how-do-I-feel-about-it heuristic, attitude heuristic, curse-of-knowledge) mentioned earlier or not, these factors probably play a role in his or her irrational decision-making process. First, because intuitive and automated responses precede more cognitive and effortful evaluations, they may dominate judgments and choices when designers have too little time for deliberate reflection (i.e., the designing task itself is complex and demanding), as was the case in our empirical study (which comprised 1 hr). Second, by providing some basis for choice, our initial automated response may discourage any further effort to find new design alternatives (this was also confirmed by our empirical study; see Figure 14). Third, judgments may be readily anchored to one's initial evaluation, even when attempts are made to supplement this with a more analytic and deliberate evaluation. Hence, the very fact that insufficient adjustment is so common is a major source of our bias in design thinking. Furthermore, the existence of insufficient adjustment puts the designer in a difficult position when determining whether a particular response is biased.

These studies might suggest how to develop a creativity-supporting system that can help expert designers avoid affect bias and design fixation and assist novice designers in approaching better design strategies. Providing relevant information about risk would allow for changes in the perception of benefit and vice versa. For instance, information stating that risk was low for a particular design option might lead to a more positive overall affect that would, in turn, increase its perceived benefit (Alhakami & Slovic, 1994).

Problem Framing: Stimulating Second-Order Connotation and Defamiliarization

Our findings suggest that novice designers think more about first-order semantic descriptions based on the similarity of surface features than second-order semantic connotations; the former would have enabled them to develop a greater number of design options. In contrast, expert designers who generate second-order connotations have a tendency to develop more creative design outcomes; however, they take more time doing so and therefore produce fewer design options (and consequently tend toward design fixation). Hence, from a design thinking perspective (and supported by our empirical study), expert designers prefer to consider more semantic descriptions before developing particular forms, which illustrates a particular design thinking process. Expert designers also seem to employ structural similarities in sketching rather than the physical or content-based similarities that novice designers tend to use. These findings are very much in line with a study concerning expert–novice differences in design-based analogizing (Ball, Ormerod, & Morley, 2004). They demonstrate that expert designers employ highly schematized knowledge structures based on the automatic recognition of familiar types of problems (what Ball et al., 2004, referred to as “schema-driven analogizing”).

Of course, second-order connotations would not be the sole driver of creative design quality. On a practical level, it is common for designers to rely on their own design intuition when creating new products, and not surprisingly, many use their own metaphor in developing early design concepts. Neustaedter and Senger's (2012) “autobiographical approach” thus offers an important suggestion, stating that the deliberate employment of the “I-for-myself” concept in the early design process can quickly generate initial, simple, and imaginable uses for the user, even when no particular design requirements were specified. That said, Tomico, Winthagen, and van Hesit (2012) noted that a first-, second-, and third-person point of view in design thinking should be incorporated together, with an openness to different backgrounds and affordability to enrich and widen the design solutions. This might include a first-person view to “discover” simple and imaginable uses, a second-person view to “define the specifications and requirements,” and a third-person view to “develop and deliver” to the users.

However, although the designer's first-person point of view is valued in problem framing, some designers remain sceptical. For instance,

I just think of it not as designing for myself and then foisting it off on others—usually we're designing with others in mind, and if we can use it ourselves for something real, that gives us this huge, rich, set of feedback and insight that value in design. (Interview excerpt from Neustaedter & Senger, 2012)

In this regard, our empirical findings provided a counterbalance to the positive aspect of the first-person view in design thinking, demonstrating that there might be other critical benefits to designers distancing themselves from the first-person view (while retaining a third-person point of view), such as enabling the designer to defamiliarize their early design concepts.

Extensive and diverse evidence shows that familiarity with a design option increases automated choice (e.g., Ball et al., 2004), as it is more readily available and provides an easier basis for a decision than deliberately thinking about each option. Furthermore, because affective evaluation happens first, the option that elicits the most familiarity may enjoy the special status of being the default option. In other words, we will choose this option unless there is decisive evidence in favor of something else. Of interest, our empirical study observed that a designer inadvertently found a bicycle wheel in the experiment room, and subsequently sketched an object related to the form of the bicycle, stating, “I had seen the mini sports shoes on the bicycle wheel in the show window; and Nike also makes sport shoes that might be something for me to make” (see Figure 18). This implies that he enjoyed the familiar external stimuli to catalyze his design cognition in the rest of his design practice. Hence, the defamiliarization technique, as widely examined in drama theory (e.g., Frederick, 1998; White, 2004), is worth applying in certain design stages. Applying this technique to design thinking—making the “familiar” strange—can thus serve to enlighten the designer not to take their own design inferences and thinking for granted.

FIGURE 18. Example of sketches influenced by an external stimulus.

Given the aforementioned information, the use of defamiliarization to prevent designers from putting themselves in an automated response mode and to draw them into an attitude of critical judgment, may lead to other reactions to their own artifact. In a real design context, an insufficient level of diversification might also cause many deficiencies in design thinking. How to identify a sufficient level of diversification is therefore a major issue to be addressed in both User-Centered Design and design thinking. The ideal cases of user studies would support this.

5.3. Further Study and Limitations

Some researchers (e.g., Tomico et al., 2012; Trotto, Hummels, & Restrepo, 2011) have positively embraced the concept of “points of view” in design education, emphasizing the radical shift of meaning as a key condition to design-driven innovation (e.g., Brown, 2009; Martin, 2009). They have claimed that designing from a first-person perspective will be meaningful; in particular, the integration of different points of view and the use of designers' intuition enriches the design process and builds a basis for a richer result with regard to meaning and, therefore, design innovation. Schön (1983) also characterized this approach as “reflection-in-action,” stating that professionals should rely less on formulas learned in schools and more on the type of improvisation learned in practice.

This article concludes that the designer needs to oscillate between high involvement with the artifact for innovation design and intentional detachment from the artifact for rationality design. By this we mean that the design thinking rationality framework would enable the designer to step back from his or her projected experience to take note of wider relationships upon which the qualities of the whole design concept will depend. Furthermore, it claims that the HCI community is in need of amalgamating a broader theme of engineering design processes and design-oriented cognitive models. A better understanding of designers' cognition would result in suggestions for a new direction for the HCI perspective built upon the engineering design process.

Of course, we are not the first researchers to study both design thinking and design decision rationality. However, our study is different in that we integrate both into a single design thinking rationality framework, drawing upon an empirical study. The framework represents designers' prominent cognitive processes (e.g., heuristics, second-order sematic connotations, defamiliarization, etc.) and proposes ways of regulating or stimulating these processes in an actual design activity.

We acknowledge that the experimental setting in our empirical study was somewhat arbitrary and not ideal for pinpointing the designers' cognition. Further work on individuals' design behavior and experimental validation is still needed. First, the present study was limited to a designer's mental categories, without explicit design sources or information being provided. A subsequent study with explicit design sources in the early design process has been carried out (Kim et al., 2013). Second, it would be fruitful to explore how a group of designers might be affected by social design practices to update or relinquish their design concepts. For example, the frequent use of the first-person view demonstrated by experts would be worrying in a situation requiring a collaborative design. Third, another potential weakness concerns the design task itself. Although the focus of our research was not to study the task effect in design thinking, the use of a simplified and well-defined design brief in a reductive condition might challenge our interpretations of the results when compared with a real design brief (Lawson, 2004). The data collection instruments also proved problematic, as participants were asked to verbalize their thoughts while sketching, though some were uncomfortable with this design exercise. A reflective verbal protocol technique might have been more effective. Fourth, the experimental design necessitated a careful recruitment strategy to eliminate systematic bias in designers' level of experience. Although the participants were recruited from a single cultural background (France), we found that there might be interindividual differences due to a wide range of ages, educational backgrounds, previous design experience, and so on. Hence, extending our accounts to other groups of designers is an important future task. Finally, independent reviewers' evaluation sessions on the design outcomes would be of great interest. Kim, Bouchard, and Ryu (2012) study has partially considered this approach. However, to evaluate the creative quality of the design outcome requires further controlled experimentation. We leave these limitations as interesting issues and avenues for further research.

Based on the theoretical foundation and empirical findings in the present study, we are currently working toward developing a novel HCI design method to effectively insert the first-person view into the user-centered design process. This further research will help to educate HCI practitioners on using artifact empathy as a novel design strategy to compensate for the user-centred design process. It is hoped that this will empower HCI designers to make better design decisions in the early stages of design.

NOTES

Additional information

Notes on contributors

Jieun Kim

Jieun Kim ([email protected]) is a design researcher with interest in design cognition, applied creativity, and emotional design, applied to topics ranging from creativity support tools, critical learning, and serious games; she is an Assistant Professor in the Graduate School of Technology and Innovation Management of Hanyang University, Korea.

Hokyoung Ryu

Hokyoung Ryu ([email protected]) is a human–computer interaction (HCI) researcher with special interests in engineering design processes and poststroke rehabilitation therapies; his HCI study has focused on developing a theory-binding design process and applying it to innovation design practices; he is an Associate Professor in the Graduate School of Technology and Innovation Management of Hanyang University, Korea.

Notes

Acknowledgments. We thank the anonymous reviewers of the article for their detailed comments and helpful suggestions. We are especially grateful to Professor Andrew Monk and Professor Carol Bouchard for their food-for-thought of this article and Ms. Ahreum Lee for her extensive literature review. The requests for the design materials should be addressed to Hokyoung Ryu.

Funding. Major funding for this work was provided by the National Research Foundation of Korea (NRF) grant (20110021398) and the Industrial Strategic Technology Development Program (10042694).

HCI Editorial Record. First received on February 15, 2013. Revisions received on October 18, 2013, and January 17, 2014. Accepted by Scott Klemmer. Final manuscript received on January 27, 2014. — Editor

APPENDIX. EXPERIMENT MATERIAL

A1. Design Brief (FR—original version)

Concevoir l'aspirateur NIKE

NIKE, Inc. est un des plus grands fournisseurs de chaussures et de vêtements pour athlètes et un important fabricant d'équipements sportifs. ‘BIG- Big-thinking Innovation group’ est une unité indépendante au sein de NIKE Inc. Si le métier de NIKE est de créer de l'inspiration et de l'innovation pour chaque athlète dans le monde, celui de BIG est d'anticiper le futur de cette innovation et d'imaginer ce que les gens désireront dans un avenir proche.

L'industrie du vêtement pour sportif et NIKE ont toujours mis l'accent sur la qualité de l'expérience qu'ils peuvent apporter aux consommateurs dans le cadre d'une activité de plein air (par ex. gym, natation); cependant, l'expérience de NIKE n'a pas encore été appliquée au domicile de tout un chacun. En fait, le marché du domicile est en plein essor dans la plupart des pays, signe d'une prévalence de l'activité au sein du foyer et par là même du foyer comme un territoire stratégique pour NIKE, Inc.

Dans ce contexte, pour la première fois dans l'histoire de NIKE, le siège de NIKE-BIG à Paris a décidé d'explorer sans se fixer de limite ce que l'expérience continue de NIKE dans le domaine du plein air pourrait apporter aux activités domestiques tout en repensant l'intimité du foyer.

Comme mini-projet, l'activité de nettoyage de la maison a été identifiée comme une des luttes émotionnelles depuis plus d'un siècle. Par conséquent, concevoir un aspirateur NIKE est du plus grand intérêt. A présent, votre objectif est d'imaginer cet aspirateur NIKE.

A2. Design Brief (EN—translated in English)

Designing NIKE Vacuum cleaner

NIKE, Inc. is one of the world's largest suppliers of athlete shoes & apparel, and a major manufacturer of sports equipment. ‘BIG- Big-thinking Innovation group’ is an independent unit within NIKE Inc. If NIKE's job is to create an inspiration and innovation for every athlete in the world, BIG's job is to anticipate the future of innovation and imagine people's desires in the near future.

The athletic wear industry and NIKE have been traditionally focused on the quality of the experiences they can bring to consumers for an outdoor activity (e.g. gym, swimming pool); however, a seamless NIKE experience has not yet come to the home. In fact, the home-based market is a fast growing business in most countries, showing the prevalence of home-based activity and the home itself as a strategic territory for NIKE, Inc.

In this light, for the first time in NIKE history, NIKE-BIG headquarter in Paris decided to explore boundless challenges about what if NIKE brings a perpetual NIKE experience from outdoor to indoor activities, and recreates the home intimacy. As a quick-in-project, ‘Cleaning home activity’ has been identified as one of the emotional struggles for over a century. Thus, designing a new NIKE vacuum cleaner is of great interest. Now, your task is to design a NIKE vacuum cleaner.