Mental models in design teams: a valid approach to performance in design collaboration?

Abstract

This article provides an overview of research into mental models in teams and discusses the relevance of this theoretical concept for design teams. Researchers in several disciplines have applied the construct of mental models to understand how people perform tasks based on their knowledge, experience and expectation. The notion has also been used to study teams and to analyse the relationship between team mental model and team performance. Five different types of mental models for studying design teams are proposed: task, process, team, competence, and context. A review the literature found only very few studies on team mental models in design-related areas. A brief overview is provided on what is known about team mental models in general, on the effect of team mental models on team performance, and on what kind of results can be transferred to design teams. A short review of measurement techniques and how they can be applied to design research is presented. Finally, implications for the field of design are discussed and a methodological framework for studying mental models in design teams is proposed.

Keywords:

• Team mental models
• Collaborative design
• Design cognition
• Shared understanding

1. Introduction

In the morning of 14 August 2005 the German pilot and Cypriot co-pilot started the scheduled flight FL 522 from Larnaca to Prague, Czech Republic. They never reached their destination but crashed some 40 km north of Athens causing 121 casualties. What happened? The pressurization system of the Boeing 737-300 had been checked the night before the start; after completion of the tests the Pressurization Mode Selector (PMS) was left in the ‘manual’ position instead of the ‘auto’ mode with the outflow valves one-third in the open position. Thus, the cabin did not pressurize after take-off. When the airplane passed through an altitude of 10,000 feet the cabin altitude alert horn sounded. Both pilots interpreted this cabin altitude alert erroneously as a take-off configuration warning because the sound is identical. However, the takeoff configuration warning only occurs on the ground. If it is activated in flight, it indicates that the cabin altitude has reached 10,000 feet. The pilots therefore focused on switching off the seemingly erroneous signal (http://aviation-safety.net/database/record.php?id=20050814-0). In this situation both pilots activated the same mental model, which may be due to the simple fact that they had encountered the take-off configuration warning more frequently, and had seldom or never heard the cabin altitude alert horn during flight before.

An international team of engineers from different disciplines is working together to develop the cabin vision for a new aircraft. The aim for the new cabin environment is not merely to feature the newest technology but mainly to enhance comfort. This includes, for example, the lighting and a guidance system for passengers, which provides the latest technical solutions for mobile phones and other kinds of communication facilities. The design of a new aircraft is a large-scale, complex project that involves not only aeronautics engineers but a multitude of experts from various disciplines, such as material science or communication and information technology. In order to complete their task, the team members have to develop a mental model of the specific content and how to approach and manage such a project.

These two examples highlight the importance of team mental models in designing and operating complex technical systems. In the first case, the team members had the same understanding of what was happening, but unfortunately their model was wrong. In the second case, the team will probably struggle to share an understanding of their project among all members. Recent developments in design management such as Integrated Product Development and supply management have succeeded to shorten lead times and to reduce production costs; yet all these developments have increased the need for effective and efficient team communication. Even if mental models are sufficiently shared among team members, it will be difficult to know if they are appropriate—especially in cases where the team is developing such a complex product with a multitude and variety of parts, which have yet to be tested and evaluated. Perhaps the team can envisage the needs of the future and the context of use in a way that both the company's shareholders and the passengers as end-users are satisfied. Yet, evaluating the success of a new design is typically multifaceted and non-trivial even in hindsight, let alone during a project. There are few notable exceptions where researchers have attempted to pinpoint design communication during meetings such as reviews (Foley and Macmillan 2005, Ostergaard et al. 2005).

Previous research in other domains (mainly on operating complex systems rather than designing them) has shown that mental models play an important part in team communication and performance (Klimoski and Mohammed 1994). Yet the actual process of how mental models are developed and how they influence what the team members do and think is still poorly understood, especially for design teams. The purpose of this paper is to introduce the concept of mental models, to review the previous literature and to critically evaluate their suitability and implications for design research.

2 The emergence of the concept of mental models

The idea that we develop mental copies of objects of the real world in our head has a long history in philosophy, dating back to the ancient Greeks. However, it was not until 1943 that Kenneth Craik elaborated upon this idea in order to apply it to studying cognitive aspects of human behaviour. In his view, the mind creates small scale models of reality (Craik 1943). Every new, unfamiliar object we perceive in the real world is modelled in a similar but adjusted representation. We reason, explain, and anticipate events based on those models. Although his notion had some influence in the emerging field of cognitive science, researchers did not use the term mental models as an explicit research paradigm until much later.

The term mental models was reintroduced to the cognitive science community in the early 1980s by Johnson-Laird (1980). Two books brought this concept back into the focus of attention. Johnson-Laird (1983) proposed mental models as the basic structure of cognition, which he used to describe how people reason about reference in terms of syllogistic reasoning (see also Oakhill and Garnham 1996). In another important publication from the same year, Gentner and Stevens (1983) introduced mental models to the ergonomic community in order to study the interaction between humans and systems. When interacting with the environment, with others, and with the artefacts of technology, people develop internal mental models of themselves and the things with which they are interacting. These models provide predictive and explanatory power for understanding these interactions (Norman 1983).

As the two different conceptualisations by Johnson-Laird (1983) and Gentner and Stevens (1983) indicate, the notion of mental models seems applicable to various disciplines, all with their own foci. Therefore, it is not surprising that a review of the definitions and use of mental models in different domains (Rouse and Morris 1986) revealed many different understandings of the concept of mental models. Rouse and Morris argue that in order to avoid just rephrasing knowledge as mental models, the particular type of knowledge of which they are composed, and the purpose for what they are used, should be considered. They suggest that research on mental models should not be seen as generic but studied specifically for each domain.

2.1 What is modelled?

Johnson-Laird (1983) proposed that at least three types of mental representations exist: ‘Propositional representations, which are strings of symbols that correspond to natural language, mental models, which are structural analogues of the world, and images, which are the perceptual correlates of models from a particular point of view’ (p. 165). He argued that both propositional representations and mental images are dependent on mental models. Depending on the task the mental models are related to, they are diversely implemented, taking various forms with different levels of accuracy. They might include concepts, propositions, scripts, frames, or mental images (Bainbridge 1992). Mental models are therefore not simply equivalent to (declarative) knowledge. Although they are stored in memory, mental models are specific to a given task at a particular time period and dynamically changing over time.

An essential issue in the conceptualisation of mental models is the relationship between representations. It is more about how knowledge is structured and related than whether specific knowledge is present. Thus, mental models reflect our tendency to categorize what we know and how this knowledge is organized (Klimoski and Mohammed 1994).

2.2 Mental models are simplified representations of the world

Summarizing the theoretical assumptions, we can state that the basic idea of mental models is that humans construct internal working models of the world. These models allow them to integrate new information and to make predictions with little mental effort. Due to their nature, these working models are necessarily simplifications of the world (Smyth et al. 1994). For example, not every feature of a scene (e.g. a pilot taking off) is included in the corresponding mental model (or ‘script’ or ‘schema’) of that scene. Mental models are therefore not necessarily accurate representations of the world. It is precisely their simplicity that makes them so useful because they enable a person to quickly understand and act even in new and unknown situations (Oakhill and Garnham 1996). Of course, simplification also has drawbacks. Especially when performing a task such as operating a complex technical system, inaccurate models can have negative effects. For example, simplifications such as mistaking accidental co-occurrences for causal effects is a frequent reason for inaccurate models (Besnard et al. 2004). As the case of the crashed airplane exemplifies, an inaccurate mental model leading to the misinterpretation of a warning signal, can have catastrophic consequences.

3 Mental models in teams

Most early research on mental models focused on the investigation of individual mental models, and it took another 10 years until the concept of mental models was introduced to team settings. Shared or team mental models (TMM) are characterized as knowledge or belief structures that are shared by members of a team, which enable them to form accurate explanations and expectations about the task, and to coordinate their actions and adapt their behaviours to the demands of the task and other team members (Cannon-Bowers et al. 1993, Klimoski and Mohammed 1994). ‘The term team mental model is not meant to only refer to multiple levels or sets of shared knowledge or just to an aggregate of the individual mental models but also to a synergistic functional aggregation of the teams mental functioning representing similarity, overlap, and complementarity.’ (Langan-Fox et al. 2004.)

As team mental models describe both individual mental models and how they are shared or distributed within a team, this concept seems very suitable to investigate how complex problems are solved within teams. Design problems can been defined as specific kinds of complex problem-solving (Hacker, Sachse and Schroda 1998, Badke-Schaub and Frankenberger 2004). Using team mental model research to investigate design problems might help to understand how the solution finding evolves and how it is communicated and (implicitly) coordinated in a team.

3.1 Content of mental models

The development of a mental model depends on the current situation, or more precisely, a mental model is built based on the requirements and constraints that determine the cognitive processes of a given situation. Usually more than one mental model is constructed. As a result, mental models are specific for each task and team configuration. Thus, usually at least two types of mental models are to be distinguished: the task model and the team model. A more detailed distinction into four models that is commonly used was proposed by Cannon-Bowers et al. (1993): the task model, the equipment model, the team model, and the team interaction model. These models were identified in studying mental models on human – system interaction; hence equipment/technology was stipulated as a separate model. The equipment model contains knowledge about the functioning and handling of the apparatus, as well as likely failures. For many studied areas as flight crews or software development teams, this distinction has been proved suitable. For design teams, however, there seem to be partly different demands. We will come to this later. Thus, we developed a slightly different model and assume that mental models can relate to knowledge about the task, the process, the group, the competence, and the context (see Neumann et al. 2006).

The task model relates to a person's stored knowledge regarding the particular task. From a design perspective, the knowledge about the technology is not different in essence from other knowledge related to a task. Therefore we treat knowledge about the equipment as part of the task model. It also includes product knowledge, such as information about relevant information of the object to be designed. Additionally, we propose a process model that refers to the knowledge of how to solve a design task. This can be problem-solving strategies as well as particular design methods that are used by designers when performing a task. The process model is different from the task model as it focuses on how to handle a task and not which knowledge of facts is needed when performing a task.

The team model includes the knowledge about the other team members, the knowledge about their abilities and the responding roles and responsibilities, and how to interact with the different team members. The competence model is a general confidence in how far the team is able to perform a task. Therefore, this model is related or based on the three other models. Competence-related issues have an impact on the task acceptance and consequently contribute to the team performance. The context model refers to all the ‘background knowledge’, which reflects the given situation such as the used media of communication, organizational aspects and facilities, and further more.

3.2 Development of mental models in teams

Figure 1 shows how mental models in teams develop. Based on the current situation, each team member A, B, C perceives reality due to his/her active perception, memory, prior knowledge, and needs. These factors can differ substantially between the individual members as each individual has their own background knowledge, prior experiences, expertise, or aims. These features cause the development of the individual mental models. When the team members exchange their models in communication, they build up a team mental model. This can be related to one of the five content aspects: the task, the process, the team, the competence, or the context. As a result, the presence and features of these five models affect team performance. Besides the mental models, the expectations of the members which are built upon the individual skills and abilities also influence the result.

Figure 1 Mental models.

3.3 Need for sharedness is domain-specific

Most research on team mental models has focused on operating complex technical systems rather than designing them (e.g. Mathieu et al. 2000, 2005, Smith-Jentsch et al. 2005, Waller et al. 2004). While only few aspects of flying an aeroplane or operating an aircraft carrier involve inventive problem-solving, most activities follow standard operations and procedures. For more creative tasks, i.e. design, the procedures to follow are not previously known. The requirements for mental models to be shared in teams might consequently be rather different.

In general, a distinction between creative teams and tactical teams can be made (Larson and LaFasto 1989). The main characteristic of tactical teams is the clarity of the task. The emphasis of the process lies on directive, highly focused tasks, unambiguous role clarity, well-defined operational standards, and accuracy. On the contrary, the main characteristic of creative teams is autonomy. Members of such teams are required to explore possibilities and alternatives and to create an atmosphere in which ideas are not prematurely discarded. It is therefore unlikely that a straightforward transfer of the previous research is possible.

From research on creativity, it is known that different views and a diversity of thinking can foster a creative problem-solving process and thereby improving the resulting solution. However, it is probably also important to share at least some aspects about the task and the team. For example, discussing the problem definition and requirements of a product facilitates the common understanding within a team when searching for a solution. It has also been shown that creative teams share their goals and their teamwork strategies in order to be efficient (Gilson and Shalley 2004), and that members of a team should be aware of their social context in order to come up with creative solution (Hargadon and Bechky 2006). All in all, there are strong indices that shared mental models have an impact on creative problem-solving.

4 Characteristics of team mental models

Besides the content of a mental model, the quality of the mental model that is shared within a team effects the outcome. Three major factors that influence the quality of team mental models are discussed here: sharedness, accuracy, and importance.

4.1 Sharedness

In the literature on mental models, there are a variety of approaches to sharedness. Although most research focuses on sharedness as knowledge that is held by all members of a team, other views are valid as well. From the definition of team mental models (Klimoski and Mohammed 1994), sharing includes both holding in common (i.e. overlapping or identical) as well as divided (i.e. complementary). Team members can have the same mental models when performing a task, which is most relevant for two persons who share one job such as piloting an aeroplane. Besides, team members can have overlapping knowledge, which might happen between different persons who perform the same task in different but connected fields such as the pilot crew and tower crew. And there is the possibility that mental models are distributed in that every member has some specialized knowledge.

For complex tasks like designing an aeroplane or just a part of it, it is obvious that there is not one shared mental model of all team members. Thus, especially in heterogeneous teams like multidisciplinary teams in which distinct team roles require unique knowledge, mental models need to be distributed throughout the team (Cooke et al. 2000). The related concept of transactive memory deals with this topic in more detail. The idea behind this concept emerged by finding that couples share their knowledge by knowing who knows what; so the knowledge from the partner can be assessed as well (Wegner et al. 1991). Whereas team mental model research emphasizes the overlap among team member knowledge, the concept of transactive memory focus on the distributed nature of information in the team (Mohammed and Dumville 2001). There seems to be some reason that too much overlapping knowledge in teams with distinct roles may be inefficient and create a redundancy of effort, and that too much distributed knowledge may undermine the ability of the team to work together as a coordinated whole. Therefore, both must be kept in balance in the team.

The question arises for what kind of knowledge a higher amount of sharedness is necessary and for which it is not. The optimal degree of sharing will likely be dependent on a number of factors, such as the specific environment in which a team operates, the nature of the task, and in terms of development in which stage of its life cycle a group is working (Mohammed and Dumville 2001).

As mentioned before, in design and other teams performing complex tasks, more sharedness is not always better. When searching for solutions, different views are useful in order to broaden the solution space. Greater divergence of mental models at the beginning of a design task (in order to generate creativity), coupled with greater convergence of mental models at the end of the task (in order to facilitate implementation), may contribute to high performance. In any case, in order to use distributed knowledge, it seems essential that there is at least a shared mental model of the team in order to know which knowledge is available in the team. Furthermore it seems likely that a shared mental model of the roles and responsibilities in the team facilitates teamwork.

4.2 Accuracy

The second aspect determining the quality of a mental model is its accuracy. Even if all members of a team agree on a common mental model, it does not imply that the model is accurate (Rentsch and Hall 1994). All members of a team can share some identical knowledge, but all of them might be wrong. In the first example presented at the beginning of the paper, both pilots had an identical mental model, but both models were inaccurate. Individuals can also construct inaccurate models, such as making causal inferences about consecutive events, which result in aeroplane crashes (Besnard et al. 2004). Although not all situations in which inaccurate models are shared lead to such catastrophic consequences, it illustrates that accuracy is of major importance when evaluating mental models. Accuracy and sharedness are related to each other. If team members have accurate mental models, they also must be shared. But since both concepts measure two different aspects of team mental models, they are not redundant. Both, similarity and accuracy have been shown to influence team performance (Edwards et al. 2006, Lim and Klein 2006).

In structured tasks, accuracy has been assessed by comparing an individual mental model with an expert's model (e.g. Edwards et al. 2006, Lim and Klein 2006). For example, when a pilot has to react to a certain constellation in a training situation, his/her mental model of that situation can be compared to that of an expert or trainer to check whether it is accurate. But how can we measure accuracy in ill-structured problems, where no one clear solution can be defined? The designers in the second example presented at the beginning of the paper might agree on the requirements for a design, but determining in how far their models are inaccurate is difficult. Team mental model researchers have acknowledged that in certain performance contexts, there are likely to be multiple, accurate models (e.g. Mathieu et al. 2005). When there are different ways in which a team can be effective, similarity might be even more important than accuracy.

4.3 Importance

The last point that contributes to the quality of a mental model is importance. Mental models that include important aspects of the task or the team might have a bigger influence on the performance. Central issues are more important to be shared than marginal issues, resulting in higher-quality mental models with a major effect on team performance. If central aspects of a task are shared, this contributes to a better understanding of the problem, for example, than when many special issues are held in common but the overall problem is not perceived in a similar way.

One way to operationalize importance is the identification of central attributes. When mental models are conceptualized as networks, a central attribute in the mental model has many links within the overall network and is therefore highly related to other attributes in the network (Mathieu et al. 2005). This centrality makes an attribute more important because it consequently determines how knowledge is structured and thereby how the task is understood. It can also be viewed as an anchoring variable for other concepts, which facilitates communication about the task.

5 Team mental models, processes and performance

The major reason to study mental models in teams, especially the sharedness between members, is the underlying assumption that shared mental models positively affect team performance (Cannon-Bowers et al. 1993, Klimoski and Mohammed 1994). Commonly held mental models are thought to provide a set of organized knowledge of the task and the team from which predictions about team member behaviour can be drawn and relied on. Two types of requirements have been proposed that call for highly shared mental models within a team: coordination and communication (Stout et al. 1996). Research has stated that sharedness of mental models is important for tasks with the following specific characteristics.

Shared mental models are important for tasks that require highly coordinated actions between different team members. In such teams, members have to act according to their predictions of each others' understanding of the task demands and their behaviour. This coordination results in better planning, thereby improving coordinated team decision making and performance (Stout et al. 1999).

Shared mental models are important for coping with tasks with difficult communication. Team performance can benefit from shared mental models in situations with a high need of information exchange in the team. In situations where only limited communication is possible, due to heavy workload and time pressure, shared mental models were found most valuable (Mathieu et al. 2000). But even when task demands like time constraints are not critical, or when tasks are performed in teams that do not share all relevant information, shared mental models improve communication and thereby performance.

Beyond these two characteristics, studies have focused on teams in dynamic environments that require highly coordinated actions. Most of them are control teams that operate systems such as flight crews (e.g. Cooke et al. 2000, Mathieu et al. 2000, Weick and Roberts 1993), power plant control crews (Waller et al. 2004), software development teams (Carley 1997, Espinosa et al. 2002), and sport teams (Eccles and Tenenbaum 2004, Webber et al. 2000). Other studies investigated teams operating tasks that allow free communication, such as management teams (Ensley and Pearce 2001), long-term student groups (Peterson et al. 2000), and development teams (Akgun et al. 2005, Davison and Blackman 2005).

In general, empirical studies have shown that similarity of mental models has a positive effect on performance (Lim and Klein 2006, Marks et al. 2002, Mathieu et al. 2006, Smith-Jentsch et al. 2005). In a meta-analysis by Griepentong and Fleming (2003), the authors found that shared mental models, as a whole, are positively related to team performance. In addition, they concluded that shared mental models are related to better performance in tasks that require higher interdependence among team members.

Taking into consideration that the relation between mental models and performance is task-dependent, it seems that it should be possible to answer what aspects of a task should be shared. However, the key elements of a task are very dependent on the job that is performed. A comparison of studies in different domains is therefore a non-trivial undertaking. What makes things even worse is that all studies use their own measurement techniques which are applied to the task and circumstances at hand. In the following sections, we will try to give a short overview of the state of the art concerning research on how aspects of content, quality, and sharedness are related to team performance. Given the diversity in domains that are studied and methods that are applied, we only pinpoint major issues rather than provide a precise picture.

5.1 What content should be shared?

Given the earlier distinction of different content of mental models, empirical studies have focused primarily on task and/or team mental models (e.g. Lim and Klein 2006, Mathieu et al. 2000). As has already been mentioned, both types of models can have a positive influence on performance. Despite recognizing that multiple types of mental models operate simultaneously, most empirical research has examined either taskwork or teamwork mental models (e.g. Edwards et al. 2006, Rentsch and Klimoski 2001, Stout et al. 1999, Webber et al. 2000). Those studies that examined both types of mental models, found that shared task models have a stronger relationship with better performance than do team interaction models (Griepentrog and Fleming 2003). However, the assessment of team mental models is more difficult than of task mental models; mental models of team members and their interaction entail probably more unconscious elements and there is no specified reference to relate to, whereas task aspects are more easily linked to ‘objective’ characteristics of the content. Recent studies have begun to explore the interaction between team and task mental models, finding support for a greater positive relationship between task mental models and effectiveness (Mathieu et al. 2006, Smith-Jentsch et al. 2005). The authors have shown that teams that develop task mental models and perform effectively also develop shared team models.

Clearly divided roles and responsibilities are also important for successful team functioning. The mental models about the team in general can thus be expected to have an impact on coordination and communication. For example, people with more experience working in teams had more detailed mental models about teamwork; novices who learned these models made their mental models more similar to the experts (Smith-Jentsch et al. 2001). What kind of aspects of mental models should be shared is thus not only dependent on the task but also on the team at hand.

5.2 Accurate, similar and important mental models = better performance?

Mental models that are shared within a team must match several key characteristics like accuracy, similarity, or importance in order to be an effective predictor for performance. In other words, they should be of high quality. Team processes and performance were found to be better among teams sharing higher-quality team mental models than among teams with low-quality models or less sharing (Mathieu et al. 2005). Accuracy is a strong predictor of team performance, and the accuracy of mental models also mediates team ability and performance (Edwards et al. 2006). It was also found that a high similarity of task and team mental models predicts team performance as does the accuracy of team mental models (Lim and Klein 2006). With respect to the importance of mental models, the relationship to team performance is more difficult to determine. It seems that shared models that are central have a stronger effect on performance than do marginal models (Mathieu et al. 2005).

5.3 Shared or distributed?

Although most studies focus on shared in the meaning of overlapping or similar, there are also studies that investigate the effects of diversity on team performance (e.g. Austin 2003). Distributed mental models (and cognitive diversity) has been shown to improve system understanding in complex tasks such as system operation and thereby improve team performance (Sauer et al. 2006). It seems very likely that diversity can facilitate the outcome in tasks which require specialized knowledge by members of a multidisciplinary team. Nevertheless, more information is needed on what aspects of task mental models should be held in common when using divergent knowledge. It might be assumed that team aspects become more important whenever diverse knowledge has to be coordinated.

5.4 More sharedness = better performance?

Summarizing the results of the presented research on mental models in teams and performance, it can be stated that there is an obvious link between sharedness and team performance. But although most studies have shown that sharedness is a facilitator in certain team tasks, there are also studies that predict no or negative effects between sharedness of team mental models and performance (Smith-Jentsch et al. 2005). The major reason for that might be a pattern of a team behaviour known as groupthink (Janis 1972); based on too much cohesion team members tend to develop a mind set where unanimity in the team overrides the motivation of team members to realistically evaluate alternative decisions, resulting in suboptimal solutions. Too much sharedness might therefore lead to a similar phenomenon which can have negative effects on team performance. There seems to be some evidence for a curvilinear relationship between the sharedness of mental models and performance, with low levels of sharedness associated with a lack of coordination, while high levels of sharedness associated with groupthink, both variants leading to poor performance.

Further, there are characteristics of the task and of the team which influence what level of sharedness is optimal for team performance. Especially in situations where tasks are highly complex, with high workload or when communication is hindered, a higher sharedness of mental models seems to have the most benefit to performance. Operational tasks with clear procedures need a highly shared task model whereas tasks that require individual decisions like creative activities might be more reliant on highly shared team models.

Given the empirical results that are summarized above, it seems likely that the investigation of team mental models can reveal insights into the cognitive processes that help to facilitate the performance of design teams and to improve the communication between members of design teams.

6 Measurement of team mental models

The measurement of team mental models should reveal the degree of convergence among team members with regard to the content of known elements as well as the structure between elements (Mohammed et al. 2000). Although the components of mental models can be requested directly from knowledgeable participants, relevant content is generally supplied by the researcher in most existing team mental model studies (e.g. Edwards et al. 2006, Ellis 2006, Lim and Klein 2006, Mathieu et al. 2000). Because the adequacy of the stimuli provided to participants will determine the resulting mental model, the choice of content should be informed by first conducting a thorough team task analysis (Blickensderfer et al. 2000) to ensure adequate coverage.

The structure of a mental model reveals the relationships between elements in an individual's mind, which is often depicted graphically. Several commonly used team mental assessment tools are briefly reviewed below.

6.1 Pathfinder, multidimensional scaling and UCINET

With these techniques, mental model content is first elicited by means of relatedness judgements, which are collected by asking participants to provide quick, intuitive judgments of the similarity between pairs of concepts presented on computer or paper. Although relatedness could be due to several factors, including co-occurrence in time or causation, respondents use their own internal standards and are not generally asked to specify their definition of similarity (Mohammed et al. 2000). The relatedness judgements are then submitted to Pathfinder (PF; e.g. Edwards et al. 2006, Lim and Klein 2006, Stout et al. 1999), multidimensional scaling (MDS, e.g. Rentsch and Klimoski 2001), or UCINET (e.g. Mathieu et al. 2000, 2005) to reveal the structure of the data. These techniques produce a spatial representation of the relationship between concepts, in which concepts are represented as nodes and the relatedness of concepts is represented as links between nodes. In addition, PF, MDS, and UCINET produce a quantitative index of the similarity between network structures among team members.

6.2 Concept mapping

With concept mapping, participants are provided with a list of relevant concepts and asked to fill in the spaces in a blank map with concepts that represent different aspects of the task domain. Respondents are also asked to indicate which concepts best represent what their teammates are expected to be doing during a task. In this way, concept mapping merges both the taskwork and teamwork dimensions of mental models into a single technique (Marks et al. 2000, 2002). Similarity between maps is determined by the number of links held in common between group members (e.g. Ellis 2006).

Mohammed and colleagues (2000) emphasize the important conceptual work that must precede mental model measurement. In order to determine the best technique to utilize, researchers must carefully consider the purpose of the research, the team context, and the participants to be sampled. Because of the complexity and multidimensional nature of team mental models, researchers propose that multiple measures are necessary for thorough assessment (e.g. Kraiger and Wenzel 1997). However, because mental model measurement using even a single tool can be logistically difficult, collecting multiple measures has proven too burdensome for most researchers. Therefore, little is known concerning how the various measurement techniques compare to one another in terms of predicting team processes and performance.

Most of the team mental model research has been conducted utilizing command and control tasks in simulations (e.g. Edwards et al. 2006, Ellis 2006, Mathieu et al. 2000, 2005, Stout et al. 1999) or in field samples (e.g. Lim and Klein 2006, Mathieu et al. 2006, Smith-Jentsch et al. 2005, Waller et al. 2004). Little is known if these measurement techniques can also be applied to studying design teams. Given the uncertainty and dynamic nature of the creative process, it is unlikely that the content of mental models could be easily specified a priori and supplied to participants as has been done for performance-focused team samples. Therefore, it seems more useful to elicit the content of mental models from expert designers using qualitative techniques at appropriate times during the process. By requesting cognitive stimuli directly from participants, researchers can better capture the individual's idiosyncratic cognitive structure content. However, a major disadvantage of an elicited approach is that comparisons across cognitive structures may be difficult due to peculiar and/or idiosyncratic responses (Mohammed et al. 2000). Although supplied content tends to facilitate the comparison of structures across respondents, the danger is that respondents' cognitive structure may not be sampled adequately. As there are trade-offs associated with each type of mental model measurement tool, careful consideration must be given to evaluating techniques in light of the goals of the research.

6.3 Observations and communication analysis

Concerning the limitations of the two previously outlined techniques to measure the development and adaptation of mental models during creative processes, another method that might provide means to include just these issues is mentioned. Observing members of a team to obtain insights about how team mental models built up seems a promising approach to investigate how mental models develop. Although observations can only rely on overt behaviour such as verbal communication or sketching as indicators of the underlying cognitive structures, this approach pays attention to the fact that mental models are process-dependent. It may not be possible to determine the content as precisely as with the previously described techniques but as we have argued before, the content in itself is changing due to the nature of designing as dealing with ill-defined problems. In the absence of predefined criteria for appropriate mental models in design, observations of team communication can help to investigate how team mental models derive in the process.

A profound conceptualization of the content of mental models that leads to valid categorization schemes is necessary to bring up meaningful results. The major issues and characteristics of team mental models that are summarized in this paper can be used as a starting point to guide more observational research in this field.

7 Implications for studying design teams

What do these findings indicate for design teams? What is special about design teams? Several characteristics which distinguish design teams from operational teams have been mentioned before. Basically, the kind of problem design teams have to solve is different from the category of routine tasks in which the concept of mental models has evolved. Design tasks are usually much more uncertain. Often there is no definitive formulation of the problem and there is not the one best solution to the problem. The consequence is that team members have to develop a common model in order to use existing knowledge and to guide new information rather than following regular operations like the standard operating procedures in flight control. According to this model the team members will develop solution ideas, which refer to their different knowledge and assumptions. Combining these different views might result in a broader solution space, but these different models might also result in lots of misunderstanding because each team member refers to his/her own background. This can go even worse in project teams with members from different disciplines. Contrary to these assumptions cognitive diversity can contribute to stimulate creative synergy on the team level. However, in the last decades research on creativity has focused on the effects of the group and organizational context on individual creativity (Amabile 1995) rather than on teams producing creative outcomes. And research of design teams concentrated more on problems and failures of design teams coping with critical situations (Badke-Schaub and Frankenberger 1999).

Therefore, we expect that the analysis of the development of mental models in design teams reveals further insights into the process of coping with complex design problems of individuals and teams referring to aspects of team, process, task and competence.

Team: From what has been found in studies regarding mental models, it can be assumed that design teams should at least have a shared mental model about the team in which they are working, especially if team members have different ideas about the task at hand or they come with diverse backgrounds. This contributes to better coordination and communication within the team, resulting in better team processes. Although there are no studies about the effect of team mental models on the resulting artifact of design teams, it can at least be expected that teams will work more efficiently if they share their mental models about the team.

That is because they can rely on each others' knowledge, even if it divergent rather than redundant. Given the studies that directly relate the existence of team mental models to performance, this will probably hold for design teams as well since team members can use their resources and expertise better if they know how those are divided in their team.

Process: The same assumption holds for shared mental models of the process. Team members that agree on how to work together to solve a design problem and to structure the procedure (e.g. by applying certain methods) will very likely perform better than teams that do not. If their working steps and strategies they apply are shared they are able to work in a more complementary way. The presented framework of mental models in teams provides the ground to study how individual considerations of the design process and the way they are distributed in a team have an influence on the outcome. This understanding would provide valuable insights in how ideas about the design process should be shared in teams, which has implications for both education and practice.

Task: In how far mental models about the task should be similar for all members in a design team depends most likely on the design task at hand and on the situation within the particular context. If team members have to arrive at a common solution task mental models should be shared up to a certain level. In the example of the flight cabin design, all designers should have a shared mental model about the attachment and wiring connection possibilities of the lights when they have to make a decision what, where, and when to place them. On the other hand, when creativity is needed to come up with new solutions to a given design problem, diversity (and thus mental models) is needed. Diversity in itself has no value in terms of performance; there may be diverse knowledge, abilities and experience in a team, however, unless it does not enlarge the scope of the information process, the benefit will be rather limited. That is because without information exchange the team members' mental models about the problem, the goal and the solution cannot be shared.

Context: Designing typically takes always part in an organizational context, with relations to clients and users and a specific market situation. Thus, an analysis of mental models in teams needs to include relevant context knowledge that reflects the given situation.

Competence: So far, theoretical concepts of team mental models have concentrated on the task, the team and the tool aspect. We assume that a shared mental model of the teams' competence regarding the team's ability to perform the task is important for the team members' motivation to contribute to solving the task. Three different situations can be distinguished, each with different impact on motivation and probably on performance. A shared high estimation of the teams' competence will enhance the motivation of the team members to work in the team; correspondingly a shared low estimation of the teams' competence will decrease motivation and effort of the team members. If there is no shared mental model about the teams' competence the team climate is likely to suffer.

More research that relates types of mental models in teams to specific design phases would provide valuable insights in how individual mental models are influencing the team mental model and how the team mental model is influencing the team members' mental models. We argue that the concept of team mental models would provide an excellent framework for studying in detail how individual and psychological factors influence the solution and consensus finding in design teams.