The multicollinearity between youth sport environment questionnaire and team assessment diagnostic measurement in sport settings

Abstract

With interdisciplinary effort, shared mental model from organizational psychology has been introduced in recent years. Even though the concept of shared mental model is established in sport psychology, it still has an operational problem. That is, different researchers have used different measures. The purpose of this research was to examine the multicollinearity between the Team Assessment Diagnostic Measurement (TADM) questionnaire, which measures shared mental model, and the Youth Sport Environment Questionnaire (YSEQ), which measures group cohesion. The participants were competitive youth soccer players. TADM and YSEQ were measured at the end of the season. Findings showed that the TADM was highly correlated with task cohesion (r = 0.81) even though VIF did not indicate multicollinearity. Therefore, TADM should be used with caution. Also, based on the definition of the shared mental model, Pathfinder or card sorting is recommended rather than using questionnaires.

Keywords:

• shared mental model
• cohesion
• multicollinearity
• team sports

PUBLIC INTEREST STATEMENT

In sport psychology, the concept of shared mental models has been introduced from organizational psychology. Shared mental model is, simply saying, the knowledge that teammates share in a team, and previous research suggested shared mental models lead to team coordination. However, this concept seems to have an operational problem. That is, the questionnaire to measure shared mental models is similar to the questionnaire to measure cohesion in a team. Therefore, these two questionnaires might measure the same variable. Even though VIF did not indicate the two questionnaires measure the same variable, correlation analysis indicated they are the same variable. Therefore, researchers should use the questionnaire with caution. In fact, a questionnaire can only measure the structure of knowledge indirectly. Therefore, different methods are recommended such as pathfinder or card sorting to measure shared mental model directly even though the importance of shared mental models should be still emphasized.

1. Introduction

In interactive sports, coordination between players is necessary to outperform the oppositions. If the coordination between players is smooth, an opposition is difficult to predict the next play, which makes difficult to defend. As a result, high quality of coordination has the potential to dominate a game in interactive sports.

To coordinate well, shared understanding of tactics is essential because of nature of interactive sports. In interactive sports, players do not have much time to communicate with each other because a game does not stop unless a referee stops the play. In this time-pressured situation, shared understanding of tactics is crucial to improve coordination as shared understanding of tactics enables the implicit prediction of information, resource requirements of teammates, and actions based on the understanding of the task (Mathieu et al., 2000). In fact, Eccles and Tenenbaum (2004) suggested that shared knowledge is necessary to show high quality of coordination in a game. Therefore, shared understanding of tactics creates high quality of coordination in a team.

To investigate shared understanding of tactics, a concept of shared mental model has been introduced from organizational and industrial psychology to sport psychology (Eccles & Tenenbaum, 2004; Filho et al., 2015; Giske et al., 2015, 2017; Webber et al., 2000). Shared mental model is defined as a “team members’ shared, organized, understanding and mental representation of knowledge about key elements of the team’s relevant environment” (Mohammed & Dumville, 2001, p. 90). According to Mohammed et al. (2010), shared mental model has four components: an equipment model (i.e., knowledge about tools and technology), a task model (i.e., understanding of work procedures, strategies, and contingency plans), a team interaction model (i.e., awareness of members’ responsibilities, role interdependencies, and communication patterns), and a team model (i.e., understanding of teammates’ preferences, skills, and habits). Also, Mathieu et al. (2000) suggested that the equipment model and task model could be called as the task shared mental model while a team interaction model and team model could be called a team shared mental model. A task shared mental model refers to how the task is accomplished (i.e., task strategies, contingencies or problems, or dealing with environmental conditions), while team shared mental model is the understanding of roles and responsibilities in a team as well as communication channels and teammates’ knowledge, skills, and tendencies.

There are two aspects of shared mental models to measure: 1) similarity and 2) accuracy of knowledge (Mohammed et al., 2010). Similarity is the agreement of team member knowledge while accuracy refers to the quality of mental models (Cannon-Bowers & Salas, 2001). To show implicit coordination, the degree to which members’ mental models are consistent or converge with one another is important. It is also important that the shared mental model is accurate. For example, in the situation where high similarity and low accuracy of shared mental model, teammates’ understanding of task priorities is shared, but the understanding itself is wrong. In this case, there are few conflicts regarding team process. However, the work is neither effective nor efficient (Lim & Klein, 2006). Therefore, similarity and accuracy of shared mental models should be measured. Considering the interdependence of team sports, high similarity of SMM in a team is essential for successful team performance (Eccles & Tenenbaum, 2007). However, measuring or determining the accuracy of team mental models is relatively vague in sport situations, because there is no absolute or ultimate manner to win a game.

If teammates have shared metal model, they have common expectations and can better coordinate actions, facilitate information processing, and allow them to adapt their behaviors to the demands of the task to improve performance (Edwards et al., 2006). In fact, the relationship between shared mental model and performance has been demonstrated in a variety of settings including the military (Gurtner et al., 2007; Lim & Klein, 2006), business (Fisher et al., 2012; Santos & Passos, 2013), and computer-generated simulation games (Edwards et al., 2006; Mathieu et al., 2000, 2010). In fact, Fisher et al. (2012) revealed implicit coordination mediated the relationship between shared mental model and performance in a business simulation. Similarity of shared mental model was positively predictive of implicit coordination which, in turn, predicted team performance. Thus, shared mental model helps group members to coordinate better.

In sport settings, shared mental models are related to important variables in sport psychology. One of the important variables is performance outcome. Shared mental model seems to be related to performance outcome as well as in different fields such as business and military. Webber et al. (2000) examined the relationship between shared mental model and performance outcome (i.e., winning percentage and the points the teams earned) by using community basketball teams. They used scenario-based questionnaire. The participants were asked to answer what kind of actions they would take in the hypothetical scenario. As a result, the similarity of shared mental models positively predicted performance outcome. Thus, even in sport settings, there is an evidence that shared mental model can positively impact team performance.

1.1. Operational problem

Although research on shared mental models in sport settings has increased gradually and the importance of shared mental models is growing, different researchers have measured shared mental model by different methods (e.g., Filho et al., 2015; Giske et al., 2017, 2015; Webber et al., 2000). As a result, there has not been any consensus within the sport psychology research community as to the development of a “gold standard” to measure shared mental model in sports settings. For example, as mentioned, Webber et al. (2000) used a scenario-based questionnaire to investigate the relationship of shared mental model and performance outcome among community basketball players. On the other hand, Giske et al. (2015) developed a questionnaire called the Shared Mental Model Scale for Sport to examine the relationship between shared mental models and role clarity in elite ice hockey and handball teams. Besides, Giske et al. (2017) also used the Shared Mental Model Scale for Sport to examine the relationship between shared mental model, role clarity, team identification, and social loafing. Also, Johnson et al. (2007) developed a questionnaire to measure general team-related knowledge, called Team Assessment Diagnostic Measurement (TADM). Filho et al. (2015) used the TADM to examine the relationships between cohesion, collective efficacy, shared mental models, and performance. Filho et al. (2014) also used the TADM to find attributes which discriminate successful and unsuccessful players. As each researcher used different measurements for shared mental models, there is no consensus to measure shared mental models in sport settings.

These questionnaires have two issues. One of the big issues is that these questionnaires can not measure the accuracy or similarity of shared mental model. Instead, they measure the belief or perception of their own shared mental model. This is the fundamental drawback of a questionnaire to measure shared mental model. Secondarily, some questionnaires, specifically the TADM, may measure different variable other than shared mental model. In fact, some of these items seem to measure cohesion instead of shared mental model. Cohesion is defined as “a dynamic process that is reflected in the tendency for a group to stick together and remain united in the pursuit of its instrumental objectives and/or for the satisfaction of member affective needs” (Carron et al., 1998, p. 213). Operationally, Youth Sport Environment Questionnaire (YSEQ) is used to measure cohesion at youth level. Comparing TADM with YSEQ, some items seem to measure the same variable. For example, the item of “My team usually discusses our goal and attains the agreement of each other” in the TADM is similar to “We all share the same commitment to our team’s goals” in the Youth Sport Environment Questionnaire (YSEQ), which is a questionnaire to measure cohesion in a team at youth level. Therefore, these two questionnaires may have multicollinearity problem. Particularly, TADM might measure cohesion instead of shared mental models as a questionnaire can only measure belief or perception, not mental representation of knowledge. If this is true, previous studies might have revealed the relationship between cohesion and other variables instead of shared mental models. Methodologically, this is impactful because the TADM has been already used in a variety of fields (Johnson & Lee, 2008; Johnson et al., 2011).

In summary, shared mental model, introduced to sport settings within the past five years, has been found to be related to team coordination and team performance. Conceptually, shared mental model is well-defined. However, there is an operational problem in sport settings. That is, the well-used questionnaire may measure cohesion instead of shared mental model, which indicates multicollinearity. However, as long as the author knows, no research examined the operational perspective of SMM in sport settings. If the researcher uses a measurement without validity confirmation, the results might be distorted. Therefore, the purpose of this study was to examine the operational problem of the construct of shared mental model. Specifically, this study examined the multicollinearity between the TADM, which is a measurement for shared mental model, and the YSEQ, which is a measurement for cohesion. It is hypothesized that the TADM and the YSEQ have multicollinearity, which implies that both two questionnaires measure the same variable. This study contributes to find another way to measure SMM.

2. Methods

2.1. Participants

The researcher contacted clubs belonging to the Calgary Minor Soccer Association, which is a boys’ and girls’ youth competitive soccer league in Calgary, Alberta, Canada. Two clubs agreed to participate in the research study. From these two clubs, data were collected from a total of six boys teams (n = 89) in U-15 (boys: N = 5) and U-17 (boys: N = 1).

2.2. Measures

2.2.1. Cohesion

Cohesion was measured by Youth Sport Environment Questionnaire (YSEQ). Eys et al. (2009) created an age-specific questionnaire for thirteen to seventeen years old due to lack of validity and reliability of group environment questionnaire which was developed by A. v. Carron et al. (1985). The YSEQ has a total of eighteen items measuring two aspects of cohesion: task cohesion and social cohesion. Task cohesion and social cohesion have eight items respectively (e.g., task cohesion: We all share the same commitment to our team’s goals, social cohesion: I invite teammates to do things with me). Additionally, two items are spurious negative items. The YSEQ uses a Likert scale from 1 (strongly disagree) to 9 (strongly agree). Average scores of task cohesion, social cohesion, and overall cohesion were calculated. Eys et al. (2009) showed reliability of YSEQ with reported Cronbach alpha of 0.89 for task cohesion and 0.94 for social cohesion.

2.2.2. Shared mental model

The presence of shared mental model was measured by the Team Assessment Diagnostic Measurement (TADM) (Johnson et al., 2007). Sikorski (2009) created a short version of it and Chronbach alpha was more than 0.70 (Filho et al., 2015). The questionnaire has 15 items and a Likert scale from 1 (strongly disagree) to 5 (strongly agree). The model consists of five subscales which are general task and team knowledge (GTTK, three items; e.g., “My team knows specific strategies for completing various goals”), general task and communication skills (GTC, three items; e.g., “My team consistently demonstrates effective listening skills”), attitude toward teammates and task (ATTK, three items; e.g., “My team likes to do various team tasks”), team dynamics and interactions (TDI, three items; e.g., “My team understands their roles and responsibilities for doing various team works.”), and team resources and working environment (TRWE, three items; e.g., There is an atmosphere of trust in my team.”). Overall shared mental model and subscales were calculated by the average of the items. The TADM has shown content validity and reliability with Cronbach alpha for each subscale reported as 0.76 for GTTK, 0.89 for GTC, 0.75 for ATTK, 0.81 for TDI, and 0.85 for TRWE (Johnson et al., 2007).

2.3. Procedure

Permission to conduct this study was obtained from the Conjoint Health Research Ethics Board from a large Western Canadian University. Invitation letters to participate in the study were then sent to the technical directors of a number of soccer clubs which have U13, U15, and U17 teams. As a result of this invitation, two clubs agreed to participate in the study. To collect the data, the researcher either went to the field before or after the practice. Time to complete the questionnaires was approximately 10–15 minutes. Data were collected at the end of the season to ensure that group processes had already been in operation for a period of time (i.e., teams had been together and practicing for the season). The presence of group stability and regular member interaction was thought to allow necessary time for the development of both group cohesion (Widmeyer et al., 1993).

2.4. Data analysis

Demographic data were collected and the average age, tenure on the team, and years of soccer experience were calculated. Reliability analysis (i.e., Cronbach alpha) was calculated for both questionnaires (YSEQ and TADM).

A correlational analysis, VIF and tolerance analyses were conducted between TADM and YSEQ to examine the multicollinearity. Above 0.8 in correlation indicates multicollinearity (Rockwell, 1975). If VIF is more than 5, the variables might have multicollinearity (Sheather, 2009).

2.5. Demographic analysis

2.5.1. Participants’ soccer experience and profile

The age of the participants ranged from 14 to 16 (Mean = 14.51, SD = 0.88). Mean length of tenure on the team was 1.79 (SD = 1.39). Mean years of soccer experience were 8.99 (SD = 2.52).

2.5.2. Psychometric analysis

Cronbach’s alpha for each subscale of each questionnaire was calculated. For YSEQ, the Cronbach’s alpha for social cohesion was 0.95 and for task cohesion was 0.92. For TADM, the Cronbach alpha was 0.69, 0.78, 0.74, 0.73, and 0.72 for GTTK, GTC, ATTK, TDI, and TRWE respectively.

2.5.3. Correlation analysis

To analyze any potential multicollinearity between TMM and cohesion, Pearson correlation analysis was conducted. All correlation is seen in Table 1. Overall shared mental model was significantly correlated with overall cohesion (r = 0.76, p < 0.01) and social cohesion (r = 0.59, p < 0.01). More importantly, shared mental model was significantly and strongly correlated with task cohesion (r = 0.81, p < 0.01). Also, VIF was 2.86.

Table 1. Correlation between YSEQ and TADM

	1	2	3	4	5	6	7	8	9
1.TC
2.SC	.56**
3.Co	.82**	.93**
4.GTTK	.63**	.44**	.59**
5. GTC	.62**	.41**	.56**	.63**
6.ATTK	.71**	.54**	.67**	.56**	.58**
7.TDI	.68**	.54**	.67**	.59**	.54**	.70**
8.TRWE	.65**	.46**	.60**	.56**	.48**	.60**	.63**
9.SMM	.81**	.59**	.76**	.80**	.79**	.86**	.84**	.81**

TC = Task Cohesion; SC = Social Cohesion; Co = Cohesion; GTTK = General Task and Team Knowledge; GTC = General Task and Communication skills; ATTK = Attitude toward Teammates and Task; TDI = Team Dynamics and Interactions; TRWE = Team Resources and Working Environment; TMM = Shared Mental Model ** p < 0.01.

3. Discussion

In the last decade, the concept of shared mental model has been introduced into the sport psychology literature (Eccles & Tenenbaum, 2004). However, shared mental model has not got the consensus operationally. That is, various researchers have used various measures for shared mental model such as Shared Mental Model Scale for Sport or scenario-based questionnaire (Giske et al., 2015, 2017; Webber et al., 2000). Of the measures, TADM was created in the field of organizational psychology, and Filho et al. (2014) and Filho et al. (2015) used TADM in sport settings. However, some items of the TADM seem very similar to the already well-established construct of cohesion. Therefore, this study examined the multicollinearity between YSEQ and TADM.

The result showed that the TADM reached 0.8 which indicates multicollinearity while VIF did not indicate the existence of multicollinearity. Therefore, the hypothesis was partially supported. Therefore, the TADM should be used with caution. Task cohesion is defined as task perspective of unity (A.V. Carron et al., 1998) and some items of the TADM follows the definition of task cohesion such as “My team knows specific strategies for completing various goals.” or “My team takes our pride in work.” These similar items might lead to the high correlation. Even though the VIF did not show multicollinearity, we consider using this questionnaire because it is risky to use a questionnaire which does not have robust validity.

Overall, it may be difficult to measure shared mental model using only quantitative questionnaires as shared mental model represents the structure of knowledge, and questionnaires can only measure the belief or perception of players. Therefore, it is not avoidable to measure a variable like team cohesion. In fact, De Church and Mesmer-Magnus (2010) showed that the effect size of the relationship between shared mental models and performance changed depending on how shared mental models were measured. Thus, different ways to measure shared mental model in sport should be developed.

Also, researchers should be cautious with using the TADM in sport settings. The TADM was made in the field of organizational psychology and the target of the participants is business people, not players in interactive sports. Therefore, some items might be confusing for sport players to answer. Therefore, when researchers use the TADM by any chance, the items should be closely looked at, and if the items do not make sense to the players, researchers should modify some parts of the questionnaire.

3.1. Recommendations to measure shared mental model

Card sorting and pathfinder are suggested methodologies in other research not related to sports (Langan-Fox et al., 2000). In the card sorting, participants write down all the concepts that are relevant to the theme. Then, the concepts are sorted depending on their similarity. After sorting, the participants receive interviews that are recorded and transcribed. Clusters of cards are photographed as a final representation of knowledge. This technique is possible to be applied to sport settings. For example, soccer players are told to write down all the concepts they can come up with about offense strategy in the team such as cross-ball, counter-attacking, and long ball. Then, they can sort out the concepts based on the similarity and create clusters of cards as the final representation of knowledge. Also, Pathfinder is another way to measure shared mental model. In this technique, participants judge the relatedness of all concepts, and concepts are linked based on their perceived relatedness. This technique is also applicable to sport settings. For example, researchers or head coaches can create defense concepts which are used in the sport such as retreating, high pressure, and 1v1 defense beforehand. Then, players can rate how much each concept is related to each other from 1 (not related at all) to 5 (very related). Base on the ratings, computerized network technique can visualize the concept network. Then, the similarity between players’ networks can be obtained as well.

One of advantages of card sorting and pathfinder is that researchers can directly see the structure of knowledge for each participant (Langan-Fox et al., 2000). These methods allow the direct visualization of the structure of knowledge while TADM or any other questionnaires can only measure shared mental model indirectly as they measure their belief or perceptions of shared mental model. Moreover, pathfinder can calculate the similarity of each mental model in a team by algorism. In this sense, team-level comparison is enabled. That is, similarity of mental models between multiple teams can be compared. Considering that the similarity of shared mental models is essential to coordinate with each other in interactive sports, these two methods are recommended to use.

Even though card sorting and pathfinder are strongly recommended, scenario-based questionnaire might be still an option to measure shared mental model. For example, a researcher creates attacking scenario and the participants are asked what kind of action they should take in the scenario. Data analysis can show how much the action is shared among the teammates. This is still an indirect way to measure shared mental model. However, this can measure the decision-making which is the result of the activation of mental model.

3.1.1. Limitations and suggestions for future research

The sample was youth boy soccer players. Therefore, it cannot be generalized to adults level or girls. To examine validity of a questionnaire is never-ending process. Therefore, future research should conduct the same study to the different samples and confirm if the result can be replicated. There is still possibility that there is no correlation between the two questionnaires for adult or girls.

Also, as mentioned before, a questionnaire can only measure a belief or perception. Therefore, other questionnaires for shared mental model like the Shared Mental Model Scale for Sport should be checked for validity. It is possible that questionnaires which are supposed to measure shared mental model actually measures different variables.

4. Conclusion

Shared mental model has been introduced in sport psychology and more and more researchers use the concept for performance enhancement. However, as the TADM is highly correlated with task cohesion, it would be prudent for future researchers to establish the way to measure shared mental model to solve the operational problem. As indicated previously, pathfinder and card sorting have been used in different research areas and may be methodologies that could be used to further explore.

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Yuto Yasuda

Yuto Yasuda is a sport psychology researcher. His main interests are group dynamics in sport settings such as cohesion, collective efficacy, and shared mental models and the ways to increase team performance. Also, he has worked as a soccer coach and sport psychologist and try to make a bridge between research and applied fields. He has a soccer coaching license in Japan and try to get a certificate as a sport psychologist in Canada as well.

David Paskevich

David Paskevich is Associate Dean (Academic), Faculty of Kinesiology, University of Calgary, and holds a PhD in Kinesiology from the University of Waterloo. Interests center upon the integration of the science-practitioner model, bringing the science of sport psychology into practical/applied settings. His research combines a variety of areas within the realm of sport (using both quantitative & qualitative methodology), and examines important theoretical and practical questions related to athletes, coaches, and officials.