Resolving uncertainties of the factor structures of the Coach-Athlete Relationship Questionnaire (CART-Q)

ABSTRACT

Objective: The 3+1Cs model and Coach-Athlete Relationship Questionnaire (CART-Q), measures the interpersonal constructs of Closeness, Commitment, Complementarity and Coorientation to appraise the nature of the relationship, yet the psychometric properties of the instrument still remain unclear. The purpose of this study was to resolve uncertainty about a suitable factor solution through applying previously used analytical methods as well as psychometric methods that are appropriate for this type of response scale, namely asymptotic distribution-free confirmatory factor analysis (CFA) and Rasch analysis.

Method: The 11-item direct and meta-perspective versions of the CART-Q were examined utilising AMOS, LISREL and Rach analysis using a sample of 251 national football and futsal coaches and athletes. A model comparison strategy with AMOS provided a direct comparison with previous studies. LISREL and Rasch analysis was conducted to provide additional information about the CART-Q psychometric properties.

Results: CFA with maximum likelihood and diagonally-weighted least squares estimation methods supported the factorial validity of the three-factor first-order model of both CART-Q versions, although a unidimensional solution was also tenable. Rasch analysis determined that the unidimensional model possessed higher reliability and provided improved coverage of participants’ scores, minimising ceiling effects.

Conclusion: Research designs that require assessment of coach-athlete relationships may prefer the unidimensional model due to its superior psychometric properties. When more detailed profile scores are required, the subscale scores can still be interpreted with adequate validity and reliability

KEY POINTS

What is already known about the topic:

1. The coach-athlete relationship is essential to enhancing performance and psychosocial well-being.

2. The 3+1Cs model and Coach-Athlete Relationship Questionnaire (CART-Q), represents the most popular approach to understanding coach-athlete relationships.

3. Previous studies investigating the CAR factorial validity have identified the 3+1Cs model as a multidimensional structure with universal applicability across cultures.

What this study adds:

1. This is the first study to perform CTT and IRT, to minimise the likelihood of psychometric findings materialising as an artefact of using a particular analytic tool.

2. Contrary to previous research, the results of this study suggest that a unidimensional structure obtained greater robustness in increased reliability, item coverage and reduced ceiling effects than the multidimensionality of the model.

3. These results suggest that a unidimensional factor solution is a more appropriate representation of the model than previously reported and provides researchers with confidence in the psychometric properties of the questionnaires.

KEYWORDS:

• 3+1Cs
• psychological properties
• New Zealand
• rasch analysis
• high-performance sport

Introduction

Sport is a contextually rich environment to study relationship interdependence, as many dyadic relationships are formed (Jowett & Ntoumanis, 2004). The coach-athlete relationship (CAR) is deemed pivotal in the success of the coaching process where both parties form close relationships with a high degree of interdependence (Jowett, 2009b; Jowett & Shanmugam, 2016). Exploratory research examining social interconnections between coaches and athletes have traditionally involved sports leadership models heavily focused on single-participant paradigms and the behaviour and personality traits of coaches. The past 20 years have seen rapid advances in CAR research, using a dyadic lens to gain a better understanding of the mutual interrelations of both individuals.

Interpersonal relationship research (Kelley et al. (1983) provided a framework for Jowett (Jowett & Cockerill, 2003; Jowett & Meek, 2000) to conceptualise the quality of the CAR as a situation in which coaches’ and athletes’ interpersonal emotions, cognitions and behaviours are mutually and causally interconnected. The CAR is entrenched in a series of changeable and multifaceted interactions where both members rely on each other during stressful situations and are connected members of a dyadic relationship. Jowett and Ntoumanis (2004) acknowledged that a tool was required to reflect the CAR psychometric properties. The CART-Q development and validation process included a systemised set of procedures to involve item generation and analysis with a panel of experts followed by a series of Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) on first-order and higher-order structured models (Jowett, 2007). EFA was used to postulate the structure of the model, then supported using CFA. CFA verified the three constructs of Closeness, Commitment and Complementarity of coaches’ and athletes’ emotions, cognitions and behaviours were observable and could be represented as a three-factor first-order model to represent the CAR multidimensional nature. Subsequent qualitative studies (Jowett & Frost, 2007; Trzaskoma-Bicsérdy et al., 2007) confirmed that this 3Cs model was appropriate. The model highlighted the extent to which the coaches and athletes perceptions are interconnected (Rhind & Jowett, 2010), emphasising the need to capture the dyadic nature of the two-person relationship (Jowett, 2006; Poczwardowski et al., 2006).

3Cs model constructs are defined as:

1. Closeness: the feeling of being emotionally close with one another, reflected in the feelings of being liked, cared and valued, and mutual trust had a positive effect on the coaches’ and athletes’ intrapersonal and interpersonal factors.

2. Commitment: a cognitive attachment and long-term orientation towards each-other.

3. Complementarity: roles, tasks, and support, reflecting the co-operative interactions of coaches’ and athletes’ responsiveness and affiliation.

Co-orientation is the fourth construct to advance the model to 3 + 1Cs (Jowett, 2007). It accentuated the extent to which coaches’ and athletes’ perceptions were interconnected, emphasising the importance of capturing the reciprocal nature of the dyadic relationship. Co-orientation suggested that coaches and athletes could perceive their relationship from two perspectives; the direct-perspective (e.g., I trust my coach/athlete), and the meta-perspective (e.g., My coach/athlete trusts me). Co-orientation assesses three dimensions of the CAR:

1. Actual similarity of coaches’ and athletes’ direct perspectives (i.e. the comparison of the coaches’ and athletes’ direct perspectives)

2. Assumed similarity (i.e. comparison of athletes’ direct perspectives with their meta-perspective and comparing coaches’ direct perspectives with their meta-perspective)

3. Empathetic understanding (i.e. the comparison of athletes’ direct perspective with the coaches’ meta-perspective or coaches’ direct perspective with the athletes’ meta-perspective) (Jowett, 2009a, 2006).

Co-orientation allowed for analysis to determine the level of agreement and disagreement between the dyadic partners, i.e. understanding the dyadic nature of the CAR (Woolliams, 2015). CAR is measured using the CART-Q (Jowett, 2009b; Jowett & Ntoumanis, 2004) involving both direct perspective and meta-perspective versions and has been cross-culturally validated (Balduck & Jowett, 2010; Yang & Jowett, 2012, 2013).

CART-Q psychometric qualities were previously assessed using Classical Test Theory (CTT) and CFA. Previous validation studies (Jowett & Ntoumanis, 2004), utilising structural equation modelling (SEM), determined that the three-factor first-order model is an appropriate tool to examine the quality of CAR. Findings supported the multidimensional nature of the relationship, underpinned by the dyadic member’s interpersonal emotions, cognitions and behaviour being mutually and causally bound. Figure 1 represents the 3Cs model of the direct perspective, and Figure 2 represents the 3Cs model for the meta-perspective employing CFA practices. This study proposes that while CTT has advanced SEM, more sensitive approaches, such as Rasch analysis need to be considered as suitable for ordinal Likert-scale data and provide detailed information about item performance.

Figure 1. 3Cs direct -perspective three-factor first-order factor model

Figure 2. 3Cs meta-perspective three-factor first-order factor model

Rasch analysis is a probabilistic model and is effective in determining the psychometric robustness of questionnaires (Lundgren & Tennant, 2011). Rasch (1960) demonstrated that the probability of an endorsement made by a person directed at a given item is a logistic function of the difference between the item parameter and the person parameter. A significant advantage of Rasch analysis over CTT is that it distinguishes between the items’ level of difficulty (the item parameter) on the latent construct and the person’s ability or propensity to endorse the item (the person parameter). The item parameters and person parameters are measured on the same scale (logits) and linearly represented demonstrating the standard error variance specific to item and person ability and difficulty (Schumacker & Smith, 2007). The Rasch model was initially developed to address dichotomous data and has been advanced through the development of the Partial Credit Model (Masters, 1982) and the Rating Scale Model (Andrich, 1978) to consider polytomous scored items. A fundamental consideration of both models is that they assume there will be variability evident in the distance between thresholds, yet the Rating Scale expects the variance to be equal across all items (Alagumalai et al., 2005). This advancement is pivotal, as polytomous Rasch analysis attends to Likert-style questionnaires allowing the model to determine whether the data in sequential categories reflects the increased property of the construct (Andrich, 1998). As the CART-Q adopts a Likert-scale ranging from 1 to 7, a person may have specific difficulty levels endorsing a particular response due to the hierarchical nature of the rating system (Kersten & Kayes, 2011).

This study utilised a systematic analysis path to complete a thorough and iterative psychometric examination of the 3 + 1Cs model and associated CART-Q to reduce the likelihood that results are an artefact of using a particular computational analysis tool. Unlike previous research on this topic, Rasch analysis was utilised post-traditional CFA methods to determine the validity and reliability of the direct perspective and meta-perspective versions of the CART-Q. The aim was to resolve uncertainty about a suitable factor solution through applying previously used analytical methods, psychometric methods that are appropriate for this type of response scale, i.e. asymptotic distribution-free CFA and Rasch analysis.

Method

Participants

A sample of 251 national New Zealand football and futsal players and national coaches (U20 men’s, U20 women’s, senior men’s and senior women’s) participated in this study (51.4% male and 48.6% female). All data were de-identified.

Instrumentation

The CART-Q direct and meta-perspective versions were used. Each 11-item questionnaire (Jowett, 2009b; Jowett & Ntoumanis, 2004) assessed the CAR for the constructs of direct-Closeness (4 items), direct-Commitment (3 items) and direct-Complementarity (4 items), and meta-Closeness (4 items), meta-Commitment (3 items) and meta-Complementarity (4 items). Items were measured on a 7-point Likert-scale ranging from 1 (“Strongly Disagree”) to 7 (“Strongly Agree”).

Procedure

The primary researcher met the participants to discuss the aims and logistics of the study, its voluntary nature, confidentiality and anonymity with individual athletes and coaches at training sessions. The information sheet and CART-Q were left with the consenting participants. The athletes and coaches were asked to place the questionnaires in an unmarked envelope and return to a collection box at a pre-determined venue convenient to the participants. The research study obtained approval from the authors’ institutional ethical committee.

Data analysis

Descriptive analyses were conducted using SPSS 25.0. Consistent with previous research (Balduck & Jowett, 2010; Balduck et al., 2011), the software package AMOS was utilised in this study for the CFA. A model comparison strategy, as suggested by MacCallum (1995), was adopted to test the CFA of the direct and meta-perspective 11-item CART-Qs. A widely accepted strategy to evaluate factorial validity of the CART-Q is to test comparative models to contribute further evidence to support a hypothesised model (Balduck & Jowett, 2010; Jowett, 2009a; Jowett & Ntoumanis, 2004; Yang & Jowett, 2010, 2013).

Three competing models (M1, M2 and M3) were compared to identify a structure that appropriately captured the CAR constructs for the direct perspective CART-Q, and three competing models (M4, M5 and M6) were compared for the meta-perspective:

1. M1 represented the original direct perspective three-factor first-order model as postulated by Jowett and Ntoumanis (2004).

2. M2 comprised of the direct perspective three-factor first-order model with minor modifications introduced to address the misspecification issues of ill-fit of two latent traits.

3. M3 represented a single-factor solution in which the 3Cs were considered as a stand-alone unidimensional construct.

4. M4 comprised of the original meta-perspective three-factor first-order model, as postulated by Jowett (2007), representing the constructs of meta-Closeness, meta-Commitment and meta-Complementarity.

5. M5 utilised a three-factor first-order model with minimal modifications to attend to variances outlined from modification indices.

6. M6 represented a single-factor solution in which the 3Cs were grouped as a unidimensional construct with meta-Closeness, meta-Commitment and meta-Complementarity considered as a stand-alone construct.

The sample size to free parameters ratios ranged from 8.7:1 to 11.4:1, with an average of 10:1 meeting the recommended 10:1 ratio (Bentler, 1995).

A series of goodness-of-fit indices were considered to evaluate the adequacy of model fit: scaled chi-square (χ²) statistic, Standardised Root Mean Square Residual (SRMR), Comparative Fit Index (CFI), Root-Mean-Square Error of Approximation (RMSEA), Tucker-Lewis Index (TLI), Akaike Information Criterion (AIC), and Expected Cross-Validation Index (ECVI). Chi-square difference tests were carried out to compare competing models. CFI and TLI values of 0.90 are somewhat acceptable, with 0.95 to 1.00 values indicating close to exact fit. SRMR value of ˂0.05 is acceptable, yet Hu and Bentler (1999) suggested a revised criterion of ˂0.08, and RMSEA values from 0.08 are considered as an acceptable cut-off value with 0.06 or higher indicating close to exact fit. As there are no identified satisfactory values for the AIC or the ECVI estimates, the lowest values, the most parsimonious, were considered to indicate a better fit when comparing the competing models. This strategy aligns with previous research protocols in testing the validity of CART-Qs (Ahmad, 2014; Balduck & Jowett, 2010). Upon determining the model fit to the data, convergent and discriminant validity of the direct and meta-perspective CART-Qs were then examined.

To enable direct comparison of the present results with those of previous studies (Balduck & Jowett, 2010; Balduck et al., 2011; Jowett & Ntoumanis, 2004; Yang & Jowett, 2010), CFA was conducted using maximum likelihood estimation with the software package AMOS v.22. Since the data are ordinal, due to the questionnaire adopting a Likert scale, CFA was also conducted using an asymptotic distribution-free method, which is deemed to be the appropriate approach for this type of data (Flora & Curran, 2004). Using the software package LISREL v.8.80 (Jöreskog & Sörbom, 1993), polychoric correlations were used with diagonally weighted least squares method of estimation.

Rasch analysis was conducted using the software package RUMM2030 (Andrich et al., 2009). Likelihood-ratio tests supported (χ²(29) = 50.64, p < 0.01, and χ²(29) = 66.26, p < 0.01, for the direct and meta versions, respectively) the appropriateness of adopting the Partial-Credit Model. An iterative process was facilitated following the guidance of the sequential steps recommended by Siegert et al. (2010) until Rasch analysis expectations were met. Individual item fit residuals should be nonsignificant and within a range of −2.50 to +2.50. Local dependency of items was examined by inspected residual correlation matrices.

Rasch analysis distinguishes between local trait dependency due to the presence of multidimensionality and local response dependency, which can occur when a response to an item is influenced by a response to another item, such as a method effect (Lundgren & Tennant, 2011). The type of dependency can be investigated by locally dependent items into subtests. If the resulting solution is unidimensional, subtests can be said to have successfully resolved local response dependency (Medvedev et al., 2018) and there is no evidence of local trait dependency.

Results

Descriptive statistics and reliability analysis

Table 1 illustrates the means, standard deviation, skewness and kurtosis scores of the CART-Q 11-item direct and 11-item meta-perspective versions. Negative univariate skewness scores were observed for all items except item 6 for both the direct perspective (item 6- “I am close to my athlete/coach”) and meta-perspective versions (item 6- “My coach/athlete is close to me”) indicating mostly positive relationship perceptions and meta-perceptions. The measures of Mardia’s multivariate kurtosis coefficients were high (67.40 and 51.59) for M2 and M5 respectively, thus rejecting the hypothesis of multivariate normality. Therefore, the robust maximum likelihood estimation method was utilised as this method offers moderately accurate standard errors when data shows multivariate non-normality (Bentler, 1995).

Table 1. Descriptive statistics for the 11-item direct perspective and meta-perspective CART-Qs

Item	Direct Perspective	M	SD	Skewness	Kurtosis
1	I like my athlete/coach	5.70	0.97	−0.58	0.13
2	I trust my athlete/coach	5.72	1.09	−0.78	0.33
3	I respect my athlete/coach	6.05	0.97	−1.14	1.70
4	I appreciate the sacrifices my athlete/coach has experienced to improve his/her performance	5.69	1.10	−0.62	−0.17
5	I am committed to my athlete/coach	5.70	1.10	−0.66	−0.07
6	I am close to my athlete/coach	4.42	1.23	0.01	−0.03
7	I think my coaching/playing career is promising with my athlete/coach	5.13	1.24	−0.05	−0.03
8	I am at ease with my athlete/coach	5.38	1.20	−0.70	0.30
9	I am responsive to my athlete/coach efforts	5.63	1.01	−0.40	−0.49
10	I am ready to do my best	6.11	0.89	−0.86	0.48
11	I adopt a friendly stance	5.97	0.82	−0.69	1.32
Item	Meta-Perspective
1	My athlete/coach likes me	5.23	1.04	−0.09	−0.17
2	My athlete/coach trusts me	5.33	1.05	−0.53	−0.05
3	My athlete/coach respects me	5.64	1.06	−0.90	0.96
4	My athlete/coach appreciates the sacrifices I have experienced to improve performance	5.10	1.18	−0.21	−0.07
5	My athlete/coach is committed to me	5.30	1.07	−0.09	−0.58
6	My athlete/coach is close to me	4.20	1.16	0.20	0.33
7	My athlete/coach believes that his/her career is promising with me	5.06	1.05	−0.36	0.24
8	My athlete/coach is at ease with me	5.17	1.13	−0.33	−0.05
9	My athlete/coach is responsive to my efforts	5.24	1.14	−0.34	−0.27
10	My athlete/coach is ready to do his/her best	6.05	0.91	−0.88	0.56
11	My athlete/coach adopts a friendly stance	5.80	1.01	−1.03	1.40

Model Evaluation via Confirmatory Factor Analysis

Fit indices (Table 2) demonstrate that M1, the original direct perspective version and M4, the original meta-perspective version of the CART-Qs do not meet the minimum recommended standard cut-off criteria (Bentler (1995); Hu and Bentler (1999). Examination of modification misspecification through modification indices (MI) was used to identify specific areas of ill-fit for potential respecification. This indicated that the covariance between item 1 (“I like my athlete/coach”) of the construct of Closeness, and item 6 (“I am close to my athlete/coach”) of Commitment and to a lesser extent, item 5 (“I am committed to my athlete/coach”) of Commitment and item 10 (“I am ready to do my best”) of Complementarity were areas for improvement. As the error covariance between the items is not unique compared to previous studies (Balduck et al., 2011) and considered substantive, respecification between the latent traits was necessary, resulting in M2 (direct version) and M5 (meta version). The respecified models with a correlated error between items 1 and 6 and items 5 and 10 were used in further analyses as M2 and M5 improved.

Table 2. Fit indices for the alternative three-factor higher-order and single-factor models for the direct and meta items using AMOS and LISREL (n = 251)

Model	df	χ^2	χ^2/df	CFI	TLI	SRMR	RMSEA (90% confidence interval)	AIC	ECVI
AMOS Direct Perspective
M1 3 Factor 1^st Order Original	41	277.82	6.78	0.89	0.84	0.06	0.14-0.17	327.83	1.31
M2 3 Factor 1^st Order Modified	37	165.18	4.46	0.93	0.90	0.05	0.10-0.14	223.18	0.89
M3 Single-Factor Solution	44	293.86	6.68	0.87	0.84	0.07	0.14-0.17	337.86	1.35
Meta-Perspective
M4 3 Factor 1^st Order Original	41	241.74	5.90	0.89	0.86	0.06	0.12-0.16	291.74	1.17
M5 3 Factor 1^st Order Modified	37	173.01	4.68	0.93	0.89	0.05	0.10-0.14	231.01	0.92
M6 Single-Factor Solution	44	257.74	5.86	0.89	0.86	0.07	0.12-0.16	301.74	1.20
LISREL
Direct Perspective
M1 3 Factor 1^st Order Original	41	168.96	4.12	0.98		0.07	0.09-0.13	218.96	0.88
M2 3 Factor 1^st Order Modified	40	132.50	3.30	0.98		0.06	0.08-0.11	184.25	0.74
M3 Single-Factor Solution	44	169.68	3.86	0.98		0.07	0.09-0.12	213.68	0.86
Meta-Perspective
M4 3 Factor 1^st Order Original	41	180.11	4.39	0.97		0.06	0.10-0.13	230.11	0.92
M5 3 Factor 1^st Order Modified	38	132.80	3.49	0.98		0.06	0.82-0.12	188.80	0.76
M6 Single-Factor Solution	44	181.21	4.19	0.97		0.07	0.10-0.13	225.21	0.90

Note. df = Degrees of Freedom, χ² = Chi-square, χ²/df = Chi-square divided by Degrees of Freedom, CFI = Comparative Fit Index, TFI = Tucker-Lewis Index, SRMR = Standardised Root Mean Square Residual, RMSEA = Root Mean Square Approximation of Error, AIC = Akaike Information criterion, ECVI = Expected Cross Validation Index.

Using AMOS, M1 and M3 of the direct perspective did not fit the data to an acceptable level as they had high corrected χ²/df ratios, low CFI, TFI and high RMSEA values. The asymptotic distribution-free fit from LISREL produced better fit indices when compared to AMOS; however, the findings demonstrated that M1 and M3 did not fit the data as well as M2. A comparison of the three direct models demonstrated that M2 had a superior fit. Chi-square difference tests between M1 and M2, and M2 and M3 added evidence to the superior fit of M2. AIC and ECVI values showed that M2 was more parsimonious than M1 and M3. RMSEA values were elevated and above the cut-off value of 0.060 for all models.

The same process was replicated for the meta-perspective CART-Q and displayed similar results. M5 demonstrated a superior fit over M4 and M6, due to M4 and M6 having higher corrected χ²/df ratios, lower CFI, TFI and higher RMSEA values. Chi-square difference tests between M4 and M5, and M5 and M6, added evidence to M5 having greater levels of superiority. M5 values for AIC and ECVI were lower than M4 and M6 demonstrating better fit.

Convergent and discriminant validity

Convergent validity of the best-fitting models (M2 for the direct perspective and M5 for the meta-perspective) examined whether each item had substantial loading to the hypothesised factors. All factor loadings for the direct perspective were moderate to high ranging from .53 (item 6 “I am close to my athlete/coach”) to .89 (item 3 “I respect my athlete/coach”). The factor loadings for the meta-perspective version ranged from .59 (item 10 “My athlete/coach is ready to do their best”) to .86 (item 3 “My athlete/coach respects me”). Additionally, convergent validity was determined from the variance estimate squared multiple correlation coefficient (R²). All the items recorded satisfactory R², only one item for Commitment recorded a value that was less than .50. Item 6 recorded an R² = .28 suggesting misfit, however, was retained to attend to the completeness of the scale.

Discriminant validity refers to the magnitude to which the 3Cs exhibit uniqueness (Li & Harmer, 1996) and was tested utilising Pearson’s correlation coefficient. Strong associations were identified between the 3Cs of both the direct and meta-perspective versions. M2 analysis showed relationships in direct-Closeness and direct-Complementarity (r = 0.92), direct-Closeness and direct-Commitment (r = 0.92) and direct-Commitment and direct-Complementarity (r = 0.85). M5 analysis showed relationships in meta-Closeness and meta-Complementarity (r = 0.92), meta-Closeness and meta-Commitment (r = 0.93), and meta-Commitment and meta-Complementarity (r = 0.93). The results suggest that the 3Cs can be perfectly correlated and considered unidimensional, therefore questioning the appropriateness of the three-factor first-order model.

Model evaluation via rasch analysis

Rasch analysis was conducted to provide additional information about the CART-Q psychometric properties, to explore the extent the scale may be treated as a single-factor model where Closeness, Commitment, and Complementarity are described as item clusters with shared variance due to local response dependency. A model was tested where these sub-scale items were entered as subtests. The first iteration of this model already provided an adequate fit (χ²(15) = 12.14, p > .05). Person Separation Index (PSI), a measure of internal consistency reliability, was excellent (0.90) thus permitting reliable measurement within participants. Item coverage was adequate, as approximately 2% of the participants were not captured by the scale (Figure 3). However, scores were negatively skewed, with the mean score located at 3.06 on the logit scale.

Figure 3. Person-item threshold distribution for the CART-Q when constructs were entered as subtests for the direct perspective data

Fitting the Rasch model to the individual scale items only resulted in nonsignificant, thus adequate fits for all three constructs. Skewness was extreme for Closeness and Complementarity (Figure 4). In both cases, over 5% of the participants’ traits could not be assessed by the items, which presents inadequate coverage.

Figure 4. Person-item threshold distribution for the CART-Q when analyses were conducted separately for each construct using the direct perspective. The top panel shows results for Closeness, the middle panel Commitment, and bottom panel Complementarity

PSI for Closeness, Commitment, and Complementarity was 0.87, 0.82, and 0.82, respectively. While these values were lower than for the single-factor model, they still indicated adequate fit.

Overall, the results indicated that both the single-factor model, as well as individual construct scores, can be used with sufficiently robust psychometric properties. Given the issues with ceiling effects and item coverage for Closeness and Complementarity, the single-factor model appears preferable, with increased reliability. The same findings were noted for the meta-perspective.

Discussion

The purpose of this study was to examine the psychometric properties of the CART-Q, utilising a systematic analysis path as a mechanism to resolve previous uncertainty about a suitable factor solution. The validity and reliability of the CART-Q direct and meta-perspective versions were examined, with national football and futsal coaches and athletes in the New Zealand context.

Previous studies investigating the factorial validity of the CAR have identified the 3Cs (Jowett, 2003; Jowett & Cockerill, 2003; Jowett & Meek, 2000) and 3 + 1Cs (Jowett, 2007) models as multidimensional structures. CFA has previously been adopted as a preferred method of data analysis and although researchers have identified covariances between the errors of commonly identified items across subscales, the three-factor first-order and high-order structures have been reported as universal applicability across cultures (Ahmad, 2014; Yang & Jowett, 2010, 2012).

The methodological procedures used in this study to examine the factorial validity and reliability of the direct and meta-perspective versions of the CART-Q have replicated most procedures with previous studies. CFA was initially utilised using AMOS, followed by a model comparison strategy. However, to minimise the likelihood of psychometric findings materialising as an artefact of using a particular analytic tool, LISREL and Rasch analysis were then adopted. The rationale for utilising Rasch analysis was to advance the psychometric properties of the 3 + 1Cs model and CART-Q as it has unique advantages over CFA to determine the validity, reliability and item difficulty of self-reporting tools that adopt a Likert-scale. This is the first study to perform CTT and IRT, highlighting the contribution to scientific knowledge to examine the psychometric properties of the CART-Q.

The analysis indicated that the three-factor first-order structure demonstrated adequate fit. Nevertheless, issues with ceiling effects and item coverage for the constructs Closeness and Complementarity were observed. Contradicting previous findings, a unidimensional structure obtained greater robustness than the multidimensionality of the model. Therefore, a unidimensional factor solution may be a more appropriate representation of the model than previously reported and provides researchers with confidence in the CART-Q psychometric properties, as earlier discrepancies have been resolved through the adoption of Rasch analysis.

The CART-Q direct and meta-perspective versions adequately measure the quality of the CAR. This study supports previous findings with a strong positive correlation between the constructs of Closeness, Commitment, and Complementarity, with notable error covariances evident concerning item 1 (“I like my athlete/coach”) and item 6 (“I am close to my athlete/coach”) to traverse the constructs of Closeness and Commitment. Rhind et al. (2012) while investigating the fundamental differences in how athletes from individual and team sports viewed the nature of their relationship with their coach, established that the 3Cs correlate more strongly in team sports than individual sports. Due to team-athletes being more inter-related than individual-athletes. Secondly, Balduck et al. (2011) identified items 1 and 6 to be problematic due to error covariances in a sample of Belgian athletes. Balduck et al. postulated that the connection between the two items might have been due to the participants interpreting the questions as being similar in meaning, suggesting the participants had difficulty discerning the difference between the “emotions of being close” and the “cognitions of being close”. Cultural nuance was rationalised as a potential reason for this misinterpretation; however, it appears that this finding may not be not unique to the Belgian cultural context as it was also found in our New Zealand sample.

Rather than cultural nuances, we suggest the environmental factors in which the coaching process takes place to be a plausible explanation for the finding that error variances of items 1 and 6 are correlated. Athletes in team sports need to be personally and instrumentally interdependent. In team sports, athletes have less opportunity to interact with their coach having to compete with other athletes for face-to-face interpersonal transactions, and coaches are challenged with dividing their time equally to interact with the athletes personally (Woolliams, 2015). A typical New Zealand national football team would travel with a squad of 23 athletes to FIFA sanctioned events with approximately two to four opportunities to tour per annum. A typical New Zealand national futsal team would travel with a squad of 14 athletes to FIFA sanctioned events with one to two opportunities per annum. The lack of events to assemble athletes is problematic, limiting the opportunity for coaches and athletes to communicate face-to-face. The scarcity of opportunities for the athletes and coaches to communicate supports previous research conducted by Rhind et al. (2012) determined athletes from individual sports reported being closer and more committed to their coach than those athletes in team sports.

Nevertheless, no significant differences were identified concerning Complementarity, noting that both individual and team athletes feel, think and behave similarly with regards to being responsive, friendly and ready to do their best. Individual sport athletes reported higher levels of coach trust, respect and appreciation for them as athletes than team athletes. This lack of opportunity for coaches and athletes to communicate one-to-one and face-to-face may be a contributing factor for athletes and coaches in team sports to experience a delayed onset of establishing close relationships, as it takes longer for the dyadic members to navigate and develop their emotions and cognitions towards their dyadic partner. Potential strategies to minimise this delayed onset of establishing close relationships and misunderstanding between the emotions and cognitions of being close could be overcome through the implementation of structured education programmes (Bennie & O’Connor, 2012). Athletes and coaches would benefit from having more in-depth knowledge of the differences between feelings and thoughts and the positive benefits of establishing high-quality relationships using the 3 + 1Cs as a best-practice model. This could lead to a greater awareness of the importance of the CAR, how to better navigate the development of high-quality relationships, and that increased levels of one-to-one communication are an essential factor.

The results exhibited a ceiling effect in the way in which the maximum score of 7 (Strongly Agree) on the Likert-scale was represented in the data. Further research to investigate adopting a scale of 1-10 rather than 1-7 may reduce the probability of a ceiling effect. Caution should be applied to this line of thinking as it will direct the participants into not having a neutral midpoint (4- neutral midpoint) as one can choose on a scale of 1-7. A scale of 1-10 will direct the participants to express either a negative or positive opinion.

Limitations

The following limitations need to be acknowledged. Firstly, the sample size used in this study was a limitation to analyse data using CFA. Even though the sample size per free parameters ratio was met, small sample size affects Chi-square statistics and RMSEA tends to over-reject true population models (Hu & Bentler, 1999) which is evident in existing CART-Q research (Balduck & Jowett, 2010; Jowett & Ntoumanis, 2004). Secondly, the sample may not allow for generalisability of the results due to the specific environmental context in which the coach-athlete relationships resides. Thirdly, it is worth noting that the coaches’ voice has not been given equal standing to the athletes’ as the national football and futsal teams require a squad of athletes and fewer coaches. Finally, further research is recommended, using Rasch analysis to examine the 3 + 1Cs model with a more comprehensive, diverse range of high-performing national teams and individual sports. This would increase the precision of the validity and reliability of both CART-Q versions to be incorporated into the CAR literature. Additionally, future research investigating how participants interpret the questions is required. There may be the need for a guide to be produced with an explanation of the questions for the participants to gain a better understanding of the meaning of each question to decrease the possibility of any misinterpretation.

Conclusion

In sum, the findings suggested that although the three-factor first-order structure demonstrated adequate fit, concerns with ceiling effects and item coverage for the constructs Closeness and Complementarity were observed. Based on the findings of this study, it is evident that a unidimensional structure obtained greater robustness than the multidimensionality of the model. Rasch analysis has highlighted that a unidimensional factor solution may be a more appropriate representation of the model than previously reported and provides researchers with confidence in the CART-Q psychometric properties.

Disclosure statement

No potential conflict of interest was reported by the authors.