TGMD-3 short version: Evidence of validity and associations with sex in Irish children

ABSTRACT

This study examined the internal structure and evidence of validity of the Test of Gross Motor Development 3^rd edition (TGMD-3) in primary school aged children. Participants (n = 1608, 47% girls, age range 5–11 years, mean age 9.2 ± 2.04) were recruited from Irish schools across twelve counties (56% rural, 44% urban). The TGMD-3 was used to measure FMS proficiency (Ulrich, 2020). A two-factor model (13 skills) was used and confirmatory indexes were calculated. The Bayesian criteria and the Composite Reliability were employed to evaluate alternative models. Relationships between the final model proposed with age, sex and BMI were calculated using a network analysis. Mplus 8.0 and Rstudio were used. A two-factor model (locomotion and object control) with adequate values (> 0.30) for the seven skills (gallop, hop, jump, two-hand strike, bounce, catch, overhand throw) presented excellent indexes. The skills with the highest indicator of strength centrality in the network were bounce and catch for both boys and girls and hop for boys and horizontal jump for girls. This study evidences the validity and reliability of the internal structure of the TGMD-3 and demonstrates that a short version of the TGMD-3, comprising seven skills is a valid measure of FMS in this population.

KEYWORDS:

• Motor competence
• fundamental movement skills
• Skill
• confirmatory factor analysis

Introduction

The development of fundamental movement skills (FMS) is recognised as providing a foundation for more advanced, complex movements needed to successfully participate in sports and physical activity (PA) across the lifespan (Gallahue et al., 2012). Evidence suggests that children and adolescents who are more proficient in FMS are more likely to participate in different forms of PA during life (Lloyd et al., 2014; Stodden et al., 2008; Utesch et al., 2018). There is however concern that FMS proficiency is low in children and adoelscents (Behan et al., 2019; O’ Brien et al., 2016a; Duncan et al., 2019; Philpott et al., 2020). As the development of FMS are a key objective of physical education curricula worldwide (Australian Curriculum, Assessment and Reporting Authority, 2012; Department for Education, 2013; Society of Health and Physical Educators SHAPE America, 2013), the monitoring of FMS in schools has become more common, either in assessing current levels of FMS proficiency of a cohort, or for tracking changes in FMS due to intervention.

Assessment of FMS is therefore an important consideration for researchers in the field. There is a plethora of tools that can be used to assess FMS, depending on a number of factors including the type of information needed, the time and resources available to administer the assessment, and the population of interest (See Eddy et al., 2019; Hulteen et al., 2020; Bardid et al., 2019 for reviews). In the UK and Ireland, the Tests of Gross Motor Development (TGMD 2^nd or 3^rd editions, Ulrich, 2000, 2020) are the most commonly utilised measure employed to assess FMS in school-based research (Behan et al., 2019; Belton et al., 2014; Bolgesr et al., 2018; O’ Brien et al., 2016a; Duncan et al., 2019; Eyre et al., 2018; Kelly et al., 2019; McGrane et al., 2018; O’Brien et al., 2018; Philpott et al., 2020). The TGMD-3 is a comprehensive battery of movement skills, comprising subtests for locomotor (i.e., running, jumping, hopping, galloping, sliding, skipping) and object-control (i.e., throwing, catching, dribbling, kicking, strike, underhand roll). As an assessment of FMS, the TGMD-3 can be used to identify developmental delay in relation to gross motor performance, to evaluate programmes aimed at enhancing FMS through intervention, and to assess changes as a function of increasing age, experience, instruction or intervention (Ulrich, 2000, 2020). However, while reliable and valid as a measure of FMS, employing the TGMD-3 requires a considerable amount of time to administer and interpret (L. Barnett et al., 2013). In addition to a considerable time burden on participants and trained staff, a specific space and equipment are required for assessment, which can make it logistically difficult to administer in school settings. As a result some researchers (e.g., Duncan et al., 2019) have only assessed specific skills from the TGMD-2 to represent FMS, and the feasibility of administering the TGMD-2 and 3 in school settings has been questioned (Tamplain et al., 2019; Valentini et al., 2018).

As a consequence, there have been studies demonstrating the validity of short forms of the TGMD-2 in Brazilian children (Valentini et al., 2018), pre-school children (Bandeira et al., 2020) and Irish Adolescents (Issartel et al., 2017). Most recently, Bandeira et al. (2020) employed confirmatory factor analysis (CFA) and machine learning through network analysis to identify a valid short form of the TGMD-2 for use with Brazilian pre-school children. Such an approach is attractive to researchers as it reduces administration time whilst still providing a robust measure of FMS for teachers, and health professionals. The work of Bandeira et al. (2020) however highlighted that there may be important cultural differences in the different skills within the TGMD-2 that would validly represent FMS for different children. It is therefore important to consider that a TGMD-2 short form derived from a Brazilian population may not be directly applicable to different sample. Similarly, while there are validated short forms for the 2^nd edition of the TGMD, to date, there has been no examination of this issue using the newer, 3^rd edition of the TGMD. This study aimed to address this issue by analysing the internal structure and evidence of validity of the TGMD-3 in a sample of primary school aged children, and to examine the potential for a short form of the TGMD-3 in this population.

Methods

Participants

As part of a national study “Moving Well-Being Well”, cross-sectional data were collected from 44 primary schools across Ireland. Participants (n = 1608, 47% girls, age range 5–11 years, mean age 9.2 ± 2.04) were recruited from these schools across twelve counties (56% rural, 44% urban). Irish schools in areas classified as “socioeconomically disadvantaged” qualify for the Delivering Equality of Opportunity in Schools (DEIS) programme. DEIS schools make up 22.3% of Irish primary schools (Irish Department of Education, Social Inclusion Section, 2017), and the sample presented here comprises of 18.8% DEIS schools. The sample contains data from the full range of the primary school cycle (Junior Infants through to sixth class), collected between March and June 2017.

Participating schools initially granted consent through each principal, and parental consent and child assent were also obtained. A unique numerical code was assigned to each participant to ensure anonymity, while age and sex were collected through the consent forms. Approval was granted from the institutional Research Ethics Committee (XXXX removed for review).

Measures

The Test of Gross Motor Development-3rd Edition (TGMD-3, Ulrich, 2020) was used to measure FMS proficiency. The TGMD-3 is a process oriented performance-based assessment and measures the criteria that make up a skill, rather than the outcome or product of that performance. As previously discussed, the TGMD-3 is comprised of locomotor (run, skip, gallop, slide, hop, and horizontal jump) and object-control (catch, overhand throw, underhand throw, kick, two handed strike, one handed strike, and stationary dribble) skill subtests (Ulrich, 2020). The TGMD-3 has been used extensively in previous research and has a high degree of validity (Temple & Foley, 2017; Webster & Ulrich, 2017 and reliability (Rey et al., 2020).

Data collection

The research team consisted of several postgraduate and graduate students, as well as full time sports coaches. Three days of formal training for all members of the research team was conducted to ensure consistency and accuracy in the testing process. A 95% inter-observer agreement was required by all team members on a pre-coded sample dataset. The lead researcher pre-coded this dataset (comprising data from 28 participants on all 13 skills) and all assessors were blind to the conditions of coding.

In line with the protocols outlined by Ulrich (2020), a visual demonstration of the skill was performed by a trained member of the research team prior to assessment. Participants were not made aware of the specific skill components being measured, and no verbal feedback was provided by the assessor. For each skill, participants first completed a familiarisation trial, followed by two trials which were then assessed live by a member of the research team. The number of skills criteria range from three to five components, dependent on the individual skill. If the participant successfully fulfilled the component criteria for any skill, a score of one was noted. A score of zero denotes that they failed to meet said criteria. The scores from both trials are totalled to produce the participant’s raw score per skill. Once each skill has been assessed, all results are combined to give a total raw score, as well as locomotor and object control skills subtotals.

The TGMD-3 has traditionally been assessed using the pen and paper method (Ulrich, 2020). Results must then be inputted into a database before undertaking any statistical analyses. This methodology is time consuming and also increases the risk of human error occurring during data entry. A unique iPad application was developed to counteract this, with the equivalent of the pen and paper version created within the app. This allowed for the participants data to be collected on electronic tablets, using their unique numerical code. When connected to a secured network, all data was uploaded to and consolidated on a secured server on the university campus. The application allows for multiple simultaneous users and caters for a secure cloud-based upload of pseudo-anonymised data. The system has been approved by the University’s Ethics committee and is also compliant with European general data protection regulations (GDPR). Further details outlined in Behan et al. (2019).

Statistical procedures

Descriptive analysis

Descriptive analyses (mean and standard deviation) were conducted to describe FMS. Asymmetry and kurtosis were used to assess the normality of the data. The Mahalanobis distance test was used to verify the presence of multivariate outliers (Marôco, 2010). To compare FMS, the t-test for independent samples was used. The effect size was calculated from the Cohen’s d, values <0.20 were considered small, <0.5 moderate and d > 0.5 large. (Cohen, 1992).

Confirmatory factorial analysis

The original dimensional structure of the TGMD-3 (Ulrich, 2020) and alternative models were considered for first and second order confirmatory factor analyses using the Mplus program (Version 8.0). The robust maximum likelihood estimator was used in all the analysis procedures (Asparouhov & Muthén, 2009). The Comparative Fit Index – CFI, the Tucker-Lewis Index – TLI (Tucker & Lewis, 1973) and the mean squared error of approximation – RMSEA (Jöreskog & Sörbom, 1981; Steiger, 1990) were used to assess the quality of the general adjustments of the tested models. RMSEA values ranged between 0.05 and 0.08 while CFI and TLI values > 0.90 indicated good fit (Hu & Bentler, 1999; Bentler, 1990). The Bayesian information criterion – BIC (Schwarz, 1978) was used to compare the models, low BIC values indicate more adjusted models (Byrne, 2013).

Network analysis

A network analysis was used to assess the relationship pattern of FMS. The network approach has been used as an alternative model for instrument validation and presents an unusual theoretical perspective. The variables of the tested model do not present a common latent cause (locomotion and object control), and because they are casually coupled, the tested model emerges from its connections instead of causation (Cramer, 2012; Schmittmann et al., 2013). In this sense, the role of each variable in the network, assessed from the strength centrality indicator, is equivalent to the factorial load of the skill in the latent model (Christensen & Golino, 2020; Christensen et al., 2020). The network perspective for instrument validation provides important information on the relationships between the tested fundamental movement skills and those skills that are most important in different contexts and settings, such as for boys and girls.

The network analysis of FMS was conducted from bivariate correlation graphs of partial correlations using the Graphical Least Absolute Shrinkage and Selection Operator – GeLASSO (Friedman et al., 2008). This algorithm corrects the sparsity of the network by presenting the most relevant relationships. The Extended Bayesian Information Criterion index – EBIC (Chen & Chen, 2008) was used to over-adjust the model. The strength centrality indicator was used to assess the role of each fundamental movement skill in the network. The analysis was conducted using Rstudio and qgraph (Epskamp et al., 2012).

Results

Table 1 shows the average values of FMS according to sex. Girls performed better in the run, skip, gallop, hop and horizontal jump skills, whereas boys performed better in the overhand and underhand throw, kick, two hand strike and one hand strike skills. There were no sex differences in the slide, catch or bounce.

Table 1. Fundamental movement skill by sex

Fundamental Movement Skills	Boys M (SD)
N = 845	Girls M (SD)
N = 763	p	Cohen´s d
Run	7.02 (1.85)	7.19 (1.60)	0.041	−0.102
Skip	3.62 (2.20)	4.36 (1.86)	< .001	−0.360
Gallop	4.47 (2.09)	5.11 (1.95)	< .001	−0.317
Slide	6.40 (2.32)	6.61 (2.07)	0.058	−0.095
Hop	6.65 (2.91)	7.23 (2.52)	< .001	−0.213
Horizontal Jump	4.71 (2.98)	5.42 (2.60)	< .001	−0.254
Catch	4.74 (1.73)	4.79 (1.67)	0.554	−0.030
Throw	4.28 (2.41)	3.14 (2.20)	< .001	0.489
Roll	5.33 (2.18)	5.07 (2.07)	0.206	0.012
Kick	6.03 (2.50)	4.98 (2.49)	< .001	0.418
Two Hand Strike	7.15 (2.61)	6.67 (2.43)	< .001	0.188
One Hand Strike	3.98 (2.95)	3.27 (2.79)	< .001	0.247
Bounce	4.01 (2.09)	3.90 (2.01)	0.270	0.055

M = mean value; SD = standard deviation.

Table 2. Model´s adjustment indexes

Model	${\chi }^2$ (df)	CFI	TLI	RMSEA	90%CI RMSEA	BIC^b	Correlation Factor
Robust method^r
Two Factor
13 skills	2932.750 (78)*	0.800	0.756	0.074	0.069–0.080	109,794.99	0.95
Two Factors
8 skills	1641.723 (28)*	0.948	0.923	0.052	0.043–0.063	57,738.75	0.96
Two Factors
7 skills	1467.555 (21) *	0.951	0.930	0.050	0.041–0.060	50,930.04	0.98
Two Factors
7 skills
Second Order	1706.650 (21) *	0.951	0.914	0.065	0.050–0.078	50,937.43	N/A

Note: r = robust standard errors and a robust (scaled) test statistic; b = Sample-size adjusted BIC. *p-value < .01.

Confirmatory factorial analysis

Step 1: Two-factor model with 13 FMS

The original model of TGMD-3 with two factors (locomotion and object control) and 13 FMS, six for locomotion (run, gallop, hop, skip, jump, slide) and seven for ball skills (two-hand strike, one-hand strike, bounce, catch, kick, overhand throw, underhand throw), was initially tested. Five Table 2 of the 13 skills: run, skip, kick, underhand throw and one-hand strike, showed low factor loads (<0.30). Adjustment rates were low and indicated a poor fit of the original model (CFI = 0.948; TLI = 0.923; [RMSEA = 0.052- CI 0.043–0.063]). The correlation between the two dimensions was high (r = 0.95).

Step 2: Model re-specification with two factors and eight fundamental movement skills

A second model was tested excluding the five low-load skills from the first model. A new dimensional structure with eight skills (gallop, hop, jump, slide, two-hand strike, bounce, catch, overhand throw) was tested. The slide ability had a low factor load (0.30), the adjustment indexes were adequate. (CFI = 0.948; TLI = 0.923; [RMSEA = 0.052 – CI 0.043–0.063]). The correlation between the two dimensions was high (r = 0.96).

Step 3: Model re-specification with two-factors and 7 fundamental movement skills

The model with 7 skills (gallop, hop, jump, two-hand strike, bounce, catch, overhand throw) and two factors showed adequate factor loads in all skills (> 0.30) and excellent general adjustment indices. (CFI = 0.951; TLI = 0.930; RMSEA = 0.050 [CI 0.041–0.060]). The correlation between the factors was high (r = 0.98).

Step 4: Two-factor model – second order

In the first 3 steps, the correlation between locomotion and ball skills factors was high (r > .90). In this way, we tested a hierarchical dimension based on a confirmatory factor analysis, the overall movement skill. In this second-order analysis, the general adjustment indices were excellent (CFI = 0.915; TLI = 0.914 [RMSEA = 0.065; CI = 0.053–0.078) and the factorial loads were adequate (> .30).

The two factors models for 7 skills and, second order two factor model for 7 skills are presented in Figure 1(a,b) respectively.

Figure 1. (a) Two-factor model – 7 skills, and (b) Two factors – 7 skills – second order.

Network analysis

Positive relationships between the skills are expressed by the blue colour, and negative relationships by the red colour in the network. The thickness of the graph indicates the weight of the ratio. From the analysis of the independent relationships between each of the variables (nodes), as well as the interactions (edges) between them, the results of the network analysis indicated that almost all FMS had reciprocal positive relationships between them in both girls (Figure 2(a)) and boys (Figure 2(b)). The skills with the highest indicator of strength centrality in the network were bounce (boys = 1.957; girls = 1.515), catch (boys = 0.231; girls = 0.974) and hop for boys (0.390) and horizontal jump for girls (0.445). The individual values for the relationships between each of the nodes that compound the network are presented as supplementary file (S1).

Figure 2. Network perspective of fundamental movement skills for a) girls Orange circles = locomotion skills (1: Gallop; 2: Hop; 3: Horizontal Jump); blue circles = object control skills (4: Catch; 5: Throw; 6: Two Hand Strike; 7: Bounce). and b) boys Orange circles = locomotion skills (1: Gallop; 2: Hop; 3: Horizontal Jump); blue circles = object control skills (4: Catch; 5: Throw; 6: Two Hand Strike; 7: Bounce).

Discussion

The present study evidences the validity and reliability of the internal structure of the TGMD-3 in Irish children and demonstrates that a short version of the TGMD-3, comprising seven skills (gallop, hop, horizontal jump, catch, throw, two hand strike, bounce) is a valid measure of FMS in this population. Importantly, network analysis indicated that the 7 skills were all positively related in both boys and girls. This proposed short version of the TGMD-3 therefore appears to be appropriate for both sexes.

The final model with 7 skills presented excellent model fit index values, which agrees with validations of the long version of TGMD-3 in Brazilian, American, Spanish, and Iranian children (Estevan et al., 2017; Mohammadi et al., 2017; Valentini et al., 2017; Webster & Ulrich, 2017). The loading of the 7 remaining skills ranged from .64 to .77, for throw and bounce, respectively, and are much higher than .30, which is considered a desirable value. However, the correlation value between the two dimensions remained very high (r = 0.98), indicating a low discriminant validity of the TGMD-3. Garn and Webster (2020) discussed issues of validity and reliability related to the TGMD-3, such as the dimensionality of the test (total FMS, locomotor skills and ball skills), which has not been rigorously explored and may potentially influence its use. Indeed, a very high correlation between the two dimensions of the test (locomotor and ball skills) destabilize the assumption of TGMD-3 as bidimensional. Results from the second order CFA analysis in this study showed that the proposed short version may be considered as a unidimensional factor and could thus be used to assess FMS, corroborating with recent results exploring its long version (Garn & Webster, 2020), and reinforces a new assumption on a unique latent construct also for the short version proposed.

In the present study, a new perspective for the validation of TGMD-3 is also suggested, though a network analysis. Network models are important tools for measuring dynamic and complex systems, such as FMS (Bandeira et al., 2020). It is reasonable to state that children from different countries and cultures display differences in their motor competence (Hulteen et al., 2018). In this sense, the specificities of each country and culture must be considered in the validation models, and the network model can be useful to discriminate differences between countries, and to determine the priority of intervention proposals, respecting cultural aspects, as previously suggested (Hulteen et al., 2018; L. M. Barnett et al., 2016),

In the emerged network models, all the nodes (skills) are related to each other, reasonably having a common cause (construct, latent trait) (Bandeira et al., 2020; Schmittmann et al., 2013). Our network analysis results showed positive relationships between the 7 remaining skills for boys and girls (including locomotion and ball domains of the original TGMD-3 version), which were closely interrelated, and contributing substantially to the emerged network. The network analysis centrality measures highlighted bounce and catch for both sexes, hop for boys, and horizontal jump for girls showed a higher strength value. These motor skills might be considered as the most important for the assessed children, reflecting the building blocks for the acquisition of other more complex skills (Newell, 2020). The measure of strength centrality allows us to individually assess the role of each node (movement skill) and how it can change the pattern of the entire network, which would be important for planning interventions strategies (Newell, 2020).

Prior research has provided evidence of validity for a short version of TGMD-2 for Brazilian preschoolers (Bandeira et al., 2020). Considering there are different skills in the TGMD-2, as used by Bandeira et al. (2020), and those in the TGMD-3, as used the present study, almost all the skills of the current short version are different from those identified in the aforementioned Brazilian study. While we recognize all the skills assessed in the TGMD-2 or 3 are important in developing positive trajectories of lifelong sports and physical activity, we suggest that not all motor skills discriminate in assessing competence in FMS, and are actually related to sports and physical activity according to the context/culture where they are assessed (L. M. Barnett et al., 2016; Newell, 2020). This assertion would hold for any short form of the TGMD as well as the full version. Taking the kick as an example, we might assume in cultures where soccer is the most prevalent sport, scores for kick would likely be higher compared to cultural contexts where soccer is less well played. Alongside the difference in sample characteristics between the present study with 5–11 year olds, and the prior work of Bandeira et al. (2020) in Brazilian preschoolers, the cultural differences in sport and movement environments between Brazil and Ireland may contribute to the reason why the skills in the short forms of the TGMD 2/3 in the present study and that or prior work differ. There are however other reasons why the skills included in the current study differ from those using the TGMD-2 in Brazilian preschoolers. For example, the criteria for scoring some of the skills in the TGMD-2 differ for the TGMD-3, making it difficult to directly compare the results of the current study with those of Bandeira et al. (2020). Likewise, there is an acknowledged developmental trajectory of motor competence, influenced by growth and maturation, genetics, affordances in the physical and social environment, which will all interact and contribute to performance on the TGMD-3 (Temple & Foley, 2017). Temple and Foley (2017) noted that this developmental validity of the TGMD-3 needs to be considered when interpreting scores from the test, where, for example, older children are expected to score more highly than their younger peers. As a consequence, although the present study used similar methods of statistical analysis to the prior work of Bandeira et al. (2020), direct comparison between the results of the current study with 5–11 year old Irish children and that of Bandeira et al. (2020) with Brazilian preschoolers should be made with caution.

Overall, the results of the current study, in evidencing the validity and reliability of the TGMD-3, present an important development for individuals interested in assessment of motor competence/FMS in children. A key consideration for researchers in this field is obtaining valid results reflecting motor competence in a time and labour economical manner. One criticism of the TGMD-3 has been the relatively high demand in terms of time and labour to collect (Eddy et al., 2019; Hulteen et al., 2020). The current study offers a way forward in addressing this issue by presenting a short version of the TGMD-3, which has potential for assessing motor competence in children. The large sample of participants examined should be considered a strength, and the TGMD-3 short form presented here can aid pragmatic and time efficient assessment of children’s motor competence. However, it is important to acknowledge the limitations of the present study. Further cross-validation of the TGMD-3 short form with an independent sample, additional to what is presented here, would be beneficial as would establishing standardised scores and normative data for the short version. Establishing these would be a useful next step to enable pragmatic use of the TGMD-3 short version with teachers, coaches and other related professionals.

This study presents a new perspective for the validation of TGMD-3 based on the network analysis. Although with different theoretical and statistical perspectives, the two validation proposals complement each other. CFA is useful to determine the composition of general scores, such as locomotion and ball skills, which is important for tracking motor delays based on scores and percentiles, while the network analysis allows for an understanding of the intrinsic correlations between skills, the role of each skill, and to determine the network sensitivity to changes through motor interventions.

Conclusions

The current study, presents an original contribution to the literature base related to motor competence in children by suggesting a short version of the TGMD-3, based on confirmatory factor analysis and network analysis, which validly represents motor competence in Irish children. The use of a machine learning approach via network analysis in the present study confirmed that the skills included in the TGMD-3 short version are similar for boys and girls. Collectively, the results of the present study identify a short form of the TGMD-3 that can be used to assess motor competence in a more time and labour efficient manner for teachers, coaches and public health practitioners.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online https://doi.org/10.1080/02640414.2021.1978161.

Additional information

Funding

Science Foundation Ireland, Award Number: SFI/12/RC/2289 Gaelic Athletic Association, Award Number: None World Technology University Network, Award Number: None