Trust the “Process”? When Fundamental Motor Skill Scores are Reliably Unreliable

References

• Aldridge, V. K., Dovey, T. M., & Wade, A. (2017). Assessing test-retest reliability of psychological measures: Persistent methodological problems. European Psychologist, 22(4), 207–218. https://doi.org/10.1027/1016-9040/a000298
   Web of Science ®Google Scholar
• Barnett, L. M., Minto, C., Lander, N., & Hardy, L. L. (2013). Interrater reliability assessment using the Test of Gross Motor Development-2. Journal of Science and Medicine in Sport, 17(6), 667–670. https://doi.org/10.1016/j.jsams.2013.09.013
   PubMed Web of Science ®Google Scholar
• Barnett, L. M., Stodden, D. F., Hulteen, R. M., & Sacko, R. S. (2020). Motor competence assessment. In T. A. Brusseau, S. J. Fairclough, & D. R. Lubans (Eds.), The Routledge handbook of youth physical activity (pp. 384–408). Routledge.
   Google Scholar
• Barnett, L. M., Webster, E. K., Hulteen, R. M., Meester, A. D., Valentini, N. C., Lenoir, M., Pesce, C., Getchell, N., Lopes, V. P., Robinson, L. E., Brian, A., & Rodrigues, L. P. (2022). Through the looking glass: A systematic review of longitudinal evidence, providing new insight for motor competence and health. Sports Medicine, 52(4), 875–920. https://doi.org/10.1007/s40279-021-01516-8
   PubMed Web of Science ®Google Scholar
• Bolger, L. E., Bolger, L. A., O’neill, C., Coughlan, E., O’brien, W., Lacey, S., Burns, C., & Bardid, F. (2020). Global levels of fundamental motor skills in children: A systematic review. Journal of Sports Sciences, 39(7), 717–753. https://doi.org/10.1080/02640414.2020.1841405
   PubMed Web of Science ®Google Scholar
• Carballo-Fazanes, A., Rey, E., Valentini, N. C., Rodríguez-Fernández, J. E., Varela-Casal, C., Rico-Díaz, J., Barcala-Furelos, R., & Abelairas-Gómez, C. (2021). Intra-rater (live vs. video assessment) and inter-rater (expert vs. novice) reliability of the Test of Gross Motor Development—third edition. International Journal of Environmental Research and Public Health, 18(4), 1652. https://doi.org/10.3390/ijerph18041652
   PubMed Web of Science ®Google Scholar
• Gabbard, C. (2021). Lifelong motor development (7th ed.). Wolters Kluwer.
   Google Scholar
• Gamer, M., Lemon, J., & Singh, I. F. P. (2019). Irr: Various coefficients of interrater reliability and agreement. R package version 0.84.1. https://CRAN/R-project.org/package=irr
   Google Scholar
• Grothe, O., & Schmid, F. (2011). Kendall’s W reconsidered. Communications in Statistics-Simulation and Computation, 40(2), 285–305. https://doi.org/10.1080/03610918.2010.538791
   Google Scholar
• Hulteen, R. M., Barnett, L. M., Morgan, P. J., Robinson, L. E., Barton, C. J., Wrotniak, B. H., & Lubans, D. R. (2018). Development, content validity and test-retest reliability of the lifelong physical activity skills battery in adolescents. Journal of Sports Sciences, 36(20), 2358–2367. https://doi.org/10.1080/02640414.2018.1458392
   PubMed Web of Science ®Google Scholar
• Hulteen, R. M., Barnett, L. M., True, L., Lander, N. J., Cruz, B. D. P., & Lonsdale, C. (2020). Validity and reliability evidence for motor competence assessments in children and adolescents: A systematic review. Journal of Sports Sciences, 38(15), 1717–1798. https://doi.org/10.1080/02640414.2020.1756674
   PubMed Web of Science ®Google Scholar
• Hulteen, R. M., Terlizzi, B., Abrams, T. C., Sacko, R. S., Meester, A. D., Pesce, C., & Stodden, D. F. (2023). Reinvest to assess: Advancing approaches to motor competence measurement across the lifespan. Sports Medicine, 53(1), 33–50. https://doi.org/10.1007/s40279-022-01750-8
   PubMed Web of Science ®Google Scholar
• Kambas, A., & Venetsanou, F. (2014). The democritos movement screening tool for preschool children (DEMOST-PRE©): Development and factorial validity. Research in Developmental Disabilities, 35(7), 1528–1533. https://doi.org/10.1016/j.ridd.2014.03.046
   PubMed Web of Science ®Google Scholar
• Kiphard, E. J., & Schilling, F. (1974). Körperkoordinations Test für Kinder. Beltz Teset GmbH.
   Google Scholar
• Kiphard, E. J., & Schilling, F. (2007). Körperkoordinations Test für Kinder. KTK. Beltz-Test.
   Google Scholar
• Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
   PubMed Web of Science ®Google Scholar
• Kottner, J., & Streiner, D. L. (2011). The difference between reliability and agreement. Journal of Clinical Epidemiology, 64(6), 701–702. https://doi.org/10.1016/j.jclinepi.2010.12.001
   PubMed Web of Science ®Google Scholar
• Lane, A. P., Molina, S. L., Tolleson, D. A., Langendorfer, S. J., Goodway, J., & Stodden, D. (2017). Developmental sequences for the standing long jump landing: A pre-longitudinal screening. Journal of Motor Learning and Development, 6(1), 114–129. https://doi.org/10.1123/jmld.2016-0058
   Google Scholar
• Langendorfer, S. J., & Roberton, M. A. (2002). Individual pathways in the development of forceful throwing. Research Quarterly for Exercise and Sport, 73(3), 245–256. https://doi.org/10.1080/02701367.2002.10609018
   PubMed Web of Science ®Google Scholar
• Logan, S. W., Ross, S. M., Chee, K., Stodden, D. F., & Robinson, L. E. (2018). Fundamental motor skills: A systematic review of terminology. Journal of Sports Sciences, 36(7), 781–796. https://doi.org/10.1080/02640414.2017.1340660
   PubMed Web of Science ®Google Scholar
• Maeng, H., Webster, E. K., Pitchford, E. A., & Ulrich, D. A. (2017). Inter- and intrarater reliabilities of the Test of Gross Motor Development—third edition among experienced TGMD-2 raters. Adapted Physical Activity Quarterly, 34(4), 442–455. https://doi.org/10.1123/apaq.2016-0026
   PubMed Web of Science ®Google Scholar
• Maïano, C., Morin, A. J., April, J., Webster, E. K., Hue, O., Dugas, C., & Ulrich, D. (2022). Psychometric properties of a french-Canadian version of the Test of Gross Motor Development–third edition (TGMD-3): A bifactor structural equation modeling approach. Measurement in Physical Education and Exercise Science, 26(1), 51–62. https://doi.org/10.1080/1091367X.2021.1946541
   Web of Science ®Google Scholar
• McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46. https://doi.org/10.1037/1082-989x.1.1.30
   Web of Science ®Google Scholar
• Nichols, T. R., Wisner, P. M., Cripe, G., & Gulabchand, L. (2010). Putting the kappa statistic to use. Quality Assurance Journal, 13(3–4), 57–61. https://doi.org/10.1002/qaj.481
   Google Scholar
• Palmer, K. K., & Brian, A. (2016). Test of Gross Motor Development-2 scores differ between expert and novice coders. Journal of Motor Learning and Development, 4(2), 142–151. https://doi.org/10.1123/jmld.2015-0035
   Google Scholar
• Palmer, K. K., Stodden, D. F., Ulrich, D. A., & Robinson, L. E. (2021). Using process- and product-oriented measures to evaluate changes in motor skills across an intervention. Measurement in Physical Education and Exercise Science, 25(3), 273–282. https://doi.org/10.1080/1091367X.2021.1876069
   PubMed Web of Science ®Google Scholar
• R Core Team. (n.d.). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Retrieved July 11, 2022, from https://www.R-project.org/
   Google Scholar
• Rintala, P. O., Sääkslahti, A. K., & Iivonen, S. (2017). Reliability assessment of scores from video-recorded TGMD-3 performances. Journal of Motor Learning and Development, 5(1), 59–68. https://doi.org/10.1123/jmld.2016-0007
   Google Scholar
• Robinson, L. E., Stodden, D. F., Barnett, L. M., Lopes, V. P., Logan, S. W., Rodrigues, L. P., & D’hondt, E. (2015). Motor competence and its effect on positive developmental trajectories of health. Sports Medicine, 45(9), 1273–1284. https://doi.org/10.1007/s40279-015-0351-6
   PubMed Web of Science ®Google Scholar
• Stodden, D. F., Goodway, J. D., Langendorfer, S. J., Roberton, M. A., Rudisill, M. E., Garcia, C., & Garcia, L. E. (2008). A developmental perspective on the role of motor skill competence in physical activity: An emergent relationship. Quest, 60(2), 290–306. https://doi.org/10.1080/00336297.2008.10483582
   Web of Science ®Google Scholar
• Sun, S. -H., Zhu, Y., Shih, C. -L., Lin, C. -H., & Wu, S. K. (2010). Development and initial validation of the preschooler gross motor quality scale. Research in Developmental Disabilities, 31(6), 1187–1196. https://doi.org/10.1016/j.ridd.2010.08.002
   PubMed Web of Science ®Google Scholar
• Ulrich, D. (1985). Test of gross motor development. Pro-Ed.
   Google Scholar
• Ulrich, D. (2000). Test of gross motor development-2nd Edition. Pro-Ed.
   Google Scholar
• Ulrich, D. A. (2017). Introduction to the special section: Evaluation of the psychometric properties of the TGMD-3. Journal of Motor Learning and Development, 5(1), 1–4. https://doi.org/10.1123/jmld.2017-0020
   Google Scholar
• Ulrich, D. (2019). Test of gross motor development-3rd Edition. Pro-Ed.
   Google Scholar
• Ulrich, D., & Webster, E. K. (n.d.). Training Videos. Retrieved September 29, 2022, from https://sites.google.com/view/tgmd-3/videos
   Google Scholar
• Webster, E. K., & Ulrich, D. A. (2017). Evaluation of the psychometric properties of the test of gross motor development—third edition. Journal of Motor Learning and Development, 5(1), 45–58. https://doi.org/10.1123/jmld.2016-0003
   Google Scholar
• Wickham, H. (2021). Tidyr: Tidy messy data. R package version 1.1.3. Tidyr: Tidy messy data. R Package Version 1.1.3. http://CRAN.R-project.org/package=tidyr
   Google Scholar
• Willerman, B. (1955). The adaptation and use of Kendall’s coefficient of concordance (W) to sociometric-type rankings. Psychological Bulletin, 52(2), 132–133. https://doi.org/10.1037/h0041665
   PubMed Web of Science ®Google Scholar
• Williams, K., Haywood, K., & VanSant, A. (1995). Throwing patterns of older adults: A follow-up investigation. International Journal of Aging & Human Development, 33(4), 279–294. https://doi.org/10.2190/891h-v1m1-493b-gp4p
   Google Scholar
• Williams, H. G., Pfeiffer, K. A., Dowda, M., Jeter, C., Jones, S., & Pate, R. R. (2009). A field-based testing protocol for assessing gross motor skills in preschool children: The children’s activity and movement in preschool study motor skills protocol. Measurement in Physical Education and Exercise Science, 13(3), 151–165. https://doi.org/10.1080/10913670903048036
   PubMedGoogle Scholar
• Zuvela, F., Bozanic, A., & Miletic, D. (2011). A new fundamental movement skills test for 8 year old children: Construction and validation. Journal of Sports Science & Medicine, 10(1), 157–163.
   PubMed Web of Science ®Google Scholar