Test–retest reliability measures for curve data: an overview with recommendations and supplementary code

References

• Abramowicz, K., Häger, C. K., Pini, A., Schelin, L., Sjöstedt de Luna, S., & Vantini, S. (2018). Nonparametric inference for functional‐on‐scalar linear models applied to knee kinematic hop data after injury of the anterior cruciate ligament. Scandinavian Journal of Statistics, 45, 1036–1061. doi:https://doi.org/10.1111/sjos.v45.4
   Google Scholar
• Atkinson, G., & Nevill, A. M. (1998). Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Medicine, 26, 217–238. doi:https://doi.org/10.2165/00007256-199826040-00002
   PubMed Web of Science ®Google Scholar
• Berchtold, A. (2016). Test–retest: agreement or reliability? Methodological Innovations, 9, 1–7. doi:https://doi.org/10.1177/2059799116672875
   Google Scholar
• Bland, J. M., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327, 307–310. doi:https://doi.org/10.1016/S0140-6736(86)90837-8
   Web of Science ®Google Scholar
• Bruton, A., Conway, J. H., & Holgate, S. T. (2000). Reliability: what is it, and how is it measured? Physiotherapy, 86, 94–99. doi:https://doi.org/10.1016/S0031-9406(05)61211-4
   Google Scholar
• Chau, T., Young, S., & Redekop, S. (2005). Managing variability in the summary and comparison of gait data. Journal of Neuroengineering and Rehabilitation, 2, 22. doi:https://doi.org/10.1186/1743-0003-2-22
   PubMedGoogle Scholar
• Chia, K., & Sangeux, M. (2017). Quantifying sources of variability in gait analysis. Gait & Posture, 56, 68–75. doi:https://doi.org/10.1016/j.gaitpost.2017.04.040
   PubMedGoogle Scholar
• Colyer, S. L., Evans, M., Cosker, D. P., & Salo, A. I. (2018). A review of the evolution of vision-based motion analysis and the integration of advanced computer vision methods towards developing a markerless system. Sports Medicine-Open, 4, 1–15. doi:https://doi.org/10.1186/s40798-017-0116-x
   PubMedGoogle Scholar
• De Vet, H. C., Terwee, C. B., Knol, D. L., & Bouter, L. M. (2006). When to use agreement versus reliability measures. Journal of Clinical Epidemiology, 59, 1033–1039. doi:https://doi.org/10.1016/j.jclinepi.2005.10.015
   PubMed Web of Science ®Google Scholar
• Donoghue, O. A., Harrison, A. J., Coffey, N., & Hayes, K. (2008). Functional data analysis of running kinematics in chronic Achilles tendon injury. Medicine & Science in Sports & Exercise, 40, 1323–1335. doi:https://doi.org/10.1249/MSS.0b013e31816c4807
   PubMed Web of Science ®Google Scholar
• Dubin, J. A., & Müller, H. G. (2005). Dynamical correlation for multivariate longitudinal data. Journal of the American Statistical Association, 100, 872–881. doi:https://doi.org/10.1198/016214504000001989
   Web of Science ®Google Scholar
• Elias, A. R., Hammill, C. D., & Mizner, R. L. (2015). Changes in quadriceps and hamstring cocontraction following landing instruction in patients with anterior cruciate ligament reconstruction. Journal of Orthopaedic & Sports Physical Therapy, 45, 273–280. doi:https://doi.org/10.2519/jospt.2015.5335
   PubMed Web of Science ®Google Scholar
• Ford, K. R., Myer, G. D., & Hewett, T. E. (2007). Reliability of landing 3D motion analysis: implications for longitudinal analyses. Medicine and Science in Sports and Exercise, 39, 2021–2028. doi:https://doi.org/10.1249/mss.0b013e318149332d
   PubMed Web of Science ®Google Scholar
• Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. P., Frith, C. D., & Frackowiak, R. S. (1994). Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping, 2, 189–210. doi:https://doi.org/10.1002/hbm.460020402
   Google Scholar
• Gorton, G. E., III, Hebert, D. A., & Gannotti, M. E. (2009). Assessment of the kinematic variability among 12 motion analysis laboratories. Gait & Posture, 29, 398–402. doi:https://doi.org/10.1016/j.gaitpost.2008.10.060
   PubMed Web of Science ®Google Scholar
• Gribbin, T. C., Slater, L. V., Herb, C. C., Hart, J. M., Chapman, R. M., Hertel, J., & Kuenze, C. M. (2016). Differences in hip-knee joint coupling during gait after anterior cruciate ligament reconstruction. Clinical Biomechanics., 32, 64–71. doi:https://doi.org/10.1016/j.clinbiomech.2016.01.006
   PubMed Web of Science ®Google Scholar
• Harrison, A. J., Ryan, W., & Hayes, K. (2007). Functional data analysis of joint coordination in the development of vertical jump performance. Sports Biomechanics, 6, 199–214. doi:https://doi.org/10.1080/14763140701323042
   PubMed Web of Science ®Google Scholar
• Hébert-Losier, K., Pini, A., Vantini, S., Strandberg, J., Abramowicz, K., Schelin, L., & Häger, C. K. (2015). One-leg hop kinematics 20 years following anterior cruciate ligament rupture: data revisited using functional data analysis. Clinical Biomechanics, 30, 1153–1161. doi:https://doi.org/10.1016/j.clinbiomech.2015.08.010
   PubMed Web of Science ®Google Scholar
• Hébert-Losier, K., Schelin, L., Tengman, E., Strong, A., & Häger, C. K. (2018). Curve analyses reveal altered knee, hip, and trunk kinematics during drop–jumps long after anterior cruciate ligament rupture. The Knee, 25, 226–239. doi:https://doi.org/10.1016/j.knee.2017.12.005
   PubMedGoogle Scholar
• Hopkins, W. G. (2000). Measures of reliability in sports medicine and science. Sports Medicine, 30, 1–15. doi:https://doi.org/10.2165/00007256-200030010-00001
   PubMed Web of Science ®Google Scholar
• Iosa, M., Cereatti, A., Merlo, A., Campanini, I., Paolucci, S., & Cappozzo, A. (2014). Assessment of waveform similarity in clinical gait data: the linear fit method. BioMed Research International. doi:https://doi.org/10.1155/2014/214156
   Google Scholar
• Jacques, J., & Preda, C. (2014). Functional data clustering: a survey. Advances in Data Analysis and Classification, 8, 231–255. doi:https://doi.org/10.1007/s11634-013-0158-y
   Web of Science ®Google Scholar
• Kadaba, M. P., Ramakrishnan, H. K., Wootten, M. E., Gainey, J., Gorton, G., & Cochran, G. V. B. (1989). Repeatability of kinematic, kinetic, and electromyographic data in normal adult gait. Journal of Orthopaedic Research, 7, 849–860. doi:https://doi.org/10.1002/jor.1100070612
   PubMed Web of Science ®Google Scholar
• Kainz, H., Graham, D., Edwards, J., Walsh, H. P., Maine, S., Boyd, R. N., … Carty, C. P. (2017). Reliability of four models for clinical gait analysis. Gait & Posture, 54, 325–331. doi:https://doi.org/10.1016/j.gaitpost.2017.04.001
   PubMed Web of Science ®Google Scholar
• Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163. doi:https://doi.org/10.1016/j.jcm.2016.02.012
   PubMed Web of Science ®Google Scholar
• Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., … Streiner, D. L. (2011). Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. Journal of Clinical Epidemiology, 64, 96–106. doi:https://doi.org/10.1016/j.jclinepi.2010.03.002
   PubMed Web of Science ®Google Scholar
• Markström, J. L., Schelin, L., & Häger, C. K. (2018). A novel standardised side hop test reliably evaluates landing mechanics for anterior cruciate ligament reconstructed persons and controls. Sports Biomechanics. doi:https://doi.org/10.1080/14763141.2018.1538385
   Google Scholar
• Marron, J. S., Ramsay, J. O., Sangalli, L. M., & Srivastava, A. (2014). Statistics of time warpings and phase variations. Electronic Journal of Statistics, 8, 1697–1702. doi:https://doi.org/10.1214/14-EJS901
   Web of Science ®Google Scholar
• McGinley, J. L., Baker, R., Wolfe, R., & Morris, M. E. (2009). The reliability of three-dimensional kinematic gait measurements: a systematic review. Gait & Posture, 29, 360–369. doi:https://doi.org/10.1016/j.gaitpost.2008.09.003
   PubMed Web of Science ®Google Scholar
• McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. doi:https://doi.org/10.1037/1082-989X.1.1.30
   Web of Science ®Google Scholar
• Meldrum, D., Shouldice, C., Conroy, R., Jones, K., & Forward, M. (2014). Test–retest reliability of three dimensional gait analysis: including a novel approach to visualising agreement of gait cycle waveforms with Bland and Altman plots. Gait & Posture, 39, 265–271. doi:https://doi.org/10.1016/j.gaitpost.2013.07.130
   PubMed Web of Science ®Google Scholar
• Milner, C. E., Westlake, C. G., & Tate, J. J. (2011). Test–retest reliability of knee biomechanics during stop jump landings. Journal of Biomechanics, 44, 1814–1816. doi:https://doi.org/10.1016/j.jbiomech.2011.04.005
   PubMed Web of Science ®Google Scholar
• Moudy, S., Richter, C., & Strike, S. (2018). Landmark registering waveform data improves the ability to predict performance measures. Journal of Biomechanics, 78, 109–117. doi:https://doi.org/10.1016/j.jbiomech.2018.07.027
   PubMedGoogle Scholar
• Pataky, T. C., Robinson, M. A., & Vanrenterghem, J. (2013). Vector field statistical analysis of kinematic and force trajectories. Journal of Biomechanics, 46, 2394–2401. doi:https://doi.org/10.1016/j.jbiomech.2013.07.031
   PubMed Web of Science ®Google Scholar
• Pollard, C. D., Stearns, K. M., Hayes, A. T., & Heiderscheit, B. C. (2015). Altered lower extremity movement variability in female soccer players during side-step cutting after anterior cruciate ligament reconstruction. American Journal of Sports Medicine., 43, 460–465. doi:https://doi.org/10.1177/0363546514560153
   PubMed Web of Science ®Google Scholar
• R Core Team. (2014). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. http://www.R-project.org/
   Google Scholar
• Ramsay, J. O., & Silverman, B. W. (2005). Functional data analysis. New York, NY: Springer.
   Google Scholar
• Rankin, G., & Stokes, M. (1998). Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clinical Rehabilitation, 12, 187–199. doi:https://doi.org/10.1191/026921598672178340
   PubMed Web of Science ®Google Scholar
• Richter, C., O’Connor, N. E., Marshall, B., & Moran, K. (2014). Comparison of discrete- point vs. dimensionality-reduction techniques for describing performance-related aspects of maximal vertical jumping. Journal of Biomechanics, 47, 3012–3017. doi:https://doi.org/10.1016/j.jbiomech.2014.07.001
   PubMed Web of Science ®Google Scholar
• Røislien, J., Rennie, L., & Skaaret, I. (2012). Functional limits of agreement: a method for assessing agreement between measurements of gait curves. Gait & Posture, 36, 495–499. doi:https://doi.org/10.1016/j.gaitpost.2012.05.001
   PubMedGoogle Scholar
• Røislien, J., Skare, Ø., Opheim, A., & Rennie, L. (2012). Evaluating the properties of the coefficient of multiple correlation (CMC) for kinematic gait data. Journal of Biomechanics, 45, 2014–2018. doi:https://doi.org/10.1016/j.jbiomech.2012.05.014
   PubMed Web of Science ®Google Scholar
• Ryan, W., Harrison, A., & Hayes, K. (2006). Functional data analysis of knee joint kinematics in the vertical jump. Sports Biomechanics, 5, 121–138. doi:https://doi.org/10.1080/14763141.2006.9628228
   PubMed Web of Science ®Google Scholar
• Sangalli, L. M., Secchi, P., Vantini, S., & Vitelli, V. (2010). K-mean alignment for curve clustering. Computational Statistics & Data Analysis, 54, 1219–1233. doi:https://doi.org/10.1016/j.csda.2009.12.008
   Web of Science ®Google Scholar
• Sangeux, M., Passmore, E., Graham, H. K., & Tirosh, O. (2016). The gait standard deviation, a single measure of kinematic variability. Gait & Posture, 46, 194–200. doi:https://doi.org/10.1016/j.gaitpost.2016.03.015
   PubMed Web of Science ®Google Scholar
• Sankey, S. P., Azidin, R. M. F. R., Robinson, M. A., Malfait, B., Deschamps, K., Verschueren, S., … Vanrenterghem, J. (2015). How reliable are knee kinematics and kinetics during side-cutting manoeuvres? Gait & Posture, 41, 905–911. doi:https://doi.org/10.1016/j.gaitpost.2015.03.014
   PubMed Web of Science ®Google Scholar
• Schwartz, M. H., Trost, J. P., & Wervey, R. A. (2004). Measurement and management of errors in quantitative gait data. Gait & Posture, 20, 196–203. doi:https://doi.org/10.1016/j.gaitpost.2003.09.011
   PubMed Web of Science ®Google Scholar
• Sinsurin, K., Vachalathiti, R., Srisangboriboon, S., & Richards, J. (2018). Knee joint coordination during single-leg landing in different directions. Sports Biomechanics. doi:https://doi.org/10.1080/14763141.2018.1510024
   PubMed Web of Science ®Google Scholar
• Sørensen, H., Goldsmith, J., & Sangalli, L. M. (2013). An introduction with medical applications to functional data analysis. Statistics in Medicine, 32, 5222–5240. doi:https://doi.org/10.1002/sim.5989
   PubMedGoogle Scholar
• Srinivasan, D., & Mathiassen, S. E. (2012). Motor variability in occupational health and performance. Clinical Biomechanics, 27, 979–993. doi:https://doi.org/10.1016/j.clinbiomech.2012.08.007
   PubMed Web of Science ®Google Scholar
• Trevethan, R. (2017). Intraclass correlation coefficients: clearing the air, extending some cautions, and making some requests. Health Services and Outcomes Research Methodology, 17, 127–143. doi:https://doi.org/10.1007/s10742-016-0156-6
   Web of Science ®Google Scholar
• Vsevolozhskaya, O. A., Greenwood, M. C., Bellante, G. J., Powell, S. L., Lawrence, R. L., & Repasky, K. S. (2013). Combining functions and the closure principle for performing follow-up tests in functional analysis of variance. Computational Statistics & Data Analysis, 67, 175–184. doi:https://doi.org/10.1016/j.csda.2013.05.005
   Google Scholar
• Warmenhoven, J., Cobley, S., Draper, C., Harrison, A., Bargary, N., & Smith, R. (2019a). Bivariate functional principal components analysis: considerations for use with multivariate movement signatures in sports biomechanics. Sports Biomechanics, 18, 10–27. doi:https://doi.org/10.1080/14763141.2017.1384050
   PubMed Web of Science ®Google Scholar
• Warmenhoven, J., Cobley, S., Draper, C., Harrison, A., Bargary, N., & Smith, R. (2019b). Considerations for the use of functional principal components analysis in sports biomechanics: Examples from on-water rowing. Sports Biomechanics, 18, 317–341. doi:https://doi.org/10.1080/14763141.2017.1392594
   PubMed Web of Science ®Google Scholar
• Weir, J. P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. The Journal of Strength & Conditioning Research, 19, 231–240. doi:https://doi.org/10.1519/15184.1
   PubMed Web of Science ®Google Scholar