Evaluating the Replicability of Sample Results: A Tutorial of Double Cross-Validation Methods

Abstract

Although replication is important to the validity of a study and is endorsed by more and more scholars, few researchers in kinesiology attend to this issue. Some researchers may believe that statistical significance and effect size are the most important statistical issues in their research and thereby may have ignored the importance of result replication. Both significance testing and effect size, however, rarely inform researchers about the likelihood that results will be replicated in different samples drawn from the same population. The primary purpose of this tutorial is to provide readers with information regarding result replicability evaluation, focusing on double cross-validation approaches. This tutorial can also serve as a practical guide for conducting replicability analyses using double cross-validation methods.