Interpreting magnitude of change in strength and conditioning: Effect size selection, threshold values and Bayesian updating

ABSTRACT

The magnitude of change following strength and conditioning (S&C) training can be evaluated comparing effect sizes to thresholds. This study conducted a series of meta-analyses and compiled results to identify thresholds specific to S&C, and create prior distributions for Bayesian updating. Pre- and post-training data from S&C interventions were translated into standardised mean difference (SMD_pre) and percentage improvement (%Improve) effect sizes. Bayesian hierarchical meta-analysis models were conducted to compare effect sizes, develop prior distributions, and estimate 0.25-, 0.5-, and 0.75-quantiles to determine small, medium, and large thresholds, respectively. Data from 643 studies comprising 6574 effect sizes were included in the analyses. Large differences in distributions for both SMD_pre and %Improve were identified across outcome domains (strength, power, jump and sprint performance), with analyses of the tails of the distributions indicating potential large overestimations of SMD_pre values. Future evaluations of S&C training will be improved using Bayesian approaches featuring the information and priors developed in this study. To facilitate an uptake of Bayesian methods within S&C, an easily accessible tool employing intuitive Bayesian updating was created. It is recommended that the tool and specific thresholds be used instead of isolated effect size calculations and Cohen’s generic values when evaluating S&C training.

KEYWORDS:

• Effect size
• power
• prior
• S&C

Introduction

Strength and conditioning (S&C) which involves the application of a range of training modalities to enhance athletic performance has become a well-established discipline within sport and exercise science (Weldon et al., 2021, 2020). Much of the popularity of S&C originates from the perspective that muscular strength is of primary importance in athletic and sports performance (Suchomel et al., 2016; Swinton et al., 2014); however, it has been argued that evidence is lacking to demonstrate causal effects (Steele et al., 2020). Additionally, muscular strength can be developed extensively with relatively limited time and cost. Research investigating training methods such as resistance, sprint and plyometrics has experienced rapid growth since the late 1990s (Kraemer et al., 2017; Ramirez-Campillo et al., 2018), with large numbers of training studies focusing on identifying the best regimes to improve different aspects of fitness. A challenge in integrating the findings of longitudinal research within a general framework includes the potential for idiosyncratic findings based on experimental choices of individual studies. Findings may be influenced by a range of moderating factors including the study population, the length of training, the outcomes assessed, and the training regimes adopted. Given the range of factors that could influence results, evidence synthesis approaches including meta-analyses may provide the most effective means of summarising results and identifying general patterns that can inform researchers and practitioners.

Evidence synthesis and the use of meta-analyses to objectively quantify various phenomena across training studies has become common in S&C (Peterson et al., 2005, 2004; Rhea, 2004; Rhea & Alderman, 2004; Rhea et al., 2002, 2003). The most frequently reported effect size statistic is the pre-standardised mean difference (SMD_pre), where the mean change is divided by the pre-training standard deviation. This statistic comprises several conceptual advantages (Caldwell & Vigotsky, 2020) including the fact that it is dimensionless and enables the synthesis of outcomes reported across different units and scales (e.g., vertical jump power in thousands of Watts, and vertical jump height in tens of centimetres). With this process there is also an implicit assumption of underlying latent constructs (i.e., strength, power, jump and sprint performance) with appropriate construct validity represented through measurement from different tests (e.g., operationalisations). One of the primary reasons for the widespread use of the SMD_pre includes the existence of commonly used threshold values to apply qualitative labels describing the magnitude of change following training as “small”, “medium”, or “large”. However, threshold values have generally used Cohen’s initial suggestions (Cohen, 1988) which were determined arbitrarily with the behavioural and social sciences in mind. To obtain suitable estimates of the population effect size, the sample standard deviation must reflect the spread of values in the population. However, if random sampling is not conducted and restricted homogenous samples are obtained, the sample standard deviation is likely to underestimate the population standard deviation and thereby overestimate SMD_pre (Baguley, 2009). An alternative effect size statistic focused on sample means that can provide simple and intuitive interpretations of the magnitude of change following training is the relative ratio (RR), calculated by taking the ratio of two means (Hedges et al., 1999). Like the SMD_pre, the RR (post-training mean divided by pre-training mean) is dimensionless enabling synthesis of outcomes across different units and scales. It has been argued that the RR, which can also be interpreted in terms of percentage improvement (e.g., 1.50 is equivalent to a 50% increase from baseline, and 0.8 is equivalent to a 20% decrease) is easier to interpret than the SMD_pre making it a more applicable summary statistic (Friedrich et al., 2011). It is important to note that the RR can be calculated from means pre- and post-training, or from the means of two different groups. The RR has received limited use in previous meta-analyses conducted in sport science (Deb et al., 2018) which may be partly due to standard thresholds not being available.

Almost all statistical analyses conducted in S&C research employ a frequentist frameworks where effect sizes are calculated anew without including prior information of likely values based on previous research. In the minority of cases where uncertainty in effect sizes is quantified, frequentist frameworks apply confidence intervals that can be challenging to interpret (Hespanhol et al., 2019). Additionally, due to the small sample-sizes frequently used in S&C training studies (Rhea et al., 2003), uncertainty in effect sizes calculated under frequentist frameworks is likely to lack precision. In contrast, Bayesian frameworks enable individuals to include prior information and express uncertainty in effect sizes estimates in a probabilistic manner (e.g., using a posterior distribution), borrowing strength from previous research to increase precision. Common critiques of Bayesian approaches include the complexity that may exist with the analysis process and challenges in creating suitable informative priors (Goodman, 1999). However, meta-analysis models estimate a set of training effects rather than a single estimate and can thus be used to develop priors that combine with new data using simple calculations to obtain normally distributed posteriors quantifying the most likely population effect sizes (Jones et al., 2018). Given the large amount of training studies that have been conducted in S&C and limitations in current frameworks to interpret the magnitude of change following training, the purpose of this study was to conduct a series of large-scale meta-analyses to generate context-specific thresholds for multiple effect sizes and develop Bayesian priors. Meta-analysis models were conducted and compared across multiple outcome domains with results incorporated into an accessible tool to better interpret future S&C training.

2. Methods

2.1. Search strategy

A search was performed for published and unpublished studies in the English language that included S&C training conducted prior to January 2018. The search was performed using Embase, Medline, Web of Science, Sport Discus and Google Scholar. Hand searching of relevant journals including Medicine and Science in Sports and Exercise, the Journal of Strength and Conditioning Research, and Research Quarterly was also conducted. Database search terms were included to identify various training modes and a range of outcome measures. The following keywords and phrases were combined with Boolean operators; “strength” OR “resistance” OR “sprint” OR “plyometric” OR “exercise” AND “intervention” OR “training” OR “program” OR “programme” AND “1RM” OR “repetition maximum” OR “speed” OR “velocity” OR “power” OR “jump” OR “change of direction” OR “agility” OR “acceleration” OR “rate of force development”. No restriction was placed on the date of the study.

2.2. Inclusion criteria

Inclusion and exclusion criteria for the current meta-analysis were set to include as many relevant S&C training modes and dependent variables as possible. Inclusion criteria comprised: 1) any training-based study ≥4 weeks; 2) healthy trained or untrained participants with a mean age between 14 and 60; 3) training group with a minimum of 4 participants; 4) pre- and post-training means and standard deviations; and 5) sufficient information provided to appropriately describe the training method. Studies comprising training that were predominantly aerobic-based or rehabilitation focused were excluded.

2.3. Study selection and data extraction

Following deduplication, a three-level selection process comprising title, then abstract then full-text screening was completed. Studies were screened and selected for inclusion independently by AM with discussions with PS and KB where required. A standardised extraction codebook was developed using Microsoft Excel, with data extracted and coded independently by four researchers (AM, JP, AH, LG) in duplicate with AM completing extraction for all studies to provide consistency. The following data were extracted: 1) study details (authors, year, total number of groups); 2) participant characteristics (final study n, gender, training status, and age); 3) outcome domain (maximum strength, power, jump performance, and sprinting performance); and 4) pre- and post-training means and standard deviations.

The following definitions were used to categorise outcome domains: 1) maximum strength: a measure of maximum force production where time was not limited (e.g., 1–6 repetition maximum, isometric mid-thigh pull, peak torque); 2) power: a direct measurement of power output measured in Watts (absolute and normalised relative to body mass); 3) jump performance: a measure of jump height or distance; and 4) sprint performance: a measurement of the time to complete a specified linear distance or the velocity achieved. Where pre- and post-training data were not presented in text but in figures, data were extracted using PlotDigitizer 2.6.8 Windows. Study authors were not contacted for data where this could not be retrieved in text or through digitization on a pragmatic decision based on resource availability.

2.4. Statistical analysis

A Bayesian framework was chosen over frequentist to provide a more flexible modelling approach and enable results to be interpreted intuitively through the reporting of the probability of parameter values, given the observed data (Kruschke & Liddell, 2018). Effect sizes and their sampling variance were calculated using group mean and standard deviation values calculated pre-training and at any subsequent time-point. The SMD_pre and RR effect sizes and their sampling variances ${\sigma }^2$ were calculated using the following formulae (Lajeunesse, 2011, 2015; Morris & DeShon, 2002):

$\mathrm{S}\mathrm{M}{\mathrm{D}}_{\mathrm{p}\mathrm{r}\mathrm{e}}=\left(1-\frac{3}{4n-5}\right)\left(\frac{{\bar{x}}_{Post}-{\bar{x}}_{Pre}}{S{d}_{Pre}}\right)$

where $n$ is the number of participants in the training and the first term comprises a small-study bias term $c\left(n-1\right)$, where $c\left(n-1\right)=1-\frac{3}{4\left(n-1\right)-1}.$

${\sigma }^2\left(\mathrm{S}\mathrm{M}{\mathrm{D}}_{\mathrm{p}\mathrm{r}\mathrm{e}}\right)=c{\left(n-1\right)}^2\left(\frac{n-1}{n\left(n-3\right)}\right)\left(2\left(1-r\right)+n{\mathrm{S}\mathrm{M}\mathrm{D}}_{\mathrm{p}\mathrm{r}\mathrm{e}}^2\right)-{\mathrm{S}\mathrm{M}\mathrm{D}}_{\mathrm{p}\mathrm{r}\mathrm{e}}^2$

where $r$ is the correlation between repeated measures.

$RR=ln\left(\frac{{\bar{x}}_{Post}}{{\bar{x}}_{Pre}}\right)+\frac{1}{2}\left(\frac{S{d}_{Post}^2}{n{\bar{x}}_{Post}^2}-\frac{S{d}_{Pre}^2}{n{\bar{x}}_{Pre}^2}\right)$

${\sigma }^2\left(RR\right)=\frac{S{d}_{Post}^2}{n{\bar{x}}_{Post}^2}+\frac{S{d}_{Pre}^2}{n{\bar{x}}_{Pre}^2}-\frac{2rS{d}_{Post}S{d}_{pre}}{n{\bar{x}}_{Post}{\bar{x}}_{pre}}$

Percentage improvement (e.g., positive value represents improvement and negative value represents a decline in performance) was then calculated using the following formulae depending on whether an increase (+) or decrease (-) in the outcome represented an improvement in performance.

$\mathrm{\%}\mathrm{I}\mathrm{m}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{v}\mathrm{e}{=}_{\begin{array}{c}+\\ -\end{array}}\left\{100\left(exp\left(RR\right)-1\right),-100\left(1-exp\left(RR\right)\right)\right\}.$

.

Prior to conducting full meta-analysis models, the tails of the empirical distributions were investigated by focusing on the most extreme 1%, 2% and 5% of values for both the SMD_pre and %Improve effect sizes. All meta-analyses were conducted using a nested four-level Bayesian mixed effects meta-analytic model (Jones et al., 2018). The series of nestings included the individual study (level 4), the outcome (level 3), the measurement occasion (level 2) and the sampling variance (level 1). Where studies included multiple time points following baseline (e.g., level 2) effect sizes were calculated for each time point relative to baseline and included in the model. To account for uncertainty in ${\sigma }^2$ due to non-reporting of $r$, the values were allowed to vary and were estimated by including an informative Gaussian prior approximating correlation values centred on 0.7 and ranging from 0.5 to 0.9. Variance partition coefficients (VPCs) were used to quantify the relative variance explained across the different levels of the hierarchy, with addition of VPCs used to estimate the expected (population) correlation between two randomly chosen elements within the same nesting structure (Hox et al., 2018). The parameters obtained from the meta-analysis models were then used to calculate small, medium and large threshold values for each of the outcome domains. This was achieved by generating posterior predictions from each meta-analysis model and calculating the 0.25-, 0.5-, and 0.75-quantiles. Posterior predictions used the posterior sample for the model parameters to simulate new data. Weakly informative Student-t prior and half Student-t priors with 3 degrees of freedom were used for intercept and variance parameters (Gelman, 2006). Outlier values were identified by adjusting the empirical distribution by a Tukey $g$-and-$h$ distribution and obtaining the 0.0035- and 0.9965-quantiles, with values beyond these points removed prior to further analysis (Verardi & Vermandele, 2018). Meta-analyses were performed using the R wrapper package brms interfaced with Stan to perform sampling (Bürkner, 2017). Convergence of parameter estimates was obtained for all models with Gelman-Rubin R-hat values below 1.1 (Gelman et al., 2014).

To build prior distributions for each outcome domain, the posterior mean and standard deviation (calculated as the square root of the sum of variance components across levels 2 to 4) obtained from the meta-analysis models along with their credible intervals (mean: 0.025 to 0.975-quantile; standard deviation: 0.125 to 0.875-quantile) were collected. An expanded grid optimisation search was used to select a mean and standard deviation value to represent the normally distributed prior $\left(\theta \sim \mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}\left({\theta }_0,{\sigma }_0^2\right)\right)$ across the credible intervals identified. For each point on the grid, the mean and standard deviation value was used to calculate the quantile value of the small, medium and large thresholds previously identified. A least squares approach was then used with the cost function equal to the squared sum of the differences between the quantile values collected and the corresponding 0.25, 0.5 and 0.75 reference values. Finally, a supplementary file was created so that the prior distributions calculated could be combined with data from future S&C training to produce posterior distributions (Jones et al., 2018) and probabilistic information on whether the training exceeds the context-specific small, medium and large thresholds. With new data, the Bayesian updating is achieved by calculating the effect size $E{S}_{\mathrm{n}\mathrm{e}\mathrm{w}}$ and standard error. The standard error is then transformed into a standard deviation of the participant-level outcome $\sigma$ using

${\sigma }_{{\mathrm{e}}_{\mathrm{n}\mathrm{e}\mathrm{w}}}^2=\frac{{\sigma }^2}{n_{new}}$

where $n_{new}$ is the number of participants in the training of interest. The prior variance ${\sigma }_0^2$ is then re-expressed so that the amount of information contained in the prior distribution is equivalent to training with $n_0$ participants where

${\sigma }_0^2=\frac{{\sigma }^2}{n_0}.$

The Bayesian updating for the posterior distribution of the effect size $\theta$ is then achieved by using the following formula (Jones et al., 2018)

$\theta |E{S}_{new}\sim \mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}\left(\frac{n_0{\theta }_0+n_{new}E{S}_{new}}{n_0+n_{new}},\frac{{\sigma }^2}{n_0+n_{new}}\right).$

3. Results

The search strategy returned 110,662 records which reduced to 2108 studies following deduplication and title screening. Following abstract screening this reduced to 973, and 706 following full-text screening. A total of 643 studies featured the required data to be included in the meta-analyses, generating a total of 6574 effect sizes. A list of the 643 studies is included in the supplementary file. Most studies comprised untrained participants (59.6%), followed by recreationally trained (33.6%) then highly trained (6.8%). The most popular training modes were standard resistance training (33.8%), followed by combined (e.g., resistance and power, 28.5%) and plyometric (12.4%). Most training comprised three sessions per week (42.2%), followed by two sessions (37.8%) then four sessions (10.5%). The majority of training included up to six exercises per session (73.5%), for three (36.6%) or four sets (25.2%). Comparison of gender identified that most studies were comprised solely of males (66.8%), followed by a similar number of studies comprising both males and females (17.1%), and then females only (16.1%). Information describing the distribution of study characteristics are presented in Table 1. Most training lasted between 6 and 12 weeks, with 25 studies (3.9%) including training longer than 25 weeks, and 14 studies including training longer than 52 weeks (2.2%). The median number of outcomes extracted from studies was 4 with interquartile range (IQR): 2–7. The median number of groups included in studies was 2 with IQR: 2–3. The most common outcome domain was maximum strength (2588 effect sizes from 419 studies), and the least common was power (1183 effect sizes from 203 studies).

Table 1. Distribution (percentiles) of study characteristics and overview of outcome domains.

Study characteristic	0%	10%	20%	30%	40%	50%	60%	70%	80%	90%	100%
Participants per study	4	14	17	19	21	24	28	32	37	51	177
Participants per group	4	7	8	9	10	11	12	14	16	21	72
Mean Age	14.0	16.7	18.9	20.0	20.4	21.0	22.0	23.0	23.8	25.0	60.0
Publication Year	1962	1996	2005	2008	2010	2012	2014	2015	2016	2017	2018
Intervention duration (weeks)	4	5	6	6	8	8	8	10	12	14	520
Outcome domain			Number of studies (%)					Number of effects (%)
Maximum strength			419 (65.2)					2588 (39.4)
Jump			379 (58.2)					1552 (23.6)
Sprint			254 (39.5)					1251 (19.0)
Power			203(31.6)					1183 (18.0)

Prior to applying the meta-analytic model, the tails of the empirical values were investigated, with each demonstrating long right tails (Table 2). Pooled across all outcome domains, the 0.975-, 0.9875- and 0.995-quantiles for SMD_pre were equal to 3.2, 4.8 and 8.6, respectively; and for %Improve were equal to 45.9%, 59.6% and 88.2%, respectively. Similar long right tails representing implausible values were identified across all outcome domains for SMD_pre but were restricted primarily to maximum strength for %improve (Table 2). Data were then prepared for meta-analysis and a total of 123 outliers were removed with lower bound SMD_pre and %improve thresholds of −0.91 and −13.4%, and upper bound SMD_pre and %improve thresholds of 6.7 and 87.6%. Effect size estimates including small, medium and large thresholds are presented in Figures 1 and 2. Numerical values and summaries of variance parameters are presented in the supplementary table. Pooled across all outcome domains the threshold value estimates for SMD_pre equalled small: 0.16 [95%CrI: 0.15 to 0.18], medium: 0.46 [95%CrI: 0.45 to 0.48], and large: 0.81 [95%CrI: 0.79 to 0.83]; and for %Improve equalled small: 2.4 [95%CrI: 2.1 to 2.6%], medium: 7.8 [95%CrI: 7.5 to 8.0%], and large: 15.1 [95%CrI: 14.7 to 15.4%]. Comparisons across outcome domains identified large differences in threshold values for both effect size statistics with the greatest values obtained for maximum strength, followed by similar values for power and jump performance, and finally, substantively smaller values for sprint performance (Figure 2).

Table 2. Direct calculation of largest and smallest 1%, 2% and 5% of effect sizes across outcome types.

Outcome	Statistic	0.005-Quantile	0.0125-Quantile	0.025-Quantile	0.5-Quantile	0.975-Quantile	0.9875-Quantile	0.995-Quantile
All	SMDpre	−0.86	−0.55	−0.32	0.49	3.2	4.8	8.6
	%Improve	−15.0%	−7.3%	−4.1%	6.9%	45.9%	59.6%	88.2%
Maximum Strength	SMDpre	−0.84	−0.43	−0.23	0.62	4.0	6.0	8.6
	%Improve	−17.0%	−9.2%	−5.0%	12.4%	59.9%	84.2%	113.9%
Jump performance	SMDpre	−0.75	−0.38	−0.25	0.47	2.5	3.3	8.0
	%Improve	−11.2%	−5.9%	−3.5%	6.3%	27.6%	34.8%	42.2%
Sprint Performance	SMDpre	−0.90	−0.71	−0.51	0.38	3.1	5.3	11.2
	%Improve	−5.3%	−3.4%	−2.6%	2.1%	19.4%	40.5%	60.8%
Power	SMDpre	−0.78	−0.58	−0.32	0.36	2.2	4.0	8.9
	%Improve	−17.1%	−11.3%	−5.0%	6.6%	38.8%	45.9%	53.9%

Figure 1. Empirical distribution and modelled outcome-specific standardised mean difference effect size thresholds.

(Below): Black curve is a density plot of the directly calculated empirical effect size values across all outcomes. Small, mid, and large thresholds represent the 0.25, 0.5 and 0.75-quantiles of predicted draws. Black diamonds are based on direct calculation and red intervals illustrate uncertainty in estimates through the median value (circle) and 95% credible interval.

Figure 2. Empirical distribution and modelled outcome-specific percentage improvement effect size thresholds.

(Below): Black curve is a density plot of the directly calculated empirical effect size values across all outcomes. Small, mid, and large thresholds represent the 0.25, 0.5 and 0.75-quantiles of predicted draws. Black diamonds are based on direct calculation and red intervals illustrate uncertainty in estimates through the median value (circle) and 95% credible interval.

Mean and standard deviation values for future normal distribution SMD_pre and %improve priors are presented in Table 3. The large difference in means across outcome domains reflected the same ordering observed across small, medium and large thresholds. However, consistently the standard deviation was similar in size to the mean, demonstrating the large variability in results that should be expected based on the training and potentially population characteristics.

Table 3. Mean and standard deviations of prior distributions for standardised mean difference (SMD_pre) and log transformed relative ratio (RR) effect sizes statistics across outcome domains.

		Prior mean	Prior standard distribution
Outcome	Statistic
All	SMDpre	0.50	0.44
	Log(RR)	0.081	0.083
Maximum Strength	SMDpre	0.62	0.53
	Log(RR)	0.128	0.105
Power	SMDpre	0.39	0.41
	Log(RR)	0.074	0.079
Jump performance	SMDpre	0.45	0.40
	Log(RR)	0.066	0.060
Sprint Performance	SMDpre	0.38	0.47
	Log(RR)	0.026	0.034

4. Discussion

The present study comprises one of the largest meta-analyses in sport and exercise science and the largest synthesis of contemporary S&C training to date. In general, the analyses showed similar findings for SMD_pre and %Improve, with the greatest effect sizes obtained for maximum strength outcomes, and substantively smaller values for sprint performance. For both effect sizes, the positive tails of the empirical distribution had the potential to exhibit extremely large values. These large values were generally restricted to maximum strength outcomes for %Improve (e.g., ~60 to 110% improvement) but were consistently large and physiologically implausible for SMD_pre across all outcome domains (~4 to 11). Development of Bayesian prior distributions resulted in relatively large spreads with standard deviation values close to the mean, and for sprint performance standard deviations were greater in value, which was consistent with the finding that a substantive proportion of the distribution included effect sizes close to zero.

Cohen’s (Cohen, 1988) generic threshold values (small: 0.20, medium: 0.50, and large: 0.80) were broadly in line with SMD_pre values obtained from the current meta-analysis when data were pooled across all outcome domains (e.g., small ~ 0.15, medium ~ 0.45, large ~ 0.80). In contrast, the small, medium and large thresholds for %Improve pooled across all outcome domains were approximately 2.5%, 8.0% and 15.0%. However, the present study identified large differences in effect size distributions for both statistics when outcome domains were analysed separately, indicating that context relevant threshold values are important to appropriately interpret the magnitude of change following S&C training. The greatest values were obtained for maximum strength outcomes, followed by similar values for power and jump performance, and finally, substantively smaller values for sprint performance. For maximum strength, the SMD_pre small, medium and large thresholds were equal to 0.25 [95%CrI: 0.22 to 0.27], 0.59 [95%CrI: 0.56 to 0.61] and 0.98 [95%CrI: 0.94 to 1.0], respectively. In contrast, expressed as a percentage improvement the thresholds were equal to 6.0% [95%CrI: 5.5 to 6.5], 13.4% [95%CrI: 12.8 to 14.1] and 22.2% [21.6 to 22.9] which are arguably easier to interpret. However, the greatest conceptual difference between the two effect sizes was evident when comparing thresholds between maximum strength and sprint performance. For sprint performance the small, medium and large SMD_pre thresholds decreased to 0.07 [95%CrI: 0.04 to 0.10], 0.37 [0.33 to 0.40] and 0.70 [0.66 to 0.74]. These results show that a substantive proportion (~15 to 20%) of the effect size distribution are close to or below zero, whereas the large sprint performance threshold is between the medium and large maximum strength thresholds. In contrast, the percentage improvement thresholds for sprint performance were equal to 0.4% [95%CrI: 0.2 to 0.6], 2.2% [95%CrI: 2.1 to 2.4] and 4.5% [4.2 to 4.8], such that all thresholds were below even the small maximum strength threshold. These observations reflect differences in relationships between means and standard deviations for outcome domains and demonstrate the conceptual difference between effect sizes describing expectations of how participants will change their relative position within a population compared to the magnitude of change relative to starting values.

The potential for restricted sampling of a population to bias standardised effect sizes such as the SMD_pre has previously been highlighted (Baguley, 2009). If the sample is a truncated sample (missing one or both tails), then the standard deviation is likely to be underestimated such that the SMD_pre will be positively biased. This scenario is likely to occur in S&C research where random sampling is uncommon and often convenience samples are used, including recruitment from a single team where participants may be relatively homogenous given similar training experiences. In contrast, sampling only from the tails is likely to overestimate the standard deviation (Baguley, 2009) and thereby negatively bias the SMD_pre. In S&C, this situation may occur in studies recruiting both males and females where the outcome variable has a large sex stratification. Analysis of the tails of the empirical distribution in the current study highlight the likely role that restricted recruitment practices have played in overestimation of some SMD_pre values generating physiologically implausible results (~4 to 11) across all outcome domains. The effect sizes we calculated were from a single training group and were not compared to a non-exercise control, it is therefore possible that the values overestimate the magnitude of change following training due to issues such as placebo and learning effects. Additionally, there are important limitations of pre-post designs compared to those that include a concurrent control (e.g., a standard training regime), and the subsequent comparative effect sizes they generate enabling inference regarding the causal effect of training regimes of interest.

The approach we used enabled more precise estimates of effect sizes and is more robust to overestimation due to sampling practices. One of the primary challenges and biggest criticisms of Bayesian methods has been the selection of appropriate priors (Goodman, 1999; Mengersen et al., 2016). Where substantive and relevant external information is present, attempts should be made to incorporate this within an informative prior (Mengersen et al., 2016). One of the most effective sources of information to build priors to better assess the magnitude of change following future training includes meta-analyses such as that presented here (Jones et al., 2018). An additional challenge in the effective use and uptake of Bayesian methods is a lack of formal training and familiarity with approaches (Bernards et al., 2017). We have made attempts to address both challenges by firstly, creating priors that are based on a large volume of research focusing on the outcome domains frequently used in S&C research; and secondly, employing a relatively simple Bayesian updating method that can be understood intuitively and facilitated in software that is familiar with both researchers and practitioners. The method adopted expresses both the prior and posterior distribution of the effect size as normal distributions that are familiar and simple to assess suitability by examining stated probabilities. For example, based on the meta-analysis results obtained here, an SMD_pre prior with mean 0.62 and standard deviation of 0.53 was developed for maximum strength outcomes. This asserts that the prior probability of obtaining an SMD_pre value greater than 0 is $p$ = 0.879 (e.g., 1 $-\mathrm{\Phi }$ ($-$ 0.62/0.53)), the probability of obtaining an SMD_pre value between 0 and 0.5 is $p$ = 0.289 ($\mathrm{\Phi }$ ((0.5 $-$ 0.62)/0.53) $-\mathrm{\Phi }$ ($-$ 0.62/0.53))), and the probability of obtaining an SMD_pre value greater than 1 is = 0.237 (1 $-\mathrm{\Phi }$ ((1 $-$ 0.62)/0.53)). A researcher and practitioner can decide to alter the mean and standard deviation if they believe the probabilities returned do not match their prior beliefs, which may be the case when considering elite athletes where positive changes are less certain. The values presented here, however, provide useful anchors as they were designed to fit an extensive amount of data collected from S&C training. Similarly, the updating process used to combine prior information with data collected and generate posterior distributions is also easily interpreted. First, the method (Jones et al., 2018) updates the posterior mean as a weighted combination of the prior mean and the effect size calculated directly from the training. The weights are determined by the uncertainty in the estimate from the data, and where for example, a small number of participants are investigated, the standard error will be large and therefore greater weight placed on the prior mean. The exact weights used are determined by matching the uncertainty in the new data and the prior, and translating the information contained in the prior to a single trial that can then be updated with the new data (Jones et al., 2018).

To demonstrate how potential issues with regard to poor estimates of the standard deviation can be addressed, the example outlined above is continued. If we assume a correlation between the pre- and post-training scores of 0.7 (a requirement to calculate uncertainty in the estimate), then combining the extremely large SMD_pre value of 4.2 with the small sample size of n = 8 generates a standard error of 1.41. Based on a frequentist approach, a 95% confidence interval for the effect size would equal 4.2 ± 1.96 $\times$ 1.41 giving a range of 1.4 to 7.0. However, given the small sample size and the large standard error, when updated in a Bayesian framework using the methods presented here and the equations in the statistical analysis section, the posterior mean and standard deviation are shrunk to 1.1 and 0.50, respectively. The effect size is still considered large but is now more plausible and can be interpreted probabilistically given the normal distribution and posterior parameters estimated (e.g., probability of at least a small change: $p$ = 0.955; probability of at least a medium change: $p$ = 0.846; and probability of at least a large change: $p$ = 0.595). Note, if the sample size was much larger, say $n$ = 100, then the directly calculated effect size increases to 4.7 (due to a reduction in the bias offset) with 95% confidence interval giving a range of 4.0 to 5.4, and the posterior mean is only shrunk to 3.5, as there is less uncertainty in the original estimate.

5. Conclusion

To assist practitioners in selecting and developing training using evidence-based practices, it is important that processes and tools are available to compare and appropriately interpret results disseminated in research. Currently, the use of effect size statistics provides the most practical method of ranking and determining which training regimes are most likely to provide a basis for the greatest improvements within a given population. There are multiple effect size statistics that can be used, each with their own strengths and weaknesses. However, regardless of the effect size statistic used, clear patterns emerge when evaluating previous research in S&C including substantive differences in distributions across outcome domains, particularly between maximum strength and sprint performance. Whilst it should be acknowledged that there are limitations in categorising continuous variables, including the generation of different qualitative interpretations of very similar values at the boundaries, the approach is widely used and can provide important insights. Knowledge that different outcome domains can generate large differences in effect size distributions has several important consequences. Firstly, interpretations on the magnitude of change following training can be greatly influenced. For example, using previous non-S&C specific thresholds, researchers and practitioners may interpret training regimes focused on developing sprint performance as being unsuitable when a more complete understanding highlights that these improvements may be comparable with magnitudes of change commonly observed, and therefore the training appropriate to use with a given population. Second, knowledge of effect size distributions has important implications for setting sample size requirements for future research studies. Effect size thresholds are commonly used for power calculations using frequentist methods and suggest that smaller sample sizes may be required for training aimed at developing maximum strength compared with training aimed at developing sprint performance. For example, using the medium effect sizes presented here and a simple one sample t-test with $\alpha$ set at 0.05, approximate sample sizes of 21 or 49 would be required for training aimed at developing maximum strength or sprint performance, respectively (Faul et al., 2009). It is important to note, effect sizes for comparative studies investigating potential differences between training regimes are likely to be considerably smaller than those presented here and therefore require greater sample sizes. In contrast, sample size approaches using Bayesian methods can be conducted using the prior distributions presented here (Jones et al., 2018).

Given the large volume of S&C research and the pace at which it is accelerating, there are potential advantages to incorporating this information within future research to make better estimates, particularly where small sample sizes are common and effect sizes may be low (Mengersen et al., 2016). Bayesian methods are well suited to this process, and it is likely that as more disciplines and research in general take advantage of the potential benefits associated with Bayesian frameworks and criticisms of null hypothesis significance testing continues to grow (Wasserstein et al., 2019), increased uptake will occur. Valid criticisms of Bayesian frameworks remain and the use of relatively strong priors for contexts where they are not suited are likely to lead to poor inferences and the description of certain training as more effective than they are. To facilitate an increased use of Bayesian methods processes are required to address two of the main challenges which include development of appropriate priors and accessible tools and procedures that are intuitive and can be carried out ideally without need of complex software. The present study has attempted to address these challenges.

Author contributions

PAS and AM designed the research. AM conducted the searches and screening. AM, LG, JP and AH extracted the data. PAS performed all statistical analyses and developed the supplementary file. PAS and AM interpreted the data analysis. PAS and AM wrote the manuscript with critical input from KB, LG, JP, AH, PM and RA. All authors read and approved the final manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed online https://doi.org/10.1080/02640414.2022.2128548.

Additional information

Funding

The authors reported there is no funding associated with the work featured in this article.