Explaining and enhancing training transfer: a consumer-centric evaluation of a leadership training

ABSTRACT

This study presents a consumer-centric approach to training evaluations by providing an integration of transfer research and testing the resulting model with data from an evidence-based leadership training. A longitudinal field study with four measurement points was used to test whether evidence-based leadership training resulted in short- and long-term training effectiveness. To explain the underlying mechanisms, the impact of the postulated primary determinants (learning, transfer motivation, and opportunity) and the central secondary determinant (training design) on transfer were tested. Self-ratings from a sample of 62 leaders of a German middle-sized organization were used. There were training effects immediately after training and 10 months later – also with regard to follower ratings. Regression analyses revealed that all three primary transfer determinants predicted transfer. However, only learning incrementally affected transfer when controlling for the other determinants. Nevertheless, the full model substantially explained long-term near (adj. R² = .47) and far (adj. R² = .59) transfer. Perceived transfer-oriented training design emerged as central secondary transfer determinant for all three primary determinants. Results are discussed in terms of the result-oriented question of whether evidence-based training pays off on an organizational level and the process-oriented question of which determinants are essential for transfer at the individual level.

KEYWORDS:

• Training effectiveness
• training evaluation
• transfer of training
• learning
• transfer motivation
• transfer opportunity

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Authors such as Holton (2005) defined only self-efficacy and learner readiness as secondary determinants and subsumed transfer opportunity under ‘ability’.

2. Of the 62 leaders, 10 leaders had one follower rating, 20 leaders had two follower ratings, one leader had three follower ratings, and one leader had four follower ratings. When there were several follower ratings, the mean of the follower ratings was used.

3. In the questionnaires the untransformed scale of −2 (has greatly deteriorated) to +2 (has greatly improved) was used.

4. The effect size measures are Pearson correlation coefficients $r=\sqrt{\frac{t^2}{({\mathrm{t}}^2+\mathrm{d}\mathrm{f})}}$.

5. Overall, 39 leaders took part in 2014, divided into four training groups. Additional 23 leaders were trained in 2016, divided into two training groups. This enabled additional wait-list-control-group analyses of pre-post measures. The group mean comparisons validated the results of Hypothesis 1: Leaders who were trained early revealed significantly higher post-measures regarding transfer motivation and learning as well as marginally significantly higher post-measures of near and far transfer compared with pre-measures of leaders who were trained later.

6. Subsequent bootstrapping analyses with bootstrap samples of N = 1,000 and a CI level of 95% confirmed the results for Hypotheses 2a and 2b.

7. A subsequent bootstrapping analysis with bootstrap samples of N = 1,000 and a CI level of 95% confirmed the significant effect of learning even in the complete model on near transfer – but not on far transfer.

8. Subsequent bootstrapping analyses with bootstrap samples of N = 1,000 confirmed these results.

9. As depicted in the theoretical model (see Figure 1) there might also be an indirect effect of transfer training design on near and far transfer via the primary determinants (learning, transfer motivation, and transfer opportunity). We therefore also calculated exploratory parallel mediation analyses using the PROCESS macro by Hayes (2013; Version 3.00, bootstrap samples of N = 5000). Transfer training design did not have a significant effect via learning (effect = .61, SE = .31; CI90% = [−.04; 1.00]), transfer motivation (effect = −.00, SE = .40; CI90% = [−.61; .65]), or transfer opportunity (effect = .31, SE = .32; CI90% = [−.18; .86]) on near transfer (N = 25). Regarding far transfer (N = 34), there was a significant indirect effect of transfer training design via learning (effect = .21, SE = .11; CI90% = [.02; .38]). However, transfer training design did not have a significant effect via transfer motivation (effect = −.07, SE = .11; CI90% = [−.25; .10]) or transfer opportunity (effect = .15, SE = .11; CI90% = [−.01; .34]) on far transfer. Due to the small sample size the results must be interpreted with caution as a type II error is possible.

10. With the final sample of 25 to 41 participants for repeated measures, a posteriori power analyses revealed moderate to high power between .70 and .96 – based on the assumption of large effect sizes (f² ≥ .35). Exceptions are the regression models that tested all three direct predictors simultaneously, with lower power ranging from .60 to .77.