Because ‘happy sheets’ are not enough – a meta-analytical evaluation of a personnel development program in academia

ABSTRACT

Although the high costs of implementing personnel development programs in enterprises and increasingly in universities as well are commonly accepted, the scientifically-grounded evaluation of a program’s effectiveness is often neglected. The aim of this paper was to evaluate a personnel development program for academics at a German university (N=1426). We investigated the program’s effectiveness on Kirkpatrick’s four levels. We examined short- and long-term effects with a repeated measurements design. In sum, 123 training programs were clustered into 26 primary studies. We conducted t-tests and aggregated the effect sizes (d) using meta-analytical techniques. We found a high short-term effect and a moderate to high long-term effect for the learning level, and a small to moderate long-term effect for the behavior level. Participants’ organizational commitment did not increase. Our results have appealing practical implications for university HR managers, trainers, and evaluators.

KEYWORDS:

• Training evaluation
• personnel development in academia
• Kirkpatrick
• meta-analysis

Acknowledgements

We want to thank Dr. Marion Schmidt-Huber and Dr. Barbara Pangert for their contribution to our manuscript, as well Lisa Weihrauch, Stefan Tretter, Christina Schober, and Anika Klein for their valuable help in data collection. We would especially thank Tatjana Sonderwald and all the trainers and participants for their support.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 $d_z={\displaystyle \frac{{\overline{x}}_2-{\overline{x}}_1}{{\hat{\mathrm{\partial }}}_D}}$ where ${\overline{x}}_1$ is the mean at measurement point 1, ${\overline{x}}_2$ is the mean at measurement point 2, ${\hat{\mathrm{\partial }}}_D$ is the standard deviation of the difference in means taking the correlation between the two measures into account ${\hat{\mathrm{\partial }}}_D={\sqrt{{\hat{\mathrm{\partial }}}_1^2+{\hat{\mathrm{\partial }}}_2^2-2\ast r_{12}\ast {\hat{\mathrm{\partial }}}_1\ast \hat{\mathrm{\partial }}}}_2$ (Dunlap et al. 1996; Bühner and Ziegler 2009).

2 $d_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}}=d_z\left[1-{\displaystyle \frac{3}{4N-9}}\right]$ where N is the sample size of the t-test corresponding to the calculated effect size (Hedges and Olkin 1985, 81).