A holistic model for evaluating the impact of individual technology-enhanced learning resources

Abstract

Background: The use of technology within education has now crossed the Rubicon; student expectations, the increasing availability of both hardware and software and the push to fully blended learning environments mean that educational institutions cannot afford to turn their backs on technology-enhanced learning (TEL). The ability to meaningfully evaluate the impact of TEL resources nevertheless remains problematic.

Aims: This paper aims to establish a robust means of evaluating individual resources and meaningfully measure their impact upon learning within the context of the program in which they are used.

Methods: Based upon the experience of developing and evaluating a range of mobile and desktop based TEL resources, this paper outlines a new four-stage evaluation process, taking into account learner satisfaction, learner gain, and the impact of a resource on both the individual and the institution in which it has been adapted.

Results: A new multi-level model of TEL resource evaluation is proposed, which includes a preliminary evaluation of need, learner satisfaction and gain, learner impact and institutional impact. Each of these levels are discussed in detail, and in relation to existing TEL evaluation frameworks.

Conclusions: This paper details a holistic, meaningful evaluation model for individual TEL resources within the specific context in which they are used. It is proposed that this model is adopted to ensure that TEL resources are evaluated in a more meaningful and robust manner than is currently undertaken.

Acknowledgements

The authors wish to thank Dr Rebecca O’Rourke and Tamsin Treasure-Jones for their thoughtful comments on drafts of this paper.

Disclosure statement

We have no declaration of interest to report.

Notes on contributors

James D. Pickering, BSc (Hons), PhD, PGCLTHE, SFHEA, is an Associate Professor of Anatomy in the Division of Anatomy, Leeds Institute of Medical Education, School of Medicine, University of Leeds, UK.

Viktoria C. T. Joynes, BA (Hons), MSc, PhD, is the Deputy Director of Studies and Lecturer in Medical Education at the School of Medicine, University of Liverpool, UK. The work discussed in this paper was undertaken during her previous role as Mobile Learning Implementation, Evaluation and Development Officer at the Leeds Institute of Medical Education, School of Medicine, University of Leeds, UK.