Making evaluation work for the practitioner evaluator: experience from the field of environmental education

Abstract

Drawing on work from environmental education, evaluation, and practitioner research, this paper presents a discussion of programme evaluation practices from the practitioner’s perspective. This discussion is informed by a meta-evaluation conducted on ten-years (2008–2018) of data collected from a suite of place-based learning programmes delivered by a charity in Ireland. Analysis of the data available allowed for an evaluation of the programmes in question, and these were then further analysed to provide the meta-evaluation. Key findings and discussions from the analysis include the impact of time constraints on evaluation; evaluation as part of organisational culture; strengths/weaknesses of evaluations led by practitioner evaluators; and opportunities provided by meta-evaluation in directing organisational change. This paper serves as a valuable resource for researchers and practitioners as it provides a framework to support future evaluations led by practitioner evaluators.

Keywords:

• Evaluation
• practitioner research
• meta-evaluation
• place-based learning
• impact measurement

Disclosure statement

No potential conflict of interest was reported by the authors.

Acknowledgements

This work was supported by the Irish Research Council Employment Based Postgraduate Programme and the Burrenbeo Trust. Our thanks to the Editor and Reviewers for their encouraging suggestions and comprehensive reviews which greatly enhanced this paper.

Notes

1 Primary school is Irish first level education, generally attended by children from the age of 4 or 5 until 12 or 13. Secondary school is Irish second level education, attended by young people from the age of 12 or 13 until 16 to 18.

2 These reflections around the role of PhD candidate as ‘meta-evaluator’ with academic supervisors and graduate committees reviewing their work and evaluation, are particularly interesting in light of Scriven’s (2009, p. vi) work which calls for improved forms of peer review and highlights that ‘choosing a meta-evaluator requires the same integrity that all evaluation requires’.

3 Due to the data available and the constraints of this study, a number of potential areas of interest or enquiry were not possible such as how the identified themes changed over time, how the programmes changed or how individuals (and their engagement) changed over time.

4 Ethical approval for this study was granted by Research Ethics Committee of the National University of Ireland, Galway. The data in the feedback archive is anonymous.