Evaluating environmental policy: the use and usefulness of experiments

ABSTRACT

This paper reflects on the use and usefulness of experiments for environmental policy evaluation. Whereas most of the literature has focused on the added value of field experiments, this paper also considers the added value of choice and lab experiments. The paper reviews the literature to assess the potential of the different methods, focusing on the type of policy issues that can be evaluated with the help of experiments. It then discusses validity issues, and how the validity of the different methods can be improved, after which it turns to the ‘policy validity’ or generalizability of the outcomes of the different methods, crucial for policy relevance. The paper ends with a reflection on how the use and usefulness of experiments for environmental policy evaluation can be improved, concluding that mixed method approaches that combine the different experimental methods seem most promising although efforts, to enhance the replicability of experiments and the building of an evidence base, are also important. Finally, to enhance the use and usefulness of experiments for environmental policy making, it is important that more attention is paid to the scalability of the experimental findings and for the inclusion of policy context in experimental design.

KEYWORDS:

• Field experiments
• choice experiments
• lab experiments
• environmental policy evaluation
• validity
• scalability

Acknowledgements

The author would like to acknowledge the role of Kees Vringer, Daan van Soest, Erik Ansink, Mark Koetse and Mirthe Boomsma in the realization of this paper. Discussions and joint projects with Daan, Erik, Mark and Mirthe helped to shape the ideas presented in the paper, and the feedback of Kees helped to improve the paper’s structure. All remaining errors are the author’s responsibility.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Such a combination of methods also fits the policy reality that most interventions are no single policy interventions but rather a mix: Bouma et al. (2019b) illustrate how this has complicated the classical approach to environmental policy evaluation. Using combinations of (experimental) evaluation methods helps to evaluate the policymix.