Deepening transparency about value-laden assumptions in energy and environmental modelling: improving best practices for both modellers and non-modellers

ABSTRACT

Transparency and openness are broadly endorsed in energy and environmental modelling and analysis, but too little attention is given to the transparency of value-laden assumptions. Current practices for transparency focus on making model source code and data available, documenting key equations and parameter values, and ensuring replicability of results. We argue that, even when followed, these guidelines are insufficient for achieving deep transparency, in the sense that results often remain driven by implicit value-laden assumptions that are opaque to other modellers and researchers, and that may not be understood by wider audiences to be controversial. This paper identifies additional best practices for achieving transparency by highlighting issues where disagreement over value judgements will persist for the foreseeable future. Recommendations for deepening transparency are developed by learning from successes and ongoing challenges represented in three case studies. We provide recommendations to accelerate the adoption of additional best practices for deepening transparency of energy and environmental modelling in policy-relevant domains, increasing stakeholder participation with non-modellers, and encouraging interdisciplinary dialogue.

Key policy insights

• Achieving all of the goals associated with transparency requires more than current practices of providing open source data, code, and model documentation.

• Greater interdisciplinary dialogue could improve transparency beyond current practices, including in model development, application, and communications.

• Better practices for addressing contentious and value-laden assumptions include providing accessible documentation for non-specialists, increasing policymaker participation to ensure that model outputs can inform questions, and performing sensitivity analyses that cover a range of reasonable views about value-laden assumptions.

• Energy and environmental modellers should account for audience-specific considerations to promote transparency, especially accounting for needs of non-modellers such as policymakers.

KEYWORDS:

• Transparency
• modelling
• economic analysis
• uncertainty
• values
• energy and environmental policy

Acknowledgments

The authors wish to thank three anonymous reviewers and the editor for their helpful suggestions. The views expressed in this paper are those of the authors alone and do not necessarily state or reflect those of EPRI, Rutgers University, or University of Maryland, Baltimore County, and no official endorsement should be inferred.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Recommendations include getting consent from intellectual property holders, choosing a platform, identifying publishable content, choosing a license, considering programming languages and frameworks, and building a community of users.

2 The replication crisis in disciplines like the social sciences have led to recommendations for better practices and changing research norms (Gertler et al., 2018), and although many suggestions are applicable for energy and environmental research, replicability is only a subset of transparency-related issues associated with prospective structural models that are the focus of this manuscript.

3 See Emmerling et al. (2019) for further discussion on the effect of the discount rate for setting climate change policy targets.

4 This discussion focuses on quantitative evaluations of uncertainty. However, we acknowledge much could be learned by improving collaborations between quantitative and qualitative research projects. Qualitative methods designed to engage broader stakeholder perspectives in the context of multiple uncertainities through, for example, NUSAP (Numeral Unit Spread Assessment Pedigree) workshops can also improve transparency and make value-laden assumptions more evident to in energy and environmental assessments (Van Der Sluijs et al., 2005; Petersen et al., 2011; Pye et al., 2018). However, we would argue that these projects fall short of deepening transparency by focusing more on stakeholder involvement than on collaboration between research groups and across disciplines.

5 For discussion of intra-country scale distributional analysis concerning the transition to low carbon energy, see Fell et al. (2019) and Oswald et al. (2020).

6 A compelling case could be made that there are too many energy models (Sarewitz, 2018) and that requiring open-source code will increase the cost of creating new models and therefore decrease the supply. However, while larger and well-resourced incumbents would likely be able to comply with these restrictions (Lange & Redlinger, 2019), the models that are more likely to be materially impacted would be smaller groups with more limited resources, and there may be strong equity arguments in favour of having these perspectives included in broader dialogues about energy and sustainability.

7 Governments may also have a role to play in improving model documentation requirements. For example, the UK has created a quality assurance manual that applies to government analysis (H.M. Treasury, 2015).