Raising the voice of science in complex socio-political contexts: an assessment of contested water decisions

ABSTRACT

Agencies are increasingly developing evidence-based policies to manage natural resources. However, the influence of science in policy is not straightforward nor guaranteed. Critiques based on literature meta-analyses or policy-maker interviews suggest deficiencies in science production and delivery with some studies highlighting the importance of human dimensions. In interviews with decision-makers in freshwater policy in New Zealand and California, we investigated barriers to using science in complex and contested policy contexts. Findings highlighted the importance of the science, scientist, decision-maker, and the decision maker’s relationship with the scientist, for improving the influence of science on policy decisions. The influence depended more on the scientist delivering the information and the audience receiving it, than on the nature of the science itself. Frameworks like CRELE (credibility, relevance, legitimacy) and ACTA (applicability, comprehensiveness, timing, accessibility) are essential but outweighed by the human dimensions of policy development. With greater public, industry and NGO oversight of policy debates related to highly contested resources like water, the volume and quality of science for policy has greatly improved, meaning CRELE and ACTA factors have less prevalence. We give three categories of recommendations for improving the use of science in decision-making – science communication, science production and policy processes.

KEYWORDS:

• Policy
• decision-making
• freshwater management
• environmental policy
• science-policy interface
• water quality

1. Introduction

The role of science in decision-making and its influence in policy development has been a topic of inquiry for decades (e.g. Jasanoff, 1990; Merton, 1945; Pielke, 2004). Despite its long history, we are still questioning how influential science is in policy decisions. The policy process is complex, frequently non-linear and influenced by the values, norms and prejudices of decision-makers and their constituents (Fernandez, 2016; French, 2018; Pielke, 2007). Policy involves deciding among competing demands, and in the environmental policy arena these decisions are generally between human health, ecosystem health, and economic well-being. Therefore, by the very nature of policy decision-making, science is only one component determining the course of action(s) to be taken for a policy decision (Pielke, 2007).

Several authors note that scientists do not understand policy processes. In particular, the non-linearity of the policy process (Fernandez, 2016; French, 2018; Juntti et al., 2009; Owens, 2005), differences in culture and timeframes between science and policy institutions (Cvitanovic et al., 2016; French, 2018; Van Buuren & Edelenbos, 2004), the need to consider multiple perspectives in decision-making (Fernandez, 2016; French, 2018), the multiple types of knowledge involved (Fernandez, 2016; French, 2018; Juntti et al., 2009; Owens, 2005) and the range of interests/stakeholders involved in policy decisions and how this influences power within decision-making (Juntti et al., 2009; Van Buuren & Edelenbos, 2004).

In complex, contested spaces such as freshwater management, the focus of our study, the policy process is particularly challenged by the vested interests of various participants. Here we explore the role of science in policy decisions and what influences the usage of science or science ‘voice’ in complex contested spaces. Given the complexity and noisiness of policy processes and the role of social media in shaping public opinion, it becomes even more important to raise the profile and impact of science in these decisions.

There are several typologies and theories for the role of science and scientists¹ in policy making for complex issues but there is little empirical evidence to verify them (Spruijt et al., 2014). Many frameworks focus on knowledge production/utilisation, one of which is that science that is credible, salient (or relevant) and legitimate (CRELE) for multiple stakeholders will be the most influential (Cash et al., 2002; Heink et al., 2015; Sarkki et al., 2013, 2015; Van Enst et al., 2014). However, Dunn and Laing (2017) question the usefulness of CRELE, noting its evidence is thin and potentially biased by the high level of involvement of research providers. Instead, based on open-ended interviews with policymakers, they found applicability, comprehensiveness, timing and accessibility (ACTA) were better predictors of policy-makers’ use of science to inform policy. A key related factor is the purpose of the science; science is either fundamental (aiming to enhance our understanding of living systems) which is needed to improve predictions or applied (aims to answer specific questions society may be asking), or policy-driven (responding to questions related to an issue). The latter two are most relevant for policy decisions and our study.

Spruijt et al. (2014) assess scientists’ roles in decision-making differently, focusing on the importance of interactions between scientists and policymakers. Using a literature meta-analysis, they identify six factors that influence the roles of scientists in complex policy issues – type of issue (level of uncertainty/complexity), type of knowledge of the scientist, core values of the scientist, scientist’s organisation, context (position of science in society), and changing beliefs of scientists. These factors attest to the credibility/relevance/legitimacy of the science and the values held by the scientist. They further suggest ways to improve how experts (should) advise on complex issues. These include improving the transparency in methods, assumptions etc., adopting a professional attitude of humility, encouraging public participation (e.g. stakeholder dialogues), considering the option of precautionary measures and revealing differing points of view within the expert community.

Based on the literature, we propose four pillars of the science-policy interface to account for the effective use of science for policy decisions – the science, the scientist, the decision-maker and the decision-maker’s relationship and/or interactions with the scientist (Figure 1). These pillars cover the characteristics of the science as well as the human dimensions that may govern its use. These cannot be treated in isolation. To date, frameworks have focused on one or two of these components rather than the interplay of all four. We take a systematic approach to assessing the role of science (and different forms of science) in decisions and the barriers to effective use of science for decision-making using these pillars. We highlight the importance of all four pillars in improving science’s influence on policy. To do this we use the ‘audience’ (i.e. stakeholders and policy-makers) of science to explore the influence of science at the science-policy interface.

Figure 1. Pillars that influence the use of science in policy decisions.

Our context is the complex (and often highly contested) issue of freshwater management in New Zealand and California. Freshwater management is often complex biophysically (e.g. surface- and groundwater interactions; quantity and quality interactions and issues), socially (e.g. diverse and often opposing needs for freshwater resources) and/or administratively (e.g. multiple agencies managing the same resource, or overlapping governance arrangements). It is also often contested where stakeholders have differing values and uses for scarce freshwater resources with agriculture being the major freshwater user and polluter but important for food production and the economy.

Both jurisdictions face freshwater management challenges and the role of science in providing policy advice has been evolving. This drove our exploration of how science is being used in management decisions and to identify what were seen as barriers/enablers to its use and, just as importantly, what decision makers/stakeholders saw as avenues for improving the use of science in policy decisions.

We investigated whether there is commonality among responses from different jurisdictions with similar policy drivers and contexts (within New Zealand) as well as different policy contexts (New Zealand vs. California). In New Zealand, all or part of the decision-making in freshwater policy development has been given to the community,² whereas, in California, more traditional consultative policy development processes are in place. Like Dunn and Laing (2017), we used open-ended, semi-structured questions with policymakers and stakeholder representatives, as users or demanders of science, to identify barriers to using science for policy making and how to improve science influence. Most other assessments use literature meta-analysis or qualitative reviews (e.g. French, 2018; Juntti et al., 2009; Spruijt et al., 2014).

We explore the CRELE and ACTA frameworks for their applicability in explaining how science influences policy in our contexts (science pillar) and identify the human dimensions (scientist and decision-maker and their relationship/interaction) that influence science use in policy decisions. We outline the roles of science identified by stakeholders and decision-makers, and barriers to using science in policy making. We conclude with comments on how to improve the effectiveness of science in noisy policy environments characterised by conflicting stakeholder views on policy approaches.

2. Methodology

2.1. Study areas

2.1.1. New Zealand context

Sixteen regional or unitary councils are responsible for managing New Zealand’s freshwater resources (surface- and groundwater quality and quantity), through the Resource Management Act (RMA) (New Zealand Government, 1991). National direction is provided via the National Policy Statements for Freshwater Management (NPS-FM, Ministry for the Environment, 2017) and National Environmental Standards, e.g. the proposed National Environmental Standard on Ecological Flows and Water Levels (Ministry for the Environment, 2008) (Figure 2). Councils set objectives for the state of freshwater bodies and set limits on resource use to meet these objectives. In many regions setting objectives and limits is done via collaborative policy processes, where community members are involved and whose mandate ranges from providing policy advice and recommendations through to providing policy direction and settings (Hughey et al., 2017). The contested nature of freshwater management was what led to using collaborative processes for stakeholders to debate and find agreement in determining water limits during policy development rather than through the law courts post-policy notification. Regionally specific limits are implemented in various ways including cap-and-trade programmes, regulation of practices, discharges or water use, requirements for farm plans, and consents/permits for certain types of land use or use intensity.

Figure 2. Simplified schematic of the New Zealand policy and regulatory framework for freshwater management.

Indigenous people perspectives (Castleden et al., 2017; Harmsworth et al., 2016; Te Aho, 2019) are growing in importance where incorporating Mātauranga Māori (Māori knowledge) is required for natural resource management, e.g. Te Tiriti o Waitangi (Treaty of Waitangi) 1840, RMA 1991 (New Zealand Government, 1991) and NPS-FM (Ministry for the Environment, 2017). In Mātauranga Māori natural resources ‘are perceived as ancestors’ (Te Aho, 2019) and inter-generational responsibilities are recognised (Durie, 2014). Treaty of Waitangi settlements have also changed the management of the Whanganui and Waikato Rivers (Te Aho, 2019), reflecting a changing legal and cultural landscape, from a mainly anthropocentric worldview to a more holistic framework that incorporates tikanga (indigenous laws and values), and aims to ‘restore and protect the health and well-being of the natural world for future generations’ (Charpleix, 2018; Te Aho, 2019).

2.1.2. California context

In California, the State Water Resources Control Board (SWRCB, within the California Environmental Protection Agency) is the regulatory agency overseeing the implementation of freshwater-related federal and state regulations, including the 1968 Antidegradation Policy, the 1969 Porter Cologne Water Quality Control Act, and enforcement actions under the 2014 Sustainable Groundwater Management Act (SGMA). A second state agency, the Department of Water Resources (DWR, within the California Natural Resources Agency) administers water resources planning, managing the California aqueduct and reservoirs, and managing SGMA implementation through technical guidance, grant programmes, and regulatory review of local groundwater sustainability plans.

The day-to-day implementation of the 1968 and 1969 water quality regulations is delegated by the SWRCB to nine Regional Water Boards (RWB) (Figure 3). Each RWB developed a basin plan identifying water bodies (surface- and groundwater), beneficial uses of water bodies, and water quality objectives associated with beneficial uses. Basin plans are the basis for designing orders containing ‘waste discharge requirements’ (WDRs or permits) for parties discharging (point and diffuse) into surface- or groundwater. Recent developments in California’s Central Valley include a basin plan amendment to manage salt and nitrate in surface- and groundwater, and agricultural WDRs regulating nitrogen management on farms. One order affects dairies and another non-dairy bovine sources. A third (Irrigated Lands Regulatory Program [ILRP]) pertains to ∼30,000 growers with over 2.5 Mha of other irrigated lands, who are represented through 14 third-party ‘agricultural coalitions’. Coalitions oversee monitoring surface- and groundwater quality and define practices/programmes to protect water quality. Analogous but slightly different programs exist in the other eight regions.

Figure 3. Simplified schematic of the Californian framework for management of freshwater quality, with a focus on the agricultural landscape in the Central Valley Regional Water Board region.

For water supply, surface water use permits are issued by the SWRCB to post-1914 appropriative users³ subject to priority rights among users, instream flow requirements for habitat protection and public trust requirements. SGMA is California’s first step toward statutory control of groundwater extraction (Figure 4). Over 200 local Groundwater Sustainability Agencies (GSAs) manage extraction using various governance formats. They are developing local Groundwater Sustainability Plans (GSPs) on which are based future monitoring and assessment of groundwater resources, public and stakeholder engagement in groundwater management, and future projects/actions to ensure sustainable groundwater management. Sustainability is defined broadly as the absence of undesirable results, including continued water-level decline, storage depletion, land subsidence, water quality degradation, seawater intrusion, and negative impacts to surface-water beneficial uses and users (Harter, 2020). SGMA implementation is a catalyst for integrated surface- and groundwater management, of water supply and quality, and is coordinated with land-use planning agencies. However, overlapping legal and policy requirements are challenging for GSP development due to scientific and legal uncertainties as well as the lack of pre-existing data (Owen et al., 2019).

Figure 4. Simplified schematic of the Californian framework for management of groundwater quantity.

2.2. Data collection and interview process

We undertook semi-structured interviews with people involved in developing freshwater policy in New Zealand (8) and California (8) between September 2018 and September 2019. The interviewees were a mix of regional/state government staff, non-government organisations, industry and researchers (Table 1). Interviewees were identified based on their involvement in freshwater policy processes. They used science in policy formulation, advocated science to influence policy development or oversaw collaborative decision-making processes (and the associated science). Social ethics approval was via the Manaaki Whenua Landcare Research social ethics approval process (number 2021/23NK) which is based on the New Zealand Association of Social Science Research code of ethics.

Table 1. Interview sample across policy contexts.^a

	All	California	New Zealand
Total	16	8	8
By sector
Government	9	2 (0,0,2,0)	7 (0,3,4,0)
Industry	4	3 (2,0,0,1)	1 (0,0,0,1)
Non-government organisations	1	1 (0,0,0,1)	0
Research^b	2	2 (2,0,0,0)	0

^a In parenthesis are the numbers of interviewees who provide science, manage science for decisions, make policy decisions and advocate for science findings.

^b The authors are involved in science used in decision-making processes in New Zealand and California (not included in tally).

The balance of interviewees between the two jurisdictions reflected the different decision-making processes. In New Zealand, representatives from regions managing freshwater using collaborative processes were interviewed. Their roles were to manage science for these processes (3 persons), advocate policy positions (1 person) and/or convert the decisions of the collaborative process into policy (4 persons). The decision-makers were community members and ‘experts’ in aspects of freshwater management, e.g. farmers, health officials, environmental NGOs, Māori, implying different barriers and pinch-points for implementation of science in policy and nuances for some CRELE and ACTA factors. These interviewees had oversight across the decision-making process and the community representatives making decisions. In California, interviewees represented a cross-section of people making policy decisions (2 persons), providing science (4 persons) and/or advocating for a policy position (2 persons) as unlike New Zealand there was not a single process designed to address all freshwater management policy for a waterbody.

Interviewees were identified by the research team with snowball sampling to identify additional participants. Once sampling saturation was reached, we stopped the interview process assuming the interviews are representative despite the relatively small number of interviewees and uneven distribution of interviewees across the different players in the science-policy realm. Auto-ethnography (Adams et al., 2015) was used to gain personal reflections based on several authors’ experiences of providing science for policy development.

Each semi-structured interview included open-ended questions (Appendix) covering introductory questions, advantages of using science to develop policy responses, barriers to the uptake of science to underpin policy responses, science gaps, closing questions on learnings from their policy process and suggestions for improving science uptake.

The initial questions helped interviewees focus on science and a policy context. The question on their definition of science provided context for responses on barriers to science uptake and general interpretation of responses. It ensured interviewees understood science, which for us covered environmental, social, economic and cultural sciences/knowledge. Cultural knowledge is important in New Zealand as Māori are treaty partners for natural resource management and their worldview has prominence in these decisions. Interviewees chose one policy process for the remaining questions. This was so responses had contextual continuity and facilitated recall (interviewees could also broaden their examples). For this assessment, we only used the questions relating to the advantages of using science to develop policy, barriers to science uptake and suggestions for improving science uptake in policy. There was a groundwater focus but responses included freshwater in general, particularly in New Zealand where policy decisions covered both surface- and groundwater. In California, groundwater is the most recent issue and responses mostly related to that. Multiple science outputs (e.g. contracted research reports, citizen monitoring, community experience) and the various types of science for decision-making – biophysical and social sciences, economics and indigenous knowledge – were considered. Interviews were recorded and qualitatively analysed using thematic coding to identify key barriers to using science in policy development and how to improve science’s influence in policy decisions.

The themes for the assessment of the barriers (Table 2) were based on findings from the literature related to the uptake of science for policy (CRELE and ACTA framings) and themes emerging from our interviewees and other scholars. In particular, Spruijt et al. (2014), Fernandez (2016) and French (2018), who note the human dimension of policy decisions. While our theme choices to represent the human dimensions drew from our interviewee responses, we also cross-checked the literature for key insights we may have overlooked. Thus, the pillars and elements to more comprehensively assess the influence of science in policy decisions are derived from these themes (Figure 5).

Figure 5. Pillars and elements that describe the influence of science in policy decisions.

Table 2. Pillars, elements and analysis themes to describe the influence of science in policy.

Pillar	Element/theme	Description
Science	Credibility	Perception that the information is scientifically adequate, and the sources are authoritative and trustworthy
	Legitimate	Information development process is unbiased and respectful of divergent stakeholder beliefs and values
	Relevance and applicability	Relevance (salience) of the information to decision-makers, particularly in terms of geographic and scalar context and timeliness
	Comprehensiveness	Evidence (science) is comprehensive and accounts for a broad range of variables
	Timing	Extent to which science delivery aligns with policy development process
Scientist	Accessibility	Accessibility, readability, and how intelligible the science is to the audience the scientist wants to influence
	Capability of scientist	Ability of the scientist to understand the needs of the decision maker
	Mind-set of scientist	Degree to which a scientist is open-minded and willing to consider other science/solutions as credible
	Scientist personality	Degree to which the scientist’s personality influences decision-makers acceptance and use of science
Decision maker	Capability of decision maker	Decision makers ability to understand and interpret science
	Mind-set of decision maker	Degree to which the decision maker is open-minded and willing to consider science that questions prior beliefs
	Political influence	Extent to which political processes or agendas influence the decision maker and the use of science for policy
Scientist – decision maker relationship	Trust	Trust in scientist
	Financial resources	Whether financial resources hindered the delivery of robust science for policy or scientist involvement in policy process; often influenced by decision makers

Two analytical approaches were then used. First, the per cent of statements across all interviewees (n = 168) referring to each assessment theme. Second, a binary (yes/no) response at the level of each interviewee (n = 16). The binary response shows the agreement between the interviewees on a theme and the per cent of statements shows the emphasis placed on each theme by all interviewees.

3. Results and discussion

3.1. Roles of science in policy

Interviewees discussed the roles of science in freshwater management and its influence on policy development. We assessed responses using a policy cycle framing (Greenhalgh, 2011). The policy cycle was useful for assessing responses, but we recognise it may not fully capture all uses of science. For example, Van Buuren and Edelenbos (2004) noted advocacy purposes as an additional use of science. While this was not noted by our interviewees as a role of science, it was acknowledged around barriers for science influence in policy decisions.

Four of the five roles of science noted by interviewees aligned with Gluckman’s ‘policy cycle’ (2018): problem identification, monitoring, policy planning and development, and policy implementation and evaluation. Differences between science disciplines emerged, e.g. hydrology for the connections between surface- and groundwater, but sociology for understanding and driving the acceptability of surface-water quality. Generally, it was thought there was ‘not enough use of science in policy development’ for freshwater management.

3.1.1. Raising awareness

Awareness raising (first role), quantified issues such as monitoring and quantifying groundwater quality and quantity. Science emphasises the need to take action and understand the context. For example, the Central Valley’s dairy industry undertook their own research on nitrogen leaching from lagoons, corrals and manure disposal areas by monitoring well nitrogen concentrations (Central Valley Dairy Representative Monitoring Program 2019). This stakeholder-led, independently certified science was accepted by academics and politicians and led to industry leaders accepting that groundwater quality issues existed, owning the problem, and taking responsibility to find solutions.

3.1.2. Defining problems

Responses highlighted the power of well-presented science. Data visualisation shows problem extent and helps define problems (second role). Science can set the direction for policy.

Satellite images from NASA and projections on the extent of groundwater overdraft in the Central Valley […] were an eye-opener and game-changer for the development of groundwater management policy.

3.1.3. Conveying knowledge and information

Unsurprisingly, science was central for conveying technical information, data and knowledge (third role). Science helps stakeholders understand complex systems. For groundwater systems, this includes biophysical and geochemical aspects (groundwater flow and quality and interconnections between surface- and groundwater) plus socio-economic aspects. Interviewees noted modelling is essential for problem definition.

3.1.4. Identifying and evaluating options

The fourth role was identifying where science guides monitoring design (i.e. when and where to sample). Science also answers ‘what if’ questions and outlines options. Interviewees from New Zealand and California highlighted that ‘small and agile scientific models [are required] for policy choice’ while also noting models should be ‘complex and robust to predict impacts of policy decisions and to balance issues between different interest groups’. Scenario modelling was seen as a powerful tool to assess options. One example described using both modelling and managed aquifer recharge pilot studies to assess options for freshwater quality improvement. In California, landscape-scale modelling (Soil & Water Assessment Tool; https://swat.tamu.edu/) showed the impact of land use on groundwater quality and helped improve the region’s Management Practices Evaluation Programs (part of ILRP). In New Zealand, a colour scheme to designate zones of nutrient or water allocation was used to identify over-allocated zones (red) and under-allocated zones (green). Planning rules to meet allocation objectives were then developed via a collaborative community process. Both New Zealand and California used scenario modelling to assess climate change effects. Evaluating options is important to balance the values/issues of different interest groups and stakeholders. It was also noted that science supports rather than leads collaborative decision-making processes.

3.1.5. Implementing and evaluating policy

Last, science plays a role in policy implementation and evaluation. One example shared was for monitoring groundwater quality and interpreting complex data to assess whether desired outcomes are being achieved.

3.2. Barriers around the uptake of science for policy decisions

The extent to which policy is informed by science is not always clear. With the growing call by policymakers and stakeholders for evidence-based policy decisions, we explored barriers to science uptake. In terms of the science pillar, our findings had some similarities to CRELE (Cash et al., 2002) and ACTA (Dunn & Laing, 2017) frameworks but they did not fully explain our interview responses (Figures 6 and 7). We found some important nuances. The greater importance of the scientist and decision-maker and their relationship/interaction pillars became apparent.

Figure 6. Percent of statements (n = 168) for each barrier to the influence of science in policy theme.

Figure 7. Number of interviewees (n = 16) who noted each barrier to the influence of science in policy theme.

3.2.1. Ability of CRELE and ACTA frameworks to explain barriers for science uptake in policy

We concur with Dunn and Laing (2017) that CRELE is not robust across all contexts and may focus too heavily on the production of scientific knowledge rather than the effectiveness of science in policy decisions. Credibility in our interviews was important and revolved around trust in the science and the scientist. If decision-makers (often community members) trusted the scientist, they trusted the science. There is closer interaction between scientists and decision-makers in collaborative processes than in more arm’s-length consultative policy processes. This may explain why collaborative processes are driven more by scientist personalities than scientific rigour, which decision-makers cannot necessarily assess.

There is a danger that decision-makers, especially in collaborative processes, are influenced by questionable scientific credibility as ‘people are captivated by bright shiny things. While other scientists may be able to see through “flashiness” the community [i.e. decision-makers] can’t’. California had a nuance where ‘agencies tend to trust research corroborated by universities’, but also noted ‘university folks don’t tend to resonate with practical ideas’ and ‘if you bring something innovative [as a science provider] to a policymaker that is beyond their realm of knowledge then they may ask “why should I believe you?”’ This may change as community-based coalitions are formed in California to manage groundwater extraction and contamination.

Intellectual alienation was common in both countries. This diminishes trust in the scientist and science and the stakeholders think ‘this guy [scientist] doesn’t know what I am doing’. In these instances, the scientist and science have lost credibility.

Relevance (CRELE; Cash et al., 2002) or applicability (ACTA; Dunn & Laing, 2017) were noted mostly by our California interviewees but were not major themes. In New Zealand, regional councils commissioned research for their policy processes, meaning the science was relevant and applicable to the respective freshwater issues. Legitimacy, while noted, was again not a central theme that arose in our study and was noted more by US interviewees.

Comprehensiveness (ACTA) was rare in our interviews and where it did arise it was different from Dunn and Laing (2017) where holism and economic impact were key. New Zealand legislation requires consideration of environmental, economic, social and cultural aspects, meaning most policy processes use science from a range of disciplines and perspectives. Comprehensiveness was noted by half the New Zealand interviewees in relation to the resources needed.

New Zealand challenges related more to the capability of the decision-maker and differences between the sciences, e.g. ‘the qualitative sciences are harder to argue objectively’, ‘[some people] don’t understand the technical results and therefore were not interested in the modelling’, and ‘some [people] struggled with economic and cultural information’. Constraints around science that integrated multiple components were also noted: ‘a big complex model is only as good as its weakest link’, and ‘different models give different results and this confuses people’, again emphasising the capability of the decision-maker.

The policy processes we assessed struggled to address the traditional lag that puts environmental issues behind socio-economic considerations (Juntti et al., 2009), but we found this dynamic was changing whereby environmental issues were increasingly driving policy change.

Biophysical scientists were able to say they were talking about real things and they [scientist] were the experts. The [community decision-makers] found it easier to question the economics as ‘everyone is an economist’. Some of the economic questions were really [about] biophysical modelling rather than [about] the economic models as the models were coupled and economists couldn’t answer the questions.

This again highlighted challenges around the capability of decision-makers.

Uncertainty increases with complexity and more comprehensive science (Jørgensen, 1990), and drew several reactions from our interviewees, often reflecting the political influence in decisions and credibility of the science. Decision-makers can ‘use science uncertainty as a good excuse to do nothing where [decision-makers] keep asking for more science to get more certainty’. In other words, uncertainty can be an excuse for regulatory inaction (Oreskes & Conway, 2010). Uncertainty also impinges on the validity of science where ‘policy folks always want certainty and when you [scientist] can’t provide certainty then the science comes across badly and not believable’. Decision-makers expect straightforward information so their decisions can provide certainty. However, most policy issues have uncertainty and inevitable social and ethical concerns (Funtowicz & Ravetz, 1990). While comprehensive science is necessary, the challenges around uncertainty can affect how it is used. Again, our responses show it is the science users rather than the science itself that influences the use of science in policy.

Timeframes and timeliness (ACTA), especially the misalignment between research and policy timelines were noted. With collaborative policy timeframes, ‘it was hard to get science ready for the next meeting’ and ‘tight timelines are really hard for scientists, policy folks and [collaborative process members] to meet but without them the process goes on forever’. A tension, not identified by Dunn and Laing (2017), was noted between timing and comprehensiveness where ‘you move to complex [integrated] models that take a long time [to develop] and most of the policy process is done before the model is giving results’. This will become an increasing issue if multi-, inter- and/or trans-disciplinary science is required to inform highly complex contested natural resource policy problems as the lead time for robust science will increase.

Accessibility (ACTA) was not raised in the same manner as Dunn and Laing (2017) where the theoretical nature of journal articles, difficulty in finding relevant research evidence and use of jargon were key issues (also noted by Cvitanovic et al., 2016). Instead, the ability of scientists to communicate science to decision-makers was the issue, particularly in collaborative processes, e.g. ‘ability of scientists to communicate results to different non-science audiences’, ‘talking to a scientist is completely unintelligible to the rest of us [policy decision-makers]’, and ‘modellers tended to go into a lot of detail and people [community decision-makers] tended to shutdown’.

3.2.2. Human dimension as a barrier for science uptake in policy decisions

The CRELE or ACTA frameworks did not capture all the factors we identified as influencing policy (Figure 6). They did not adequately reflect the human dimensions (scientist and decision-maker and their relationship/interaction pillars) around who is delivering/receiving science, and what and how science is used. It was these human aspects that were often a barrier to science’s effective influence on policy. One common human factor, noted earlier, was the high variability in competence of decision-makers to use science effectively, e.g. ‘level of scientific literacy in US society is low’, ‘people don’t understand the limits of science’. Gluckman (2013) noted that misinterpretation and bias of science findings more likely arises where decision-makers are not formally trained in research methodologies. Trust in the scientist and scientist’s personality were two other factors noted, as highlighted in the credibility discussion above.

The mind-set of decision-makers was a challenge raised in both jurisdictions. ‘Stakeholders [within collaborative processes] need an open mind but many came in with a set of views and it was hard to change their minds [regardless of the science]’. ‘There is a disdain for knowledge and the rigour of science by people, for example, ‘[who implicitly state] don’t bother me with the details', 'the decisions get made with the gut’, and ‘when [decision-makers] say they want science to inform their work they only want one idea, they don’t want the range of possibilities or anything different to what they thought they wanted’.

Political influences were also noted. Vested interests and the mind-set of decision-makers influence the type of science, e.g. ‘paid science’, ‘cherry-picking of the science’, ‘selective use of science’ and ‘easy solutions’. Cynically, science is used to strategically or symbolically legitimise policy solutions shaped by political processes (Juntti et al., 2009; Owens, 2005) ‘Science can get in the way of political agendas. [Decision-makers] don’t want to acknowledge the science because it doesn’t fit their agenda’. The negative aspects of political influence appeared more apparent in California. In New Zealand, it was noted that ‘different groups wanted the problem to be caused by different things’. In this instance, ‘more science was useful as it helped move the conversation from real cause to the fair thing to do’.

Despite vested interests sometimes influencing decision-making, it did not appear to be the ‘norm’ for our policy contexts. This is perhaps due to greater stakeholder scrutiny with more collaborative policy processes in New Zealand, and in California the starkly opposing views on water management intensifying with greater resource scarcity. Top-down policy making and resource abundance tend to maintain the status quo. However, there is greater transparency and less ability to manipulate science to support a particular stance when collaborative decision-making is used where resource scarcity is increasing. In some contexts, science and decision-making are always subject to political influence and pressure (Juntti et al., 2009; Radaelli, 1995).

The devolution of decisions also influenced how science was used. In New Zealand it was noted ‘any decision still needs to go through a political process and that the final decision may be made by people who had not seen all the information’. Both the recipient of the science and the political process are key considerations for how science influences policy decisions. Mind-set and capability of scientists also affect science influence ‘[scientists have a] lack of understanding of the roles of different stakeholders’ and ‘[scientists] have to [be] … not afraid of caveats or articulating where the uncertainty is but being able to give a range of what may be more likely or not’. Last but not least, lack of resources for applied science was noted as a major barrier.

Our findings highlight that it is not just science itself that influences the voice of science but also the human dimension; the audience receiving the science and who delivered the science were vital. This is not to say CRELE or ACTA should be dismissed; rather these alone are insufficient and arguably are less important than the human dimensions of policy development. With increased public, industry, and NGO oversight of policy debates in highly contested spaces such as freshwater, the volume, nature, and quality of relevant science have greatly improved. As a result, the science deficiencies relating to CRELE and ACTA factors appear less prevalent.

We acknowledge there are differences in the purpose of ‘science’ in previous studies which looked broadly at the body of literature/science being produced whereas our science was most often formulated/commissioned to answer specific questions for a freshwater issue. Therefore, many CRELE or ACTA factors were redundant or less important. Similarly, the empirical study by Cvitanovic et al. (2016) used a broader definition of science, making some of their barriers less relevant for us. These frameworks are just the starting point for what is important in improving the influence of science in policy development.

3.3. Improving the use of science to influence and inform policy

Our interviewees provided several practical insights and these resonated with the authors’ own experiences and observations from numerous natural resource decision-making processes they have been involved in.

Improved science communication is a key aspect that can increase science uptake for policy (Spruijt et al., 2014). One interviewee observed

that the attributes of a good [science] communicator were they are passionate, stay within the boundaries of the science, great translators, and use multiple modes of science delivery. A good presentation tells what the conclusions were, tells what was found and then summarises again. Saying the same thing several times and in different ways is important for complex information.

While traditionally science is delivered to decision-makers through technical reports and journal articles, this is changing. Collaborative policy processes, useful for complex and contested spaces (Gregory et al., 2012), involve greater interaction between decision-makers and scientists, increasing the importance of presentation-style delivery.

Science to support implementation is often encapsulated in ‘tools’ which should be user-friendly and easy to understand. This presents challenges as scientists need to translate often complex concepts into simple messages for target audience(s). Communication deficiencies are not new (e.g. Owens, 2005), but they remain unresolved challenges for scientists. While using knowledge-brokers to help science communication is one solution (Cvitanovic et al., 2016; Dunn & Laing, 2017; Fischer, 2002), some interviewees noted scientists are best placed to communicate their findings. Perhaps a different role for knowledge-brokers is to translate lay knowledge for incorporation into traditional scientific analysis (e.g. modelling frameworks, survey instruments or statistical functional forms) and complex (and potentially uncertain) scientific results for less technical audiences. Thus, communication is likely an inevitable personal journey for every scientist as they engage with different decision-makers and as their science context changes (environmental, political, economic and social) and as new knowledge emerges.

Several useful insights from this study may improve scientist-decision maker interactions. Conceptual models of the science process and how science links together may enable buy-in from all participants on the science process and the linkages between different sciences being produced. These models are useful to remind people how the science fits together and ensure that knowledge/discussions have considered all components of the conceptual model (e.g. Greater Wellington Regional Council, 2016). Any knowledge/science generated for issues could include quantitative (including monetised information) and/or qualitative information/knowledge.

Conceptual models are useful for communication and also clarify common misunderstandings of policy processes (Fernandez, 2016; French, 2018; Juntti et al., 2009; Owens, 2005). A conceptual model can show the complexity and non-linearity of issues, demonstrate why and how multiple types of knowledge are needed and used, and facilitate decision-makers’ use of multiple perspectives. They can also highlight the value of different types of knowledge and ensure they are included in decision-making. This helps single-issue stakeholders broaden their outlooks and collaborate, particularly where they are making decisions with minimal policy-making training.

Models are frequently used to understand the implications of future scenarios, including alternative policy options. However, natural resource management (e.g. freshwater management) often requires complex models and/or uses several models to assess scenarios. These modelling exercises take time, leading to timing issues (noted as a barrier earlier) and difficulty with communicating complex (and often uncertain) modelling results. One potential solution is to complement complex models with smaller agile models. More agile models can identify potential scenarios for further exploration, giving time to develop the complex models and verify, confirm, or dispute the results of the agile models. One interviewee noted: ‘if the different [science] aspects are done separately and well then a good policy person/process can weigh up the bits better than having a big model with lots of uncertainty’.

Tied to the timing of science delivery for policy processes are the type(s) of research undertaken. More emphasis is often put on biophysical science early in policy processes to characterise issues, understand issue extent/severity and demonstrate to the public, stakeholders and decision-makers that an issue exists. Unfortunately, this is frequently at the expense of other science, e.g. characterisation of the community is frequently done at later stages but could provide useful insights early in policy processes. It is common for economic or social impacts or alternative scenarios to be delivered late in policy processes and often in condensed timeframes to meet deadlines. All science/knowledge components, ideally, would be available to decision-makers at a similar time.

The policy process influences how science informs decisions. Early engagement of scientists with the community, stakeholders and decision-makers improves the use of science in decision-making. Naturally, this brings its own challenges – scientist discomfort engaging outside the science community, resources needed early in processes before sufficient budget is available, and the full extent of issues may not be understood, meaning not all relevant science/knowledge needs can be quantified. There are benefits, however. Scientists better understand the community, stakeholders and decision-makers, and therefore, are better able to communicate science findings and uncertainties. It is easier to co-develop the science knowledge-base and test any results with the community (Fischer, 2002), stakeholders and decision-makers, and provide timely science/knowledge for different policy phases.

4. Conclusions

there is no recipe for an easy role of scientists in public contexts. (Fernandez, 2016, p. 172)

Our research helps to empirically verify the applicability of the CRELE and ACTA frameworks for explaining the roles of science and scientists in policy making for complex issues and, importantly, systematically highlights the importance of the human dimension to what and how science is used in policy decisions. Freshwater management is a contested and complex space in New Zealand and California due to outward circumstances (e.g. climate change, water scarcity, eutrophication and contamination of freshwater bodies), which have increased general awareness that fresh water is a limited, vulnerable and highly pressured resource. Policy debates are subjected to close scrutiny by the public, industry and NGOs, which has improved science quality and volume. Our results showed that in this particular context factors of the CRELE and ACTA frameworks are less prevalent than the interactions and relationships between scientists and decision-makers and the characteristics of individual scientists and decision-makers. These human factors need more consideration to enhance the voice of science in policy.

Decision-makers clearly believed science was important to underpin policy making, so the question is how to improve science’s influence on these decisions, especially where stakeholders want more evidence-based policy.

We found four pillars to consider for improving the influence of science in noisy policy arenas – the science, the scientist, the decision maker and the decision-maker’s relationship/interactions with the scientist. The latter three ‘human’ pillars are likely even more important for complex contested issues like freshwater management.

The common barriers shared by New Zealand and California in freshwater management suggest that our findings are likely to have universal relevance. However, barriers related to negative political influence were one aspect that appeared more prevalent in the Californian context. While it is not unrealistic to expect political influences in policy development and implementation, it is reasonable to expect decision-makers to be transparent when facing the complexity of interacting biophysical, economic, social and cultural spheres that influence freshwater resource management. The role of science is to provide the evidence on all these spheres and give a ‘voice’ to the science that underpins these spheres to enable decision-makers to engage with the science and consider all aspects of an issue.

With greater community involvement in freshwater management decisions, e.g. collaborative processes in New Zealand and GSAs in California, transitioning to participatory research approaches could increase science uptake for policy making. Participatory research approaches can increase the extent to which decision-makers perceive science to be salient, credible and legitimate (Cvitanovic et al., 2016), increase emphasis on the stakeholder perspective ranging from problem formulation to validation of research results (Juntti et al., 2009) and build social capital and trust with stakeholders, allowing different types of knowledge to emerge (Fernandez, 2016).

One area that arose in New Zealand was the role of indigenous knowledge. Globally, we see increasing recognition and integration of indigenous input into natural resource management science and policy (Berkes, 2018; Raymond et al., 2010; United Nations General Assembly (UNGA), 2007) adding another important dimension to already often complex issues. Different countries have different motivations for such knowledge integration, including the need to develop resilient social-ecological systems, maintain biodiversity, and fill gaps in understanding which science failed to fill (Bohensky & Maru, 2011). The latter is the key driver for New Zealand’s inclusion of Mātauranga Māori into policy decisions, and its inclusion in policy processes is being legislated. With this recognition, engagement with indigenous communities in resource management decisions now influences policy making (Diver, 2017).

While policymakers often noted that scientists did not understand the policy world, this applies in reverse too. Policymakers may not appreciate the requirements of scientists to produce robust science. In some instances, the delivery and timeliness of applied science for policy may not align. Communication was noted as a key barrier in our study. There are many approaches to improve communication including greater empathy on the part of the policymakers and the scientists.

We, as scientists, must acknowledge and embrace multiple types of knowledge and enhance our relationship/interactions with policy stakeholders. This does not mean that all our science needs to be multi-, inter- or trans-disciplinary, but we must understand and appreciate where science fits within broader community and public-policy contexts, and how to give our science a ‘voice’ in noisy policy arenas.

Acknowledgements

We are grateful to the participants in our interviews. We would also like to thank Tony Corbett and Cissy Pan for his graphics support.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the New Zealand Ministry of Business, Innovation and Employment's Catalyst Fund Seeding Grant [grant number 34178] and The New Zealand Institute for Plant and Food Research’s Sustainable Agro-ecosystems research programme funded by the New Zealand Ministry of Business, Innovation and Employment Strategic Science Investment Fund.

Notes on contributors

Suzie Greenhalgh

Suzie Greenhalgh is a resource economist and Portfolio Leader for Society, Culture and Policy at Manaaki Whenua Landcare Research NZ. She holds a PhD in resource economics and Masters degrees in Economics and Soil Science. Much of her research has focused on environment and agricultural policy, economic instruments, and participatory/collaborative approaches to address environmental issues such as climate change, freshwater degradation, and loss of biodiversity.

Karin Müller

Karin Müller is a Senior Environmental Scientist in the Cropping Systems & Environment Science Group at The New Zealand Institute for Plant and Food Research in Hamilton, New Zealand. Her research focuses on the effect of land use and management practices on the environment. In particular, she is interested in studying processes at the soil-water interface and the health of agro-ecosystems in general.

Steve Thomas

Steve Thomas is a senior researcher at the New Zealand Institute for Plant and Food Research. He holds a PhD in Soil Science and a Masters in Land and Water Management. His research has primarily focussed on mitigating environmental losses to water and air from agricultural systems. Currently he is working in collaborative research programmes to understand the societal, cultural, economic, and environmental drivers and barriers for transitioning to agricultural land uses that have low environmental impact, with a particular interest in approaches co-developed with stakeholders.

Marsha L. Campbell

Marsha Campbell is a Farm Advisor (emeritus) at the University of California Cooperative Extension. She holds a Masters degree in Agronomy and Soils. Her areas of expertise include forage crops, plant management systems, soil, plant, water, nutrient relationships, and natural resource management. She has worked extensively with farmers in California’s San Joaquin Valley.

Thomas Harter

Thomas Harter is the Robert M. Hagan Endowed Chair for Water Resources Management and Policy at the University of California, Davis. He holds a joint appointment as Professor and Cooperative Extension Specialist in the Department of Land, Air, and Water Resources, is currently chair of the Hydrologic Sciences Graduate Group, and, as Associate Director of the Center for Watershed Sciences, is a team partner for the World Water Center. Dr. Harter holds PhD in Hydrology. His research and extension emphasize the nexus between groundwater and agriculture with a focus on nonpoint-source pollution of groundwater, sustainable groundwater management, groundwater and vadose zone modelling, groundwater resources evaluation under uncertainty, groundwater-surface water interaction, and on contaminant transport.

Notes

1 Scientist is used throughout as a general term for researchers from any discipline that provide evidence or knowledge on an issue.

2 At the time of our assessment collaborative policy processes were being undertaken to develop freshwater policy. More regulatory top-down approaches have since emerged for managing freshwater in New Zealand. The 2014 and 2017 NPS-FM are the relevant versions of the NPS-FM for our analyis.

3 Pre-1914 appropriative users and riparian users are subject to the same public trust and waste and unreasonable use requirements but are generally more senior and do not have to have a permit, although they do have to report their yearly use.

Appendix. Semi-structured interview questions

Introductory questions

• What do you consider science around groundwater to cover?

• Can you outline where science has been used to support policy decisions for groundwater?

• Can you choose a specific example of a groundwater policy issue? [the remaining questions were answered in relation to this example]

Advantages of using science to develop policy responses to groundwater problems

• What advantages have you seen for using science (biophysical, economic, social and cultural) to develop polices to address groundwater issues?

• If policy-makers were to rank order the importance of biophysical, economic, social and cultural science in informing the policy decision, what would that order be?

• If stakeholders were to rank order the importance of biophysical, economic, social and cultural science in informing the policy decision, what would that order be?

Barriers encountered for the uptake of science to underpin policy responses

• What barriers have you encountered in the use and uptake of science for informing policy responses for managing groundwater?

Knowledge/science gaps

• Can you provide some examples of science gaps you have noted while addressing the groundwater issue you chose?

• Are some of these gaps more urgent than others?

Closing questions

• What have you learnt from using science to underpin groundwater decisions?

• What would you do differently if you had your time again with this issue?

• Do you have any suggestions for improving the role of science in policy development?

• Is there anything else you would like to add around the use of science to develop policy responses to groundwater issues?