How can management of uncertainty in sustainable diversion limits be advanced in the review of the Murray-Darling Basin Plan?

ABSTRACT

This paper explores how scientific uncertainty was addressed in the Australian Government’s decision-making process leading to the establishment of the ‘sustainable diversion limit’ (SDL) for the Murray-Darling Basin Plan in 2012. Our research draws on a systematic document analysis to generate an argument map representing the reasoning presented in government reports and legislation, as well as interviews with current and former public servants and researchers who have knowledge of the decision-making process. Looking towards the review of the Basin Plan, this analysis provides additional evidence for a number of areas to be discussed to advance the management of uncertainty in the associated decision-making. First, there is a need to address the challenges to adaptive management, as adaptive management delegates much of the uncertainty to future changes in policy as improved knowledge is gained. Second, there is a need to generate and use new knowledge for the review since the legislative mandate to use the ‘best available science’ enabled the use of pre-existing models and historical climate data. Third, there is a need for dialogue on handling of trade-offs between objectives given that uncertainty creates space for political pressures from interest groups. Fourth, there should be wider transparency about uncertainty in decision-making processes.

KEYWORDS:

• Murray-Darling Basin
• uncertainty
• sustainable diversion limit

1. Introduction

The setting of water policy, as occurs with Sustainable Diversion Limits in the Murray-Darling Basin, must consider the hydrosocial system in which such decisions are embedded. However, the governance and management of water in hydrosocial systems unavoidably involve decision making in a context of ubiquitous uncertainty given the inherent complexity of water systems, deficiencies in the availability and quality of informative data, and the fundamental uncertainty about how these systems might change into the future. Although some uncertainties can be reduced through further research or informed data collection, many are irreducible, requiring managers to grapple with what is known and unknown while still making decisions that can be scientifically defensible and socially robust. This situation is further complicated by general recognition that conditions of uncertainty can exacerbate conflicts over the appropriate responses or management strategies, and indeed uncertainty itself can be politicised in ways that may be used to justify stalling or delaying action. Consequently, understanding how organisations make decisions under conditions of uncertainty is vital to enable effective water governance, and to develop and improve strategies for how uncertainty is managed and communicated in these contexts.

Water governance, meaning the ‘political, social, economic and administrative systems that are in place to develop and manage water resources, and the delivery of water services, at different levels of society’ (Global Water Partnership 2002), often relies heavily on models that represent aspects of the hydrological, climatological, and increasingly the social system underlying support for decision making. For the reasons given above, these models, no matter how detailed, cannot fully represent a system or predict the future, and thus are always uncertain.

Uncertainty can be defined as a lack of complete information on a subject (Walker et al. 2003). It is not limited to statistical uncertainty, which is quantifiable and is often the focus of uncertainty analysis (Bark et al. 2013; Walker et al. 2003). Some uncertainties (epistemic uncertainty) can be reduced, for example through data collection to improve knowledge of the system, whereas some uncertainty (known as stochastic uncertainty) cannot be reduced because it relates to inherent variability (Ascough et al. 2008; Walker et al. 2003). Dewulf and Briesbroek (2018) describe different considerations for each category. There are also normative uncertainties relating to values in the decision making process that warrant being taken into account (Bark et al. 2013), and ambiguity wherein actors have different frames of reference (Dewulf and Biesbroek 2018). Another perspective here is that decision makers are increasingly faced with what is often called ‘deep uncertainty’, meaning that they cannot agree on the conceptual models, probability distributions, or which outcomes are wanted (Lempert, Popper, and Bankes 2003).

Growing recognition of the challenges uncertainty poses to decision making has inspired interdisciplinary scholarship that considers these challenges from a number of angles. Post-normal science has long sought to engage with policy issues in the context of high uncertainty by recognising and being transparent about uncertainty, rather than attempting to banish it. It is also explicit about the normative dimensions that shape policy decisions, and focuses on participation and dialogue that engages with uncertainty and values rather than defer to science to justify decisions (Funtowicz and Ravetz 1993). In the policy sciences, Rittel and Webber (1973) challenged the idea that science can be used for complex, contested problems, known as ‘wicked problems’ while more recent work indicates that these problems can be addressed through innovations in mainstream policy making such as collaborative governance, acknowledging the limitations of knowledge and by facilitating dialogue (Head 2019). Scholarship from science and technology studies has emphasised the ways in which uncertainty can be politicised, and the subjective nature of how uncertainties are understood and quantified (Miller 2008). Uncertainty does not exist independently of a decision but instead can influence how decisions are justified and what decisions are made (Stirling 2010). Although necessary (Lewandowsky, Ballard, and Pancost 2015; Van Asselt and Rotmans 2002), the communication of uncertainty to all stakeholders is not straightforward. A perceived need for scientific certainty to underpin decision making can drive experts to present greater certainty, or to present uncertainties as measurable risks (Stirling 2010). In environmental contexts, scientific knowledge is often politicised as multiple disciplinary approaches and bodies of evidence can be interpreted to support specific viewpoints (Sarewitz 2004).

Across this scholarship, it is evident that in decision making under conditions of uncertainty scientific knowledge is not the only factor that shapes decision-making outcomes. Uncertainty can create a space where social and political factors drive heightened contestation over appropriate courses of action. Yet in such contexts, uncertainty is often used as the rouse to justify action (or lack thereof).

This paper considers the highly controversial establishment of a ‘sustainable diversion limit’ (SDL) in Australia’s Murray-Darling Basin, a historically overallocated system, where there is sustained conflict over the governance of the Basin’s water resources. The Water Act (Commonwealth of Australia, 2007) required the establishment of ‘maximum long‑term annual average quantities of water that can be taken, on a sustainable basis, from’ the Basin water resources as a whole and the water resources of each water resource plan area. These averages are the ‘long‑term average sustainable diversion limit’ (item 6 of the table in subsection 22(1)), but the Water Act did not specify how the SDL would be established (Section 23), providing flexibility for the Murray-Darling Basin Authority (MDBA) – the organisation charged with identifying the SDL – to develop a methodology and determine what form the SDL would take.

The decision-making process leading to the SDL involved numerous sources of uncertainty including with respect to climate change, natural climate variability and hydrological data. There have been several analyses of uncertainty in relation to the SDL, including identifying and quantifying uncertainties in the SDL decision (MDBA 2011b) and in earlier modelling that was later updated and used in the decision (Van Dijk et al. 2008), as well as assessing risks to specific outcomes (Bark et al. 2013).

This paper explores approaches to uncertainty used by the Australian Government and their communication of uncertainty in the process of establishing the SDL in 2012. We define ‘approaches to uncertainty’ as actions taken or planned in response to uncertainty in the decision-making process. The approaches to uncertainty within the SDL decision-making process can give insights into how uncertainty was dealt with in a complex decision where there were competing water user perspectives, numerous sources of uncertainty across a large and variable Basin, and multiple government and private actors involved. A better understanding of what occurred then could contribute to improving future decision making for the SDL and other complex water governance decisions while managing and embracing uncertainty.

2. Background

The Murray-Darling Basin covers over one million square kilometres in south-eastern Australia, extending over four states and one territory (hereafter the Basin states) that have jurisdiction over water resources under the Australian constitution (Ross and Connell 2016). Its water resources have great economic, cultural and environmental value, but decades of overextraction of water and regulation of river flows have led to poor ecosystem health and the loss of biodiversity (Davies et al. 2010; Hart 2016). Worsening conditions in the Millennium Drought from 1997 to 2010 catalysed a series of reforms (Ross and Connell 2016) culminating in the Water Act (Commonwealth of Australia, 2007), which established the MDBA and made it responsible for creating a Basin Plan with sustainable diversion limits as defined above. The difference between the existing level of extraction and the SDL would be allocated to environmental flows. This paper focuses on the decision-making process for the total surface water sustainable diversion limit and does not discuss the separate process through which groundwater SDLs were determined.

The Water Act 2007 s23 (Commonwealth of Australia, 2007) required that the SDL reflect an ‘ecologically sustainable level of take’. The Act also refers to ‘the use and management of the Basin water resources in a way that optimises economic, social and environmental outcomes’ (s20) and requires the Basin Plan (and therefore the SDL) to be determined on the basis of the ‘best available scientific knowledge and socioeconomic analysis’ (s21), a phrase that Lindsay (2020) attempted to interpret in the context of the Act but argued needs transparency in the tension between ‘best’ and ‘available’.

In the period between the Water Act 2007 and the Basin Plan 2012, the MDBA determined a volume of water for the SDL (MDBA 2011b), hereafter the SDL decision-making process. In 2008, the Commonwealth Scientific and Industrial Research Organisation (CSIRO) completed the Sustainable Yields Project, which estimated water availability in the Basin, drawing on decades of hydrological modelling of the Basin (CSIRO 2008). Subsequently, the MDBA (2010) released the ‘Guide to the Proposed Basin Plan’ with a range of possible SDL volumes for public consultation. After a strong backlash to the volume of water proposed to be ‘recovered’ for the environment from communities in the Basin, the government adopted a ‘triple bottom line’ interpretation of the Water Act which gave equal consideration to economic, social and environmental outcomes (Walker 2019). In 2011, the MDBA completed modelling and analysis that supported a higher SDL (less water recovery) than the range proposed in the Guide (MDBA 2011b). The 2012 Basin Plan established this volume as the SDL and required Basin state governments to develop Water Resource Plans (WRPs) to ensure the SDL would be complied with from 2019 (Commonwealth of Australia, 2012). As the SDL is a long-term average, the volume of water allocated for consumptive use, and therefore the volume of environmental flows, is adapted each year in response to stored water levels and rainfall, and environmental flows are managed adaptively to help achieve environmental outcomes (MDBA 2019).

The SDL has been criticised both for not allocating sufficient water to the environment and for allocating too much water to the environment, and potentially causing economic impacts on communities in the Basin (Commonwealth of Australia 2011; Pittock, Grafton, and Williams 2015; Walker 2019). In a South Australian Royal Commission, Walker (2019) contends that the legislation should be interpreted to require that the SDL not compromise environmental objectives and the SDL volume decision went against this assertion. The Royal Commission also critiqued the scientific underpinning of the process to determine the SDL, concluding that the MDBA’s decision to use the historical climate record rather than future projections was ‘unlawful’ (2019, 54).

Since 2012, the SDL was increased for catchments in the northern part of the Basin in response to the Northern Basin Review, which involved further research and community consultation (MDBA 2016). There has also been an SDL Adjustment Mechanism which allows the SDL to be increased or decreased by up to 5% if supply and efficiency measures ensure the same outcomes (MDBA 2015). The completion of some WRPs has been delayed (MDBA 2021) and the Wentworth Group of Concerned Scientists (2020) has presented evidence that less environmental water is being delivered than expected under the Basin Plan. Given these developments, the SDL could be reconsidered in the Review of the Basin Plan which is planned for 2026. As a fundamental concern, this paper explores and reflects on how uncertainty was approached when the SDL was established in 2012, and how that approach could be improved in the review of the Plan.

3. Methods

3.1. Overview

We used a mixed-methods approach that involved document analysis to develop an argument map, along with semi-structured interviews. The document analysis reconstructed the Australian government’s approach to uncertainty in the SDL decision-making process and represented it as an argument map. Interviews of key informants with knowledge of the SDL decision-making process complemented the argument mapping by providing further information and perspectives that contextualised the document analysis. Through analysis of the argument map, we derived a set of approaches to uncertainty that decision makers used for the SDL decision, while thematic analysis of the interview data provided insights on how and why those approaches were used.

3.2. Argument mapping

We created an argument map based on analysis of several key documents to reconstruct the implicit argument for the SDL volume decision in 2012 and therefore to examine the role of uncertainty in the decision-making process. An argument is a set of reasons, or premises, supporting a conclusion (Macagno and Walton 2015). An argument map visually represents all parts of an argument including reasons, conclusions, and relationships between them such as whether supporting or countering (Facione and Gittens 2015). For example, Knüsel et al. (2020) assess model uncertainty by creating a comprehensive argument map of the assumptions and justifications for a model and assessing whether the premises are true. Renton and Macintosh (2007) explore the potential of argument maps to engage stakeholders in policy debates by representing information from documents and enabling stakeholders to understand the reasons presented in the debate.

An argument map can reveal approaches to uncertainty that are not necessarily clear when analysing documents section-by-section to identify explicit communication of approaches to uncertainty, which may not be comprehensive. It facilitates an exploration of how uncertainties were or were not addressed over time.

3.3. Document selection

4.

4.1.

The selection criteria for the documents were as follows. First, given the aim to understand the reasoning of government decision-makers, documents were confined to Australian Government legislation and reports published by Australian Government agencies. Australian Government publications contain an implicit argument for why the final SDL volume was chosen through justifications of decisions and discussions of uncertainties and alternative options. Second, the timeframe was limited to the decision-making process described above, which began with the Water Act 2007 and concluded with the Basin Plan 2012. Third, the documents had to provide a substantial insight into the decision-making process.

We found the documents by reviewing peer reviewed literature and reports and investigations on the SDL decision to identify documents that received attention and in particular those that were regularly cited in the MDBA’s descriptions of the decision-making process. In cases of multiple reports or volumes published at the same time, we selected a summary report or one focused on uncertainty. From this, we selected six documents to analyse: the Water Act 2007, the Basin Plan, and four reports, as listed in Table 1.

Table 1. Documents analysed to develop the argument map.

Document	Relevance to research aims
Water Act 2007 (Commonwealth of Australia, 2012)	Defined the SDL and established requirements for it
Uncertainty in river modelling across the Murray-Darling Basin (Van Dijk et al. 2008)	Described uncertainty in the hydrological modelling from the Sustainable Yields Project, which the MDBA later adapted for use in the SDL determination
The Guide to the Proposed Basin Plan: Overview (MDBA 2010)	Described the first approach to determining possible SDL volumes
The proposed ‘environmentally sustainable level of take’ for surface water of the Murray–Darling Basin: Method and outcomes (MDBA 2011b)	Described the methods and reasoning in support of the volume that became the 2012 SDL
Hydrologic modelling to inform the proposed Basin Plan: Method and results (MDBA 2012)	Provided additional information on the methods for determining the SDL volume
Basin Plan 2012 (Commonwealth of Australia, 2012)	Established the SDL and set requirements for its implementation

3.4. Document analysis

In each document, we identified sections of text, between one sentence and a few paragraphs in length, that referenced uncertainty or similar concepts (such as risk and knowledge gaps) or described aspects of the decision-making process that were necessary to understand how uncertainty interacted with other reasoning. Each of these sections of text represented an assertion in the implicit argument for the SDL decision. We assigned each assertion a code and a one sentence summary. We assigned some codes multiple times across multiple documents where similar points were made. We identified 122 assertions (provided in supplementary material) through an iterative process of reviewing the documents multiple times. Interviews (see below) informed later iterations of the document analysis by drawing attention to topics that received comments from the interview participants.

In each document, we identified sections of text, between one sentence and a few paragraphs in length, that referenced uncertainty or similar concepts (such as risk and knowledge gaps) or described aspects of the decision-making process that were necessary to understand how uncertainty interacted with other reasoning. Each of these sections of text represented an assertion in the implicit argument for the SDL decision. We assigned each assertion a code and a one sentence summary. We assigned some codes multiple times across multiple documents where similar points were made. We identified 122 assertions (provided in supplementary material) through an iterative process of reviewing the documents multiple times. Interviews (see below) informed later iterations of the document analysis by drawing attention to topics that received comments from the interview participants.

3.5. Constructing the argument map

Constructing the argument map began with selecting an end point. Given that environmental outcomes were a significant focus of the six documents, the end point is the assertion that the environmental objectives of the Basin Plan would be achieved. The end point was not intended to directly reflect the priorities of the decision makers, noting that the objectives are contested (Walker 2019), and it does not change the scope of statements in the argument map, only whether they are arguments for or against.

We placed assertions, as identified and summarised in the document analysis process (see above), into the argument map. Each assertion is a reason for or against the subsequent assertion to which it is connected in the argument map. The placement of the assertions within the argument map depended on evidence from the documents, which included clear wording, such as the word ‘therefore’, the structure of a section in the document, and, where necessary, general interpretation of the meaning. We created the argument map using Freemind software (Foltinm et al. 2020) to visually display it in a flow chart style.

3.6. Interviews

3.6.1. Purpose

We conducted interviews because they could provide contextual information and perspectives that provided greater detail than in government documents and legislation, including insights into the discussions that took place while those decisions were being made and the broader context. They were intended to provide rich insights that would complement the understanding gained through the argument map. The interviews were conducted in accordance with ANU Human Research Ethics (Protocol 2020/135).

3.6.2. Participant selection

A key informant approach was adopted because participants could provide in-depth discussions about the topic from their experience and specialist knowledge. The interviews were semi-structured to allow participants to explore a broad range of relevant issues within this complex case study and allow the interviewer to adapt the questions to the varied technical backgrounds of the participants, such as policy or hydrology.

Participants were selected on the criterion that they had knowledge of the decision-making process through experience in the Australian Government or in research related to the SDL. They were selected through purposive sampling, in which researchers select interview participants without a pre-determined numbers of participants (Flick 2007).

There were ten participants across eight interviews. Two participants were employed at the MDBA during the decision-making process and three at CSIRO during the Sustainable Yields Project, all in leadership positions relevant to the SDL decision. Another two participants were employed at the MDBA subsequent to the decision-making process. The remaining three participants were involved in research that was relevant to the SDL. At the time of the interviews, three participants were employed at the MDBA while other participants were employed in research institutions or as independent consultants. Participant backgrounds included hydrology, ecology and policy.

3.6.3. Interview questions and procedure

The interviews followed a semi-structured interview guide with follow-up questions (Adams 2015) to elicit further information or to seek information on apparent gaps in the argument map. Topics covered included: participants’ perspectives of the meaning of uncertainty; description of distinct parts of the decision-making process identified from earlier iterations of the argument map; the sources of uncertainty in the process; how they were managed, and why; possible alternative approaches to managing uncertainty and how uncertainty was communicated to the general public. We adapted the questions to suit each interview participant’s knowledge and interests. Interviews lasted between 20 and 60 minutes, were conducted through video calling technology or in-person and were recorded and transcribed.

3.7. Analysis

The analysis of the argument map and interview data was designed to identify themes and approaches to uncertainty used by the Australian Government in the SDL decision-making process.

To analyse the interviews, we used thematic analysis (Braun and Clarke 2006) and Dedoose software (Dedoose 2020) to iteratively develop a code book with 22 concepts, relating to seven broader themes. These codes did not directly overlap with those used in the argument map. However, the argument map informed the codes used in the interview analysis, in the sense that we looked at the subunits in the argument map and gave more attention to identifying concepts related to them in the interview data. Therefore, we did not take a completely open approach to coding, in which codes emerge exclusively from the data without a pre-determined list, but instead a hybrid approach as described in Rubin and Rubin (2005) in which codes emerge from both the data as well as other sources.

To analyse the argument map, we divided it into subunits where many nodes converged and related to one topic, for example hydrological models. The argument map remained exclusively representative of the document content and did not include data from interviews.

To identify approaches to uncertainty used by government decision makers, we began by looking at each subunit in the argument map for one or more approaches. These approaches came from the argument map, not the interviews, because the argument map was developed from government documents. The criteria for an ‘approach to uncertainty’ were that it was something decision makers did in response to uncertainty and that it could be described without reference to any part of the SDL decision, thus ensuring that it is a generic approach to uncertainty that could be applied in other decisions rather than merely an action in this specific decision-making process. We derived three themes by looking for similarities between approaches to uncertainty and labelling them with key terms that appeared across the documents.

We then used the interview data to expand on the descriptions of each approach and identify any alternative explanations for or interpretations of decisions. Some interview codes, such as ‘politics of climate’, related to only one section of the argument map, while others, such as ‘knowledge gaps’, crossed multiple sections. We also included a theme about communication of uncertainty that did not include specific approaches because it drew on discussion from interviews that spanned the communication across the decision-making process without distinct steps.

3.8. Limitations

Constructing the argument map required a time-intensive iterative process to ensure connection to documents. Due to its complexity, the visual representation of the argument map had to be mainly analysed in parts. The interviews did not, and were not intended, to capture all perspectives or to represent all stakeholder groups. Differing views on uncertainty between interview participants and the interviewer created some challenges in coming to a common understanding on the interview questions, for example with some participants describing or quantifying sources of uncertainty in the data and models. We aimed to ensure that responses covered the implications of those uncertainties for the SDL decision and decision-making process.

4. Results

This section presents the structure of the argument map and then thirteen approaches to uncertainty found in our analysis of the SDL decision-making process, grouped under four key themes. Figure 1 provides an overview of the argument map structure and the subunits within it. It includes subunits with reasoning around the selection of the hydrological models, the climate data, the indicator sites to represent ecosystems in the Basin and the flow indicators to identify important flow conditions at those sites. The conclusions of these four subunits support a process for assessing possible SDL volumes. Along with subunits on selecting which volumes to consider and setting objectives, together they support the determination of the SDL volume. To support our final conclusion as to whether environmental objectives would be achieved, subunits are added relating to adaptive management and recognition that the SDL is a long term average dependent on implementation of the Basin Plan.

Figure 1. Overview of the structure of the argument map showing the main subunits. The arguments in each subunit provide support for those above.

The argument map is available in supplementary material as are the assertions, the associated codes, the documents they were derived from, and the evidence of a connection between assertions used to structure the argument map. All results below where the argument map is referenced originate in the analysed documents and are connected to them through the information in the supplementary material.

From the argument map and interviews, we identified thirteen approaches to uncertainty in the SDL decision-making process, shown in bold below. Some of these approaches may not traditionally be seen as related to uncertainty. They fell into four themes: adaptive management, best available science, trade-offs between uncertainty and objectives, and communication.

4.1. Theme 1. Adaptive management

The MDBA framed the SDL as a ‘starting point’ within an adaptive management framework (MDBA 2011b, p. ix). Adaptive management, a principle of the Basin Plan, was a key part of how the MDBA addressed uncertainty according to both the argument map and interviews. The documents acknowledged many uncertainties within the process but justified the response to uncertainty in the context that they intend to improve the knowledge base, reduce uncertainty and adapt (see argument map). Interview participants noted that the SDL volume would have serious environmental impacts in the timeframe before any planned reviews, and therefore that the initial approach to uncertainty was significant even within an adaptive management framework.

Adaptive management involves planning for future changes such as the legislative requirements for reviews of the Basin Plan and SDL, which could allow for the SDL to be updated as uncertainties are reduced. However, some interview participants had concerns about whether adaptive management of changes to the SDL would occur, for example whether the institutions involved had the necessary supporting culture. One participant said adaptive management is possible but it would depend on political will, leadership and a sense of crisis, such as drought.

The definition of the SDL as a long-term average quantity of water, as opposed to the same threshold every year, allows for flexibility since it leaves more detailed decisions to other mechanisms, particularly the Water Resource Plans developed by states. The SDL was intended to provide the minimum long term average volume of water needed for environmental outcomes, and therefore the MDBA acknowledges that outcomes would depend on the environmental water being delivered in a way that supports environmental benefits (argument map). Uncertainties related to the timing and location of environmental watering were therefore not directly addressed in determination of the SDL.

Accordingly, the interview participants argued that whether and how implementation would occur is an important source of uncertainty for meeting objectives. The legislation does constrain implementation to reduce uncertainty in whether the environmental water will be delivered, for example through monitoring and compliance. However, interview participants referred to recent reports (Wentworth Group of Concerned Scientists 2020) and media such as Four Corners ‘Pumped’ (Ferguson 2017) that raise concerns about less environmental water being delivered than expected under the SDL.

4.2. Theme 2. Best available science

The determination of the SDL began with setting objectives and targets (MDBA 2010, 67–68, 2011b, 23–26). The decision makers appealed to past decisions when setting these objectives, particularly in international environmental agreements (argument map). Interview participants raised concerns, however, with the interpretation of the Ramsar Convention on Wetlands and whether there was sufficient community engagement in setting objectives, including on decisions about which wetlands and ecosystems to protect with the limited environmental water that would be allocated. After the public response to the Guide to the Proposed Basin Plan, the MDBA concluded that the SDL should ‘optimise environmental, economic and social outcomes to achieve a healthy working Basin’ (MDBA 2011b, 101). This decision was seen by interview participants to be contrary to both the mandate to act on the basis of the ‘best available science’ in determining an environmentally sustainable level of take, while also undermining the focus on environmental objectives in the Water Act (2007) as opposed to giving equal weight to environmental, social and economic outcomes (Walker 2019).

For the SDL process, the MDBA mainly repurposed existing data and models to understand the hydrological system, thus taking on their associated uncertainties (argument map). Most interview participants gave a sense of a rushed process between 2007 and 2012 in which decisions were made on the basis of available tools and expedient processes. The MDBA also identified time limitations as a reason for using existing models (argument map). The basis of the hydrological modelling was CSIRO’s Sustainable Yields Project from 2008, which estimated water availability in the Basin under four development and climate scenarios, using existing river models (argument map). Some interview participants described the disadvantages of using multiple models developed for different purposes and with data gaps, arguing that a Basin-wide hydrological model would have been valuable. According to the MDBA, improving the hydrological models would not have reduced overall uncertainty since natural variability and climate change outweighed other uncertainties in the hydrology (argument map). However, interview participants raised concerns about whether these models and data were fit-for-purpose and whether there were missed opportunities to reduce uncertainty, even within the timeframe, such as using remote sensing data.

Climate change was identified in interviews as an important source of uncertainty. In the 2010 Guide to the Proposed Basin Plan the MDBA proposed a 3% reduction in the SDL to address climate change, which was projected to cause a drying trend in the Basin (MDBA 2010). However, the final SDL decision exclusively used a 114 year time series of historical climate data (MDBA 2011b). The reasons presented in the documents included that the historical data allowed for the SDL volumes to be tested under variable conditions, that natural variability would be greater than the impacts of climate change in the planning timeframe and that Water Resource Plans would address climate change (argument map). Interview participants from the MDBA referred to the high levels of uncertainty in the projections at the time. Therefore, the use of historical climate was presented as an example of using more certain information. Other interview participants said uncertainty did not justify the use of historical climate since there was already sufficient certainty before 2012 of declining rainfall in the Basin. They described it as a political decision due to pressure to have a higher SDL (see Theme 3 below), which could not easily be justified with a drying climate. There were also questions in the interviews about the extent to which the SDL could address climate change given that, as a long-term average volume, it is not as responsive to extreme wet and dry periods as other measures can be.

To assess the ecological outcomes of proposed SDL volumes, within the context of limited available data, the MDBA used the hydrological indicator site method to focus efforts on sites where there was greater certainty in ecological and hydrological knowledge. Thus, they based decisions on locations with greater knowledge, which addressed some uncertainty but also involved uncertainty in whether these sites could represent the whole Basin (argument map). Some interview participants identified uncertainties and gaps in the ecological data as a significant problem, such as a need for more mapping of wetlands, floodplain hydrology and threatened species, as well as identification of thresholds of environmental water that ecosystems required.

The final part of the SDL decision making focused on three volumes of water as potential SDLs, thus assessing pre-determined options. The MDBA explained this as allowing for more detailed modelling and analysis of those options (argument map). The three options were a volume at the higher end of the range proposed in the Guide to the Basin Plan, which came from an earlier modelling process, and two alternatives that were ±400 GL/y. Therefore, the potential SDLs assessed involved relatively low volumes of water recovery for the environment. Concerns about socioeconomic impacts of water recovery and community feedback were considered in this decision. The MDBA acknowledged that these options involved higher uncertainty in whether environmental objectives could be achieved (argument map).

4.3. Theme 3: Trade-offs between uncertainty and objectives

The MDBA faced a trade-off between greater certainty in environmental outcomes or better socioeconomic outcomes (argument map). A higher SDL would allow for less water recovery for the environment and therefore less certainty in expected outcomes for ecosystems but also reduced socioeconomic impacts as Basin communities and agricultural interests adjust to having less water available for consumptive uses (MDBA 2011a). The documents explicitly acknowledged accepting uncertainty to balance trade-offs between objectives (argument map). Specifically, when selecting the volumes that would be considered further and when making the final SDL decision, the MDBA accepted higher uncertainty in environmental outcomes to reduce socioeconomic impacts (argument map). Some interview participants placed the decision in the context of the political climate in which the decision-making process took place, with the final decision needing to be approved in legislation by the Australian parliament. The quantification of targets related to cultural flows was excluded from the analysis, citing uncertainty as the reason (argument map).

The MDBA identified several ‘flow indicators’ for each hydrological indicator site, describing volume and duration of flow as outcomes of potential SDL volumes, and determining target frequencies for flow indicators. Instead of a single target frequency, they provided a range by listing a ‘high uncertainty’ and ‘low uncertainty’ target for each flow indicator, reflecting the level of certainty in reaching environmental targets. The justification was uncertainty in the frequency needed to support outcomes for ecosystems (argument map).

For each of three options, the MDBA modelled allocating a given volume of water to environmental flows instead of consumptive use using the hydrological models and historical climate data. They generated a list of flow events, such as a volume of water in a specified time period, that occurred only in a ‘without-development’ scenario (hydrologic models approximating the Basin in its natural state with no consumptive use) but not in a scenario ‘with current human development’. For each proposed SDL volume, a tool selected a subset of these events that could be reinstated at each hydrological indicator site with the aim of achieving the target frequency for flow indicators. The MDBA selected a defensible approximation of reality by making assumptions about the timing and location of water recovery with events limited to those that the Australian Government could achieve through the hypothetical purchase of water entitlements with selected assumptions (argument map).

The MDBA assessed the three possible SDL volumes by considering whether flow events were successful in the modelled outcomes. An event could be considered ‘successful’ where over 90% of the volume was delivered. Partially successful events were also given consideration. They explained this as a response to uncertainty in the thresholds of environmental water required to achieve an environmental outcome (argument map). Therefore, they were evaluating results based on awareness of uncertainty.

However, some interview participants said this made it unclear how or even whether the scientific knowledge and modelling supported the decision since it was not apparent under what circumstances a volume option would or would not have been accepted as the SDL. Some interview participants said that allowing for a range of indicator frequencies (see above) and allowing for the partial success of events supported a politically acceptable SDL decision that in their view did not allow sufficient water recovery to meet environmental objectives, particularly in relation to wetlands. They argued that uncertainty was used as an excuse and that reducing uncertainty would not have impacted the decision since there were political considerations driving the outcomes of the process.

4.4. Theme 4: Communication of uncertainty

The argument map captures numerous acknowledgements of uncertainty across the documents analysed in relation to the SDL and its outcomes. As noted in Theme 1, the SDL decision was framed in the context of uncertainty as a ‘starting point for adaptive management’ (MDBA 2011b) rather than presenting it as a final and certain decision.

However, according to some interview participants there is a common public misconception that the expected environmental outcomes of the SDL are relatively certain; when looking to all forms of communication, the MDBA did not communicate uncertainty well to the Australian public. Participants noted the challenges of communicating uncertainty in a broader context in which policy makers are under pressure to justify their decisions and appear certain.

Some participants also argued that there was limited transparency in the whole decision-making process, making it difficult to understand why decisions were made and therefore to understand how uncertainty was managed. Participants discussed the importance of community involvement in decision making for the SDL, particularly in setting objectives, and that community engagement prior to the 2012 SDL decision was not adequate although some noted improvements after the response to the Guide to the Proposed Basin Plan (MDBA 2010).

5. Discussion

The determination of the SDL was a massive undertaking, drawing on a knowledge base with numerous sources of uncertainty to reach an agreement on a specific number for each of the defined catchments. It therefore contains many positive actions as well as lessons to be learnt regarding approaches to uncertainty. The recommendations in this section provide a broad canvas to pinpoint areas for improvement in relation to each of the four themes rather than a single way forward. They have been written to inform potential review of the surface water SDL in the Review of the Basin Plan. They also remain relevant to other water resource decision-making issues characterised by various types of uncertainty, multiple socio-environmental objectives, multiple stakeholders and perspectives, and an intention to be informed by scientific research.

The recommendations are drawn from analysis of the government decision-making process and are not intended to represent all stakeholder perspectives. Given that communication of uncertainty is a key theme discussed, further research could usefully consider how uncertainty in the SDL was interpreted by the MDBA’s stakeholders, and how they may be engaged in future decision making concerning how to address uncertainty in the decision-making process. There is an implied sequencing to these recommendations – earlier recommendations must be addressed before others can realise their full benefits.

5.1. Communication

5.1.1. Foster public awareness of uncertainty

Although the documents analysed largely acknowledged uncertainty, they played a limited role in shaping stakeholder perspectives. Moreover, the documents do not easily reveal the approaches taken to address uncertainty. There were also limitations in regard to the communication of uncertainty prior to the 2012 SDL decision, as identified by interview participants and others (Crase, O’Keefe, and Dollery 2013), suggesting concerns with transparency and legitimacy, which are principles of good governance (Lockwood et al. 2010). Legitimate decision-making processes have a clear and transparent connection to the democratic system, supporting legislation and government agencies (Lockwood et al. 2010). Transparency around uncertainty, rather than attempting to control it, is critical for decision making in contested spaces (Funtowicz and Ravetz 1993; Scoones and Stirling 2020).

The importance of uncertainty communication is well-established (see for example Van Asselt and Rotmans 2002). However, the SDL decision-making process fits a broader pattern of experts and decision-makers not drawing attention to uncertainty due to pressures to appear certain about the benefits of policies and the underlying scientific advice (Stirling 2010). But underestimating sources of uncertainty in setting SDLs could have serious impacts on communities, for example on the ability of farmers to make well-informed decisions about the future based on water availability. Further efforts could consider public and decision-maker perspectives on the transparency of the approaches used to address uncertainty, whilst acknowledging the complexities associated with applying a mandate to follow the ‘the best available science’ in contexts plagued by uncertainty.

5.1.2 Support open dialogue concerning uncertainty management

There is a need for agreement and understanding of how uncertainty should be addressed in decisions related to the SDL given multiple socio-economic objectives and an adaptive management framework. This should be advanced taking stakeholder views into account. With post-normal science, transparency about uncertainty and dialogue about the normative aspects of a decision are essential (Funtowicz and Ravetz 1993). Uncertainty categorised as ‘ambiguity’ is related to different frames of reference between people and therefore can only be reduced by processes to come to agreement or to work with the different views (Dewulf and Biesbroek 2018). Procedures could be introduced to ensure a common understanding of how uncertainty should be managed and to ensure that there is accountability, transparency and awareness of the contested nature of the decision when the Basin Plan is revised (Alexandra 2020). The need for dialogue specifically regarding trade-offs and best available science is further discussed below.

5.2. Adaptive management

5.2.1. Assess and strengthen capacity for adaptive management

The MDBA delegated uncertainty management to the future through adaptive management, expecting future actions to address current gaps in the knowledge base. This requires confidence that effective adaptive management will occur, a conviction which some interview participants questioned. Adaptive management originated with Holling (1978) as an iterative process that reduces uncertainty by learning from monitoring of previous decisions and adjusting them accordingly. It is often aspired to (e.g. Rist et al., 2013) but difficult to implement because institutions tend to lack the capacity to learn and not have a culture that is willing to accept failure (Williams 2011). Others have noted the critical role that adaptive management has in the Basin Plan (Crase 2012; Hart 2016) and specific challenges such as multiple jurisdictions and organisations (Thompson et al. 2019) and dealing with a variable climate and climate change (Hart 2016).

Building confidence in and strengthening the capacity for adaptive management is therefore required for institutions and individuals involved in implementing, reviewing, and possibly amending the SDL. This includes tackling how responsibilities for addressing uncertainty in achieving outcomes are shared between the MDBA, Basin states, water users, and other supporting institutions.

A critical first step would be to further assess the existing capacities and challenges, including the ability to progressively improve scientific knowledge over time with dedicated funding and a strategic integration of all monitoring, and to use that knowledge to amend the SDL. Some improvements to adaptive capacity may already be occurring, so are not covered in this analysis. However, the two notable examples of adjusting the SDL, the SDL Adjustment Mechanism and the Northern Basin Review (Commonwealth of Australia, 2017; MDBA 2016) both resulted in less water directly allocated to the environment and did not address all concerns raised in the interviews, such as the effect of climate change on water availability.

5.2.2. Fund monitoring to enable adaptive management of implementation

Crucially, the monitoring component of adaptive management should, as a fundamental step, consider whether volumes of environmental water are being delivered as expected in the implementation of the SDL and whether they are causing the expected ecological responses. The need for ongoing monitoring and follow-up is a common challenge to adaptive management (Carl 2007; Keith et al. 2011; Westgate, Likens, and Lindenmayer 2013). There is evidence of gaps in the associated data and that not all water is being delivered to the environment as intended (see for example Wentworth Group of Concerned Scientists 2020). There has been some major monitoring activity, such as the Commonwealth Environmental Water Office’s Long Term Intervention Monitoring Project (Hale et al. 2020). Although evaluation by the MDBA (2020) reported progress on environmental outcomes under the Basin Plan, it also noted that some components of the Basin Plan have not been implemented yet.

5.2.3 Acknowledge the challenges in changing the SDL volume

Adaptive management in relation to the SDL itself, rather than its implementation, would involve changing the SDL volume. Depending on how it is adjusted, that action could have major social, ecological or economic impacts. Moreover, adjustments are difficult to achieve, given the extent of the decision-making process and the need for support in the Australian parliament. The political conflict leading to the 2012 decision might mean there is limited willingness to repeat the process to modify water allocations to the environment. Even if the SDL is not updated, the argument map and interviews did place the SDL within a range of measures for managing water in the Basin Plan and by Basin states. The process of allocating water already has adaptive elements, with water allocations reduced when there is less water available.

Alternative ways to conceptualise the SDL, instead of as a long-term annual average, warrant being explored. Although there is some potential to improve outcomes without changing it, the SDL, to the extent that it is enforced, is the ultimate limit of water extraction in the Basin. Therefore, desired outcomes will be difficult to attain unless the SDL aligns reasonably well with overarching objectives, particularly in a changing climate.

5.3. Trade-offs between uncertainty and objectives

5.3.1. Dialogue on handling of trade-offs in the presence of uncertainty

The allocation of a finite resource inherently requires trade-offs between which socio-environmental values should be maintained. And once a hierarchy of values is established, these should be reflected within the objectives that guide decision making. However, in this case, debate over the triple bottom line interpretation of the Water Act created uncertainty in the overarching objectives, which interacted with uncertainty in the scientific knowledge and created space for trade-offs. Although the decision-making process involved extensive work in setting objectives at different spatial levels, there was not consensus or clarity on them, as reflected in the interviews and in the acknowledgements of subjective and value-based decisions throughout the argument map. One way forward is to specify goals at the level of the Basin Plan that are specific, measurable, achievable, realistic and time bounded, or ‘SMART goals’, as used in the strategy for managing environmental flows (MDBA 2019).

As the interpretation of the ultimate objectives in the Water Act is still in disagreement, and the debate around trade-offs therein are likely to be exacerbated in a changing climate, there is a need for broader dialogue and agreement on the desired trade-offs between socioeconomic and environmental objectives, which may change over time. While decision-making around trade-offs should be informed by science, ultimately these are value judgements that require institutional arrangements that enable inclusive dialogue about what should be prioritised (Pahl-Wostl, Palmer, and Richards 2013). From one perspective, an approach to policy that embraces uncertainty can be an opportunity to challenge dominant ideas of progress and engage with a plural vision of the future (Scoones and Stirling 2020). Further work is also needed to consider how uncertainty relating to different objectives should be handled under the existing legislation or by amending the legislation to provide clarity.

5.3.2. Reduce uncertainty around objectives

Since objectives at a Basin scale can be particularly difficult for stakeholders to agree on, alternatives or complementary approaches to the SDL should also be considered. For example, smaller scale objectives, such as specific environmental water targets, could be an area with greater consensus. Additionally, there could be a triage approach (Colloff and Pittock 2022), differentiating aspirations and red lines, and identifying tipping points, such as Adaptation Tipping Points for drying wetlands (Nanda et al. 2018). A methodology for iterative discovery of such options (Fu, Guillaume, and Jakeman 2015) could allow for feasible management targets to be identified. Following the dialogue on handling trade-offs (see above), well-specified environmental, economic and social objectives should be developed and given a social licence through this process. There should be community engagement on envisaging the desired future state of the Basin, including serious consideration of Indigenous cultural values.

Coherence between the modelling approach behind the SDL and the policy and water management approach is required, where the former used key sites and assumptions about water delivery to assess whether objectives could be met, and the latter may in reality deliver water in ways that differ from the assumptions on which the SDL was determined. The Basin-Wide Environmental Watering Strategy (MDBA 2019) provides an ongoing effort to make the environmental objectives and targets better specified and able to be assessed.

5.3.3. Consider implications of climate projections within the SDL and beyond

There is a clear need to respond to climate change, which the MDBA has recognised in their evaluation of the Basin Plan (MDBA 2020). Under one interpretation of the precautionary principle (see Basin Plan 2012 s 8.38), action to respond to climate change should not be delayed just because knowledge remains uncertain. The challenges experienced in 2012 remain, since assessments of water recovery based on projected climate have yet to occur at the Basin scale. Our interviews indicated that part of the trade-off to support socioeconomic outcomes was the lack of political appetite to directly consider projected climate, which was conveyed as being too uncertain at the time. This reasoning is further supported in the literature on the Basin Plan (Alexandra 2020) and is consistent with a broad tendency for scientific uncertainty to be used to delay regulations (Freudenburg, Gramling, and Davidson 2008). On the other hand, an independent review of the Guide to the Proposed Basin Plan (Briscoe and Likens 2010) recommended that the MDBA not incorporate climate change until the next Basin Plan, citing both the uncertainties and the added difficulties in decision making and communication if climate change were involved.

Since the Basin is projected to have less and more variable rainfall in future (Grafton et al. 2012; MDBA 2020), the simplest response would be to decrease the SDL. However, the impacts on Basin communities, who are also grappling with decreased allocations as a consequence of reduced inflows, might not be acceptable to decision makers. The existing trade-offs between objectives will become more challenging with less water overall. Additionally, as noted in interviews, the SDL currently reflects an average and thus does not explicitly address increased variability. For ongoing consideration of climate change, conceptualising the SDL as a single number for each catchment might be of little use whereas more responsive strategies in space and time could help.

5.4. Best available science

5.4.1. Transparent definition of best available science

There should be transparency and dialogue around what ‘best available science’ means in a context characterised by uncertainty and value-based decisions. Similar mandates to the one in the Water Act (2007) around best available science are common internationally but often difficult to interpret (Bisbal 2002; Esch et al. 2018). In theory, as discussed also in the context of the Basin Plan, a mandate can support transparency and evidence-based decisions (Lindsay 2020) but it can also give the public a misleading impression that the decision is objective rather than value-based (Doremus 2004). For complex problems such as setting the SDL, scientific knowledge is just one consideration of decision makers (Evans et al. 2017) and it cannot have an effective role while normative values remain unarticulated (Sarewitz 2004). In the context of the 2012 SDL decision, some interview participants did not think that reducing the scientific uncertainties would have influenced the decision-making process. The SDL is a normative decision that can be informed but not dictated by science.

5.4.2. Improving best available science

However, improving scientific knowledge to reduce epistemic uncertainty can still have an important role if community engagement regarding the normative underpinnings of objectives happens in parallel. An important step is to further assess all uncertainty (Marchau et al. 2019) and rank their priorities. There was analysis of uncertainty in the models adapted for the SDL decision-making process (see for example Van Dijk et al. 2008), but it was largely focused on listing and, where possible, quantifying specific sources of uncertainty. The literature cautions against overemphasising quantifiable uncertainties as they may have the smallest impacts (Bark et al. 2013; Walker et al. 2003) and highlights the significance of considering decision making under deep uncertainty, with a greater focus on exploring how uncertainties could impact on outcomes of decisions (Lempert, Popper, and Bankes 2003; Marchau et al. 2019). The Intergovernmental Panel on Climate Change recommends using calibrated language to express the degree of confidence and to quantify the likelihood probabilistically (Mastrandrea et al. 2010).

A fulsome analysis of uncertainty that distinguishes the critical sources can help identify what data should be collected. Within the adaptive management framework, increased investments to address knowledge gaps should be considered where appropriate from a cost-benefit perspective. Some uncertainties are irreducible, such as natural variability. Some uncertainties that can be reduced may have a relatively small impact on the SDL and thus reducing uncertainty in them may not reduce the overall uncertainty. Rather than aiming to reduce all uncertainties, the cost and benefits should be considered in relation to the nature of the uncertainty and its impact on overall uncertainty. Some work is already occurring through the Murray-Darling Water and Environment Research Program, consistent with recognition by the MDBA of gaps in scientific knowledge and monitoring for adaptive management (MDBA 2020). It includes research on climate change adaptation, hydrology and environmental outcomes to support better models and understanding for review of the Basin Plan. To help ensure that this and other scientific work is relevant to and impactful on policy, scientists and policymakers should develop networks that build trust over time and encourage co-production (Cairney, Oliver, and Wellstead 2016).

6. Conclusions

Major efforts are needed to identify a clear path forward in regard to uncertainty management in relation to the SDL decision-making process. As noted above, our recommendations canvas fruitful areas to examine. Our analysis has focused on uncertainty management in relation to the 2012 surface water SDL, which was in itself a significant decision but which also interacts with many other measures to manage the Basin’s water resources at different scales. It has also considered relevant decision making within the Australian government. Future work could gainfully take into account uncertainty in a broader range of decisions within the Basin, especially where they are intended to be evidence-based, and could directly include the perspectives of a broader range of stakeholders, including Indigenous Australians.

The challenges of the decision-making process to establish the 2012 SDL highlight the importance of transparency about the trade-offs between socio-economic objectives, the uncertainties in the best available science, and how uncertainty is to be managed in that context. Without transparency, uncertainty can create space for political pressures for a higher or lower SDL. A successful adaptive management approach relies on a capacity to adapt within institutions and may primarily depend on mechanisms that are more flexible than the SDL. Within a more transparent and participatory process, improved data and models can have an important role in meeting the ‘best available science’ mandate, particularly in filling the gaps related to the consequences of climate change for the Basin. But such an inclusive process also can be designed to clarify and share priorities as to what uncertainties matter.

Declaration of interest statement

The authors report there are no competing interests to declare.

Acknowledgements

We thank the interview participants for their contributions. We thank four anonymous reviewers for their useful input in improving the paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The work was supported by the Australian Research Council [DE190100317]; Australian Research Council [DE2001922].

Notes on contributors

Leila H. Noble

Leila Noble is a PhD candidate at ANU’s Fenner School of Environment and Society and Institute for Water Futures. She is interested in how to sustainably manage water resources to meet multiple needs under uncertainty and climate change.

Joseph H.A. Guillaume

Joseph Guillaume is a Fellow at the Fenner School of Environment and Society and Institute for Water Futures, ANU. He specialises in uncertainty management in decision support, with a particular focus on water resources and the use of integrated modelling.

Carina Wyborn

Carina Wyborn is an Associate Professor with the ANU’s Institute for Water Futures and Fenner School of Environment and Society. She is an interdisciplinary social scientist with background in science and technology studies, and human ecology. Carina’s research examines how land and water managers make decisions about future environmental change, focusing on the intersection of science, politics, policy, and practice. She currently holds an ARC Discovery Early Career Research Award, which involves research on foresight practices and anticipatory governance to identify methods that enable diverse actors to negotiate shared pathways for water reform in the Murray Darling Basin.

Anthony J. Jakeman

Tony Jakeman is Emeritus Professor with the ANU’s Institute for Water Futures and Fenner School of Environment and Society. He is an interdisciplinary scientist with background in computational mathematics and modelling. He has long-term experience in integrated assessment and modelling with most focus on water resource systems and hydrology.