Embracing Uncertainty in Policy-Making: The Case of the Water Sector

Abstract

Recent research on policy-making under uncertainty in the water sector has contributed to our understanding of types and sources of uncertainty as well as to the development of tools and approaches to manage uncertainty. This paper reviews the literature and identifies several strands of systematic bias, notably an emphasis on natural sources of uncertainty over human sources; a tendency to treat sources of uncertainty as independent and a corresponding neglect of interaction between sources; and a focus on tools to reduce or contain uncertainty rather than to accommodate it. The papers in this issue contribute to overcoming these biases.

1 Introduction

Rising concerns over climate change have placed policy-making under uncertainty in the spotlight in recent years (Hall et al., 2012; Polasky, Carpenter, Folke, & Keeler, 2011; Yousefpour et al., 2012). On the one hand, while there is no doubt that greenhouse gas emissions will have a major impact on climate, scientists and researchers have yet to settle on predictions of the extent of climate change in the near future, an essential condition for decisive action on the issue (Knutti & Sedláček, 2013; Weitzman, 2009; Whitmarsh, 2011). On the other hand, the stakes of delaying actions would be prohibitively high due to the magnitude of the potential impact. An approach of avoiding or ignoring uncertainty by repeatedly putting off decisions, a common response by policy-makers faced with uncertainty, would be grossly inadequate with regard to climate change (Adger et al., 2011; Eriksen et al., 2011; Hobson & Niemeyer, 2011; Morton, Rabinovich, Marshall, & Bretschneider, 2011).

Recent research on the typology of uncertainty has greatly contributed to improving our understanding of the nature of the challenges in dealing with uncertainty in policymaking (Koppenjan & Klijn, 2004; Walker, Marchau, & Swanson, 2010). First of all, there is a high level of agreement among scholars in distinguishing between epistemic and ontological uncertainty, where epistemic uncertainty stems from imperfect knowledge of a system while ontological uncertainty relates to inherent variability and unpredictability in the system itself (Brugnach, Dewulf, Pahl-Wostl, & Taillieu, 2008; Isendahl et al., 2009; Walker, Haasnoot, & Kwakkel, 2013). Such a distinction is critical because it suggests that different methods will be required to deal with different types of uncertainty. For example, more research and technological innovations would help to reduce epistemic uncertainty while little can be done to reduce ontological uncertainty.

In addition, advances have been made in considering a broader range of sources of uncertainty beyond the natural system. Since much uncertainty in policy-making is rooted in imperfect knowledge about human behaviour, as well as inherent variability and unpredictability of such behaviour, and therefore, uncertainty in the economic, social and political systems is just as critical to policy-making as that stemming from the natural system (Brugnach et al., 2008). More importantly, the interplay of uncertainty originating from different systems has added another layer of complexity in dealing with uncertainty in policy-making (Drieschova & Fischhendler, 2012).

Scholars have also broadened the study of sources of uncertainty beyond imperfect knowledge and variability to encompass the subjective nature of uncertainty. The fact that there are multiple stakeholders involved in the policy process, each with their own belief systems, views, preferences and interests, and thus their own interpretations of the same information, gives rise to a new type of uncertainty – ambiguity, “a situation in which a decision-maker does not have a unique and complete understanding to be managed” (Brugnach, Tagg, Keil, & de Lange, 2007).

Despite the advances made in developing a typology of uncertainty, systematic treatment of mechanisms, tools, and techniques in dealing with uncertainty in policymaking has been rare. Techniques taught extensively in public policy schools, such as sensitivity analysis, decision-tree analysis, system dynamics modelling and Monte Carlo simulation, invariably require the specification of not only future states but also their probability distribution, a condition rarely satisfied in the real world of policy-making. While the institutionalist literature has shown how institutions – the formal and informal rules and norms that shape interactions – can be harnessed to reduce uncertainty (North 1993, Calvert, 1995), it has focused mainly on factors external to the policy system, thus paying little attention to how institutions might constrain or exacerbate external sources of uncertainty (Shepsle, 1986).

Furthermore, little effort has been given to address complexities arisen from the co-existence and interaction among multiple sources of uncertainty. Identifying and treating sources of uncertainty in isolation may result in policy that is ill-prepared to deal with a situation in which multiple risks are realized at once. Moreover, while there have been attempts to address epistemic uncertainty resulting from imperfect knowledge and in measuring ontological uncertainty, effort in addressing ontological uncertainty and in dealing with ambiguity resulting from multiple knowledge frameworks has been scarce (Dewulf et al., 2005).

The water sector may provide fruitful ground to assess both the potential and limitations of some recent attempts aiming at dealing with uncertainty in policy-making. First of all, as an essential resource for human survival, the sector has received a disproportionately high level of attention unmatched in many other sectors, especially in countries and regions encountering water shortages, and substantial effort has been made in dealing with uncertainty in policy-making in water sector. Concerns about climate change reinforce this, as much of the impact is expected to be felt through the availability and quality of water (Stern, 2007).

In addition, there is a wide spectrum of policy sub-fields within water sector, such as flood prevention, security of urban water supply, and resource allocation in international river basins, each with its own unique set of characteristics with regard to nature and sources of uncertainty. Thus, experience in the sector may offer lessons useful for a range of other policy fields.

Lastly, the understanding of the sources of uncertainty has extended beyond variability in the natural system such as the hydrological cycle, to examine imperfect knowledge and variability in economic, social and political system, as decision-making in the water sector is no longer dominated solely by hydrologists and engineers. The broadening of participation in policy-making in the water sector has resulted in new thinking about how to deal with uncertainty that might be of interest to other sectors.

This theme issue brings together a collection of papers addressing uncertainty in policy-making in the context of water sector. The papers extend our understanding of uncertainty by considering political, social and economic sources of uncertainty in addition to natural sources, and by considering subjective uncertainty in multi-actor decision processes. Furthermore, they draw attention to how some of the mechanisms adopted by decision-makers to deal with uncertainty – laws, regulations, contracts – may in fact be subject to uncertainty themselves.

The papers also contribute to the evaluation and extension of tools for policy-making under uncertainty, showing how and under what conditions innovative methods can be successfully applied and scaled up. In order for these tools to be genuinely useful for policy-makers in the policy design process, their value needs to be demonstrated and policy-makers need to have confidence in the soundness of the methods.

A theme running through all the papers in this issue is the value of acknowledging and working with uncertainty in the policy-making process, rather than seeking at all costs to reduce it. By recognizing the limits of their knowledge, decision-makers may give themselves greater flexibility and better prepare the societies they govern for an uncertain future.

2 Embracing uncertainty in policy-making in water sector: a review of literature

Spurred by the urgent need to tackle water security issues in many countries, and a deepening understanding of climate change, considerable effort has been made in recent years to address uncertainty in policy-making in water sector. Advances are being made in the use of information technology, analytical approaches, managerial practices and policy systems. We review recent literature in each of these areas in turn in the following sections.

2.1 Advances in the use of information technology

Advances in the use of information technology can play a key role in addressing epistemic uncertainty. Access to satellite data, for example, has become easier and less expensive, opening up the possibility for national authorities to monitor surface and groundwaters effectively (Guzinski et al., 2014; Houghton-Carr & Fry, 2006; Rodell & Famiglietti, 2002) and to improve the accuracy of hydrological models (Van Dijk & Renzullo, 2011). This will be particularly useful in developing countries where traditional data collection and management systems are more limited (Houghton-Carr & Fry, 2006).

More data can also help to reduce uncertainty due to social and economic factors. At the utility level, for example, smart water metres provide the potential to understand the water usage patterns of individual consumers (Britton, Stewart, & O’Halloran, 2013), and potentially to inform the design of incentives to increase water efficiency (Cole & Stewart, 2013). Larger volumes of data allow for better calibration of existing models. Advances are also apparent in increased processing capacity, through integration or harmonization of computational models (Harvey, Hall, & Peppe, 2012; Jones & Page, 2001) and more extensive use of risk-based sampling (Dawson, Hall, Sayers, Bates, & Rosu, 2005).

It is important to note, however, that more information does not necessarily reduce uncertainty. It may just lead to a greater recognition of the extent of uncertainty or locating its sources, or to create an illusion of certainty where the reality is very uncertain (van der Keur et al., 2010).

Several authors point to the need to improve models to take account of uncertainty while ensuring that they remain practical for use by decision-makers, through data visualization tools, integration of expert opinion and linking models across academic disciplines (Buckley & O’Kane, 1998; Deletic et al., 2012). In this respect, there are clear differences between facets of water policy: while uncertainty about climatic variability is comparatively well integrated in riverine and coastal flood modelling, less attention has been given to surface flooding or urban drainage modelling (Buckley & O’Kane, 1998; Deletic et al., 2012). Similarly, data and models of social and economic sources of uncertainty, such as household water demand or permeable surface area, are limited.

2.2 Advances in analytical approaches

Standard statistical analysis may be adequate to manage some ontological uncertainty – uncertainty related to random variation – when long data records are available and the probability distribution is known and static. However, these conditions are unlikely to be met in relation to many aspects of water policy (Wasson, in this issue). Climate change makes it impossible to hold to the stationarity assumption for natural phenomena, and stationarity has arguably never been an appropriate assumption for non-natural sources of uncertainty.

There is a growing literature on the application of alternative probability distributions (fat-tailed distributions),and extreme value statistics (Katz, 2010; Shah & Ando, 2015; Te Linde, Aerts, Bakker, & Kwadijk, 2010; Towler et al., 2010) and simulation of low probability events through resampling data (Te Linde et al., 2010) in relation to different aspects of water policy. Alternatives to frequentist statistics, notably Bayesian inference, are also receiving growing attention (Freni & Mannina, 2012; Thyer et al., 2009). However, these techniques all still stand outside the mainstream of policy design and a review of the literature did not uncover any examples of the application of Bayesian statistics in policy design. Further work will be needed to show how they can be used in practice in a policy context.

Traditional methods of policy selection like cost-benefit or cost-effectiveness analysis usually take into account uncertainty only in a limited fashion, if at all, perhaps by conducting sensitivity analysis in a small range around a central forecast. These methods therefore cannot demonstrate the value of flexibility. A number of methods are being developed to address this, such as real options analysis (ROA). Real options valuation assigns a financial value to flexibility, making it possible to integrate it more easily into standard cost-benefit approaches. In the water sector, ROA has proved to be a useful tool to compare infrastructure investments, and to conduct comparative evaluations of packages of hard (infrastructure) and soft measures and in the timing of investments and in determining the scale and location of investments. It has been applied in relation to water supply planning (Zhang & Babovic, 2012), major water resource infrastructure investments (Jeuland & Whittington, 2014), coastal flood defence (Gersonius, Ashley, Pathirana, & Zevenbergen, 2013; Kontogianni, Tourkolias, Damigos, & Skourtos, 2014; Linquiti & Vonortas, 2012) and urban drainage (Deng et al., 2013; Park, Kim, & Kim, 2014).

ROA can accommodate different sources of uncertainty, depending on how the scenarios are defined, including political, social and economic variables (Zhang and Babovic (2012), Wu et al. (in this issue). ROA can also been used in combination with other approaches, such as adaptation pathways (Buurman and Babovic, in this issue).

The challenge for ROA is not so much in the refinement of the technique but in gaining acceptance of the method in the policy design process. ROA is data and skill-intensive and requires considerable processing capacity. To be applied, policy advisors need to have the requisite skills and resources to conduct the analysis, and decision-makers need to have the financial capacity and the knowledge to commission the analysis, interpret and use the results. While this may be the case in some policy domains, like environmental policy, where officials themselves may have a scientific background, it may well not be the case at higher levels of decision-making and in non-scientific policy fields.

Scenario-based planning is increasingly widely employed in policy design. In the water sector, the scenarios themselves are often defined in climatic terms but may also include different assumptions on costs and availability of technology, demand (population, economic growth), political, regulatory and behavioural trends. Scenario selection can take place through stakeholder consultation (Reeder & Ranger, 2011) or through more formal techniques like algorithm-based scenario discovery (Groves & Lempert, 2007). Scenarios can be employed to compare the costs and benefits of different investment packages under each scenario. However, this approach does not allow for comparison between the packages if no further information is available on which scenario will pertain.

2.3 Advances in policy design

Alongside these advances in tools and techniques, there is growing interest in how the policy process itself can be refined or reshaped to deal better with uncertainty. These approaches have been developed in relation to climate change generally (Hallegatte, 2009; Walker, Rahman, & Cave, 2001; Walker et al., 2010) but scholars working in the water sector have been at the forefront of method development (Groves & Lempert, 2007; Haasnoot, Middelkoop, Offermans, van Beek, & van Deursen, 2012; Moench, 2010) and in demonstrating how they could be applied (Fletcher et al., 2013; Kwadijk et al., 2010).

On the one hand, this might involve deeper and broader involvement of stakeholders in water governance under structures of co-management and collaboration or co-delivery (Huitema et al., 2009). Other approaches call for a re-conception of management from prediction and control to “adaptive management” and place “learning” at the centre of policy design (Pahl-Wostl, 2007). In this more inclusive interpretation of the process, learning is not just the preserve of policy-makers but also involves social learning and the participatory assessment of resource management measures.

The importance of learning from monitoring and review is echoed in the work of Walker et al. (2013). They call for plans to be continuously monitored and updated to take into account new information but point out that it may be difficult to do so in practice because of the considerable resources required and potential resistance to planned policy adaptation from actors responsible for policy implementation. Furthermore, a lack of operational method and standardized procedures (Kato & Ahern, 2008) and a framework for their evaluation (Gregory, Ohlson, & Arvai, 2006) for these adaptive approaches may hold back their application. The explicit adoption of adaptive management as an approach to policy-making in the water sector in some places will provide an opportunity to assess the usefulness of these approaches in future studies.

Another promising approach to policy-making under uncertainty is “adaptation tipping points” (ATP). ATP offers an alternative approach to top-down scenario-based planning – rather than analysing which policy package performs best for a given scenario, ATP seeks to identify the point at which a policy package would become inadequate (Kwadijk et al., 2010). ATP is thus an effective way to convey to policy-makers the effectiveness of current water management approaches given a certain set of policy objectives.

Adaptation pathways is a complementary approach, in which tipping points are mapped into pathways, indicating at which point policy-makers would need to take decisions to add extra measures in order to meet policy objectives. Haasnoot et al. (2012)et al. apply the adaptation pathways approach to Rhine delta water management under uncertainty about rising sea levels. Instead of working under the assumption of a certain level of sea level rise, the authors begin with the policy objective and under what conditions that objective continue to be met. From there, they identify the conditions under which these objectives would cease to be met unless some policy measure are adapted. These policy measures are explicitly identified in advance. Similarly, Walker et al. (2013) and Swanson et al. (2010) demonstrate the applicability of adaptation pathways on three different river deltas that may suffer from floods and droughts.

These approaches need not be seen as mutually exclusive. Instead, they may be complementary to one another. For example, adaptation pathways can be combined with ROA, which makes it possible to integrate the value of flexibility into the analysis (Reeder & Ranger, 2011).

2.4 Gaps in the literature

This review reveals certain strands of systematic bias in the way that policy-making under uncertainty is analyzed and addressed. Firstly, there is a strong bias towards to the development and demonstration of methods rather than the comparative evaluation of their application. Given the relatively recent nature of some of this literature, and the limited number of examples in which policy decisions have actually been made based on the application of these methods, it is perhaps not surprising that much of it is normative – authors are seeking to recommend approaches for adoption by policy-makers which they claim are superior based on the sophistication of the method itself, rather than by demonstrating its usefulness in selecting more effective policies.

Secondly, there is a bias towards methods which address physical sources of uncertainty over human and behavioural uncertainty. This is particularly striking in relation to flood risk, for which climactic and physical data collection and modelling can be highly sophisticated but little effort is made to link this up with the building regulations, spatial planning processes and so on which are critical in determining the degree of vulnerability to floods. Furthermore, there is a tendency to treat sources of uncertainty as independent of one another. Yet, sources of uncertainty may often be part of an interlinked system. Political uncertainty may interact with climatic uncertainty, as Wu et al. (in this issue) show. It can also be seen in sharp relief in the case of Jakarta's water supply, where political, macroeconomic, physical and behavioural factors interacted to undermine service quality and investment (House, in this issue).

A further strand of systematic bias in the literature is the focus on sources of uncertainty external to the policy process, and the minimal attention paid to uncertainty within the policy process itself. An assumption seems to lie behind many of the papers on methods that decision-makers take proportionate decisions, based on evidence, and using a long-term planning horizon. In a democratic system with short electoral cycles, this is unlikely to be the case much of the time. This neglect of uncertainty within the policy process runs the risk of generating policy recommendations which are unrealistic and rejected by decision-makers.

Finally, uncertainty is usually framed as a problem for policy-makers and the literature focuses on negative shocks. Less attention has been paid to the possibility of beneficial shocks and how policy can take advantage of these. Furthermore, the recommendations associated with adaptive policy-making, such as co-management and greater involvement of non-government actors throughout the policy design process may also contribute to improved governance overall.

3 Contributions in this Issue

Wasson's paper addresses epistemic and ontological uncertainty in relation to flood management. He shows how the dominant paradigm in long-term flood risk forecasting is fundamentally inadequate in its ability to predict and help societies prepare for future events, notably low-probability, high-impact events, leaving societies at risk. In his thorough, critical review of dominant practice in the sector, Wasson demonstrates how limitations in the data employed, the probability distributions assumed, without a satisfactory understanding of the physical causes of floods, reduces the effectiveness of flood risk management.

Looking to the future, he suggests a number of ways in which flood forecasting could be improved, by drawing on additional sources of data, such as historic and paleogeological records, and alternative probability distributions. However, he points out that the flood forecasting is almost exclusively focused on epistemic and ontological uncertainty in relation to natural systems. While social, economic and political sources of uncertainty are neglected in the analysis, policy-makers understanding of the vulnerability will be limited, as will their ability to select effective policies.

Leong and Qian's paper is explicitly concerned with political uncertainty surrounding water supply and the interaction between this source of uncertainty and natural, social, economic and technological uncertainty. Looking at the historical development of Singapore's water supply policy over the last 50 years, they show how the effectiveness of the city state's water policy has been in part due to the recognition of uncertainty and the adaptive ‘learning by doing’ forms of policy-making that have been followed. In so doing, they challenge the prevailing view, which presents water policy in the city state as a clear, long-term strategy. The case provides support for experimentation, evaluation and the updating of policies and plans to take account of new information in situations with multiple sources of uncertainty.

Wu, Jeuland and Whittington's contribution brings to the fore the subjective nature of uncertainty and the complexity generated by multiple decision-makers in the context of an international river basin. Using the case of the Nile, they develop a hydro-economic model and examine optimal policy decisions on water use for each of the riparian countries under different scenarios, providing important new understanding of governments’ decisions when there is considerable ambiguity surrounding the future objectives and behaviour of other governments, in addition to epistemic and ontological uncertainty.

The paper draws attention to the interaction between sources of uncertainty in complex systems – in this case, hydrological, economic and political uncertainty. The case of the Nile in which riparian countries are heavily inter-dependent serves as a valuable example of these complex systems and helps to explain why seemingly optimal policy solutions cannot be reached. It remains unclear, however, how this problem might be overcome. Certainly, actions to reduce political uncertainty, such as a water treaty, or more open sharing of data, might help countries to move to more optimal cooperation across the river basin.

House's paper addresses subjective uncertainty in a multi-actor framework at a different scale: that of the urban water utility. Examining the case of a public-private partnership contracts for the provision of water services, she shows how the central mechanism intended to reduce uncertainty from – the contract – was itself subject to uncertainty and thus failed to restore the viability of the concession in the absence of other supportive institutions. As in the case of the Nile, uncertainty in the Jakarta PPP stemmed from many sources. Hydrological uncertainty was relatively easier to manage than economic, social and political uncertainty.

The analysis also brings to light a parallel with Wasson's critique of traditional flood forecasting, as the parties involved in negotiating the original contracts considered only a very limited range of future scenarios. Ontological uncertainty was taken into account, but only in the limited sense of considering limited variability around a central forecast value. Epistemic uncertainty, meanwhile, was addressed through a process of contract ‘rebasing’ which would take place every five years to adjust prices according to new information, but the absence of transparency and verification mechanisms made the information revelation process ineffective.

Buurman and Babovic show how tools and techniques for policy-making under uncertainty can be combined effectively to operationalize the adaptive policy-making approach. They show that while the adaptation pathways approach can help policy-makers to optimize the timing and sequence of multiple interventions, one of its limitations is that it does not provide for the evaluation of the initial selection of policy instruments or the preferred pathway. They address this by combining adaptation pathways with ROA, which allows for the quantification of the value of flexibility. ROA and adaptation pathways can then be built into a framework of adaptive policy-making, in which the performance of the instruments selected is monitored and considered against new information about the environment, as it becomes available.

Their paper is mainly concerned with external sources of uncertainty rather than uncertainty and ambiguity within the policy process itself. Future research might consider how policy-makers interpret and act on the results of their combined ROA & ATP assessment given their subjective viewpoint.

The papers in this issue demonstrate the broad spectrum of uncertainties faced in policy-making in the water sector and a range of perspectives. They serve to emphasize the very considerable risks in the sector which have in the past been masked to some extent by the certainty with which scientific or engineering analysis has been presented to policy-makers.

Taking forward the themes raised here, there is a need to delve further into the structure and process of a policy design to support learning and adjustment over time. Learning needs not just to relate to new information about trends but also to a deeper understanding of the complex systems underlying policy fields, links and feedback loops, as well as changing societal preferences.

It is clear that operationalizing these innovative approaches will be very challenging, not least because of the difficulties in understanding and communicating uncertainty from the researcher, to the advisor, to the policy-maker and to the public. Despite the benefits of innovative methods and approaches, policy-makers will continue to face tough trade-offs between alternative objectives and between costs and benefits and will need to define the acceptable level of risk and who will carry the risks.