Are climate models “ready for prime time” in water resources management applications, or is more research needed?

Abstract

Citation Kundzewicz, Z. W. & Stakhiv, E. Z. (2010) Are climate models “ready for prime time” in water resources management applications, or is more research needed? Editorial. Hydrol. Sci. J. 55(7), 1085–1089.

Many issues related to climate change and its impacts, mitigation and adaptation raise considerable controversies. Paraphrasing Hulme (2009), we disagree about climate change, because we obtain discrepant pieces of information and we interpret them differently, each from our respective disciplinary background and scientific training. We are seeing a key change in perspective – one that has been encouraged for the past decade by the climate modelling community – in that the examination of the utility of climate models has shifted from the modellers concerned with mitigation options and policies to the prospective users dealing with realistic adaptation strategies, i.e. the hydrological and water management community. It is not at all surprising that the shift from climate mitigation concerns, for which these models were designed and have the greatest utility, to the adaptation side of the ledger, has introduced considerable controversy to the debates. Actually, this should not even be considered a “controversy”, but a classical scientific debate about the efficacy of general circulation models (GCMs) for adaptation-type analyses – especially as applied to the design of hydraulic structures. It is unfair, therefore, to characterize this debate as one fuelled by climate change critics. Rather, these are pragmatic concerns, raised by hydrologists and water management practitioners, about how useful the GCMs are for the much more detailed level of analysis (and predictability) required for site-specific water management decisions (infrastructure planning, design and operations). Simply put, the current suite of climate models were not developed to provide the level of accuracy required for adaptation-type analysis. They were designed to provide a broad assessment of the response of the global climate system to greenhouse-gas (GHG) forcings, and to serve as the basis for devising a set of GHG emissions policies to slow down the rate of growth of GHGs, and, by this, to mitigate global warming impacts. To expect more from these models is simply unrealistic at this time, as they do not even perform well as weather prediction models.

Wilby (2010), in an accompanying opinion paper, recognizes this discrepancy – between what the models were designed for, and their potential extended uses, which are being vigorously promoted by the climate community, without adequate peer review and debate as to whether these extended uses are appropriate. This is essentially what the debate is now about – are these models “ready for prime time” – i.e. can they be used in real applications in the water management sector and infrastructure planning and design realm. Wilby (2010) offers a set of principles for comparing models and for undertaking what are essentially research efforts to improve the current suite of models to serve better the needs of adaptation-type activities. Indeed, a very important question would be: how do we improve these models so that they are more useful for the next level of analysis, beyond broad policy-type statements on mitigation strategies?

Climate change information is highly uncertain – in fact the unknowns about climate change dynamics go beyond our understanding of classical risk and uncertainty analysis – there are true unknowns, and no combination of clever statistical methods can reveal what those unknowns may be – they reside in the realm of more research on climate dynamics and feedback loops. We know increasingly well that we do not know enough. Yet, there is a growing, almost insatiable demand for 100 years of climate prognostications and dubious information, regardless of its utility. These demands are, understandably, driven by practical considerations, particularly related to freshwater resources, especially ecosystems, which are vulnerable to climate change. This is true also of water-related infrastructure that has to be refurbished and then serve for many decades of uncertain non-stationary climate. In the absence of crisp results produced by the science, the concepts of safety factors and precautionary allowances are being envisaged as part of “climate proofing” exercises. This has always been a major part of the water management set of tools, as historically they have dealt with risk and uncertainty in all aspects of their work (Stakhiv, 1998).

In order to project the future behaviour of the climate system, mathematical climate models have been developed to relate GHG forcing specifically to future potential climate states and, hence, to develop climate projections for the future. The Earth system is very complex and highly nonlinear. In addition to external drivers, such as solar activity, the Earth's orbit, volcanic eruptions, properties of the atmosphere and land surface, there are internal feedbacks in the system, diminishing or amplifying the effects of GHGs and generating variability. Advanced climate models mimic essential physical mechanisms and internal feedbacks. Such models have been found to reproduce broad observed features of recent and past climate at larger scales (continental and above), while estimates for some climate variables (e.g. temperature) are more accurate than for others. Unfortunately, precipitation, the principal input to freshwater systems, is not adequately simulated in present climate models (Kundzewicz et al., 2008) and models largely disagree in the projection of changes, even if there are some areas of stronger consensus, such as the Mediterranean (Giorgi & Bi, 2005).

In general, despite the considerable progress achieved, GCMs still cannot reconstruct the important details of the climate at smaller scales (regional to local). They cannot resolve sub-grid processes, e.g. related to topography and land use. Hence, other techniques, such as regional climate models (RCMs), or downscaling methods, have been developed. A typical RCM grid is of the order of 10–50 km, although some climate simulations have used smaller grids, but usually only for a shorter temporal horizon of simulations. Alternatively, statistical downscaling can be used, based on relationships linking large-scale atmospheric variables (predictors) and local/regional climate variables (predictands), cf. Wilby (2010).

However, it should be understood that RCMs operate under a set of boundary conditions set by whatever GCM is being used. Hence, if the GCM does not do an adequate job of reproducing the climate signal of a particular region, the RCM will simply mimic those inaccuracies and biases, and propagate the uncertainties even further, albeit at a regional scale. It is not clear how the coupling of a RCM to a flawed GCM can provide more refined insights, any more than can statistical downscaling.

The paper by Anagnostopoulos et al. (2010) and the opinion paper by Wilby (2010) tackle the utility and accuracy of information generated by climate models and its use in water resources applications. Anagnostopoulos et al. (2010) show that climate models, on their own, cannot accurately reconstruct the past even at sub-continental to continental scales, and perform poorly at regional scales. Comparison of model performance with station (data-based cell vs model-based cell) data is relatively poor, being somewhat better for temperature, and worse for precipitation. Actually, we have known this for quite some time and, in fact, climate modellers disseminate this message, warning the users that confidence in models decreases at smaller scales. If this is indeed true, then can the use of RCMs and statistical downscaling improve our insight as part of suggested “vulnerability analyses”? Hagemann et al. (2006) evaluated precipitation at the regional scale by comparing model simulations corresponding to various resolutions with observational data in selected catchments representing major river systems on Earth in different climate zones. They found that precipitation bias was low in the catchment of the Mississippi, but very high (in excess of 80%) at the Ganges and Brahmaputra, for all resolutions considered.

Nevertheless, many hydro-climatologists expect that there should be some degree of practically useful correspondence between the spatial average of station data and the results of climate modelling in the past to present range (hindcasting). If the correspondence is poor, improvement of these features should be one of the major research priorities. Present climate models contain considerable biases in their climatology and do not fit gridded station data well; hence, a need for bias correction comes about. Yet these “bias corrections” merely represent an ad hoc curve-fitting exercise of convenience, rather than a result of impeccable physically-based theory.

In order for climate models to be trustworthy for extending their utility to problems other than what they were designed to address, they should compare reasonably well to observations. Calibration and verification are the principal foundations of evaluating model utility. In an earlier paper, Koutsoyiannis et al. (2008) tried to estimate a point value based on a linear combination of model-simulated values for surrounding cells and then to compare a station data value of temperature or precipitation. They analysed eight stations and showed that models perform poorly even at large temporal (30-year climatic) scale. The best linear unbiased estimate (BLUE) of a point value based on a linear combination of neighbouring cells, while possibly the best (as suggested by the name), is still far from being good. Anagnostopoulos et al. (2010) report on an extension of that study to larger spatial scales. They aggregated the observation values both spatially (to a large area of the contiguous USA) and temporally (to shifted 30-year climate standard normal period), and compared with climate model simulations.

The match between model simulations and station data was not found to be satisfactory. “It cannot be satisfactory because of the mismatch between points and gridded data,” say climatologists. Anagnostopoulos et al. (2010) quantify, through rigorous analytical methods, the discrepancy between observations and model outputs. Even though the purpose of climate models is to mimic gross climate changes corresponding to changing emissions scenarios (emissions, land use, etc.), they should still be expected to reproduce the recent past 30 years of climate accurately. “Then we are disappointed,” say many hydrologists, who may have unrealistically high expectations, and who apply these models to problems for which the GCMs are unsuited – as for example, deriving frequency distributions from downscaled precipitation results at a river basin or watershed scale.

Water managers understand that climate models are improving, but are not good enough yet for the types of analyses that they routinely undertake. There are still processes missing, or poorly represented. More comprehensive data on the atmosphere, oceans, land surface and soil moisture, vegetation state, and sea ice are necessary. Some data (e.g. on aerosols) are missing; others are inadequate to calibrate and validate the models. Complex nonlinear feedbacks are difficult to grasp; others are simply unknown.

The original paper submitted by Anagnostopoulos et al. to Hydrological Sciences Journal (HSJ) generated polarized recommendations by the referees. The ratings by the reviewers spanned the broadest possible range, from “very good to excellent” to “poor to fair; reject outright”. This clearly illustrates that the community is largely divided. Apparently, one cannot be neutral to such a paper – one either strongly agrees or strongly disagrees. The editor has to take responsibility for a decision on acceptance or rejection, being aware of a real and serious possibility of making substantial errors, such as (in analogy to hypothesis testing) editorial errors of the first kind (publishing papers that do not deserve publication) and of the second kind (rejecting papers that deserve publication), cf. Kundzewicz (2002).

Criticism of models belongs to a standard menu of the so-called climate sceptics. If, on the one hand, models are the principal instrument in climate change attribution and projections for the future, then emphasizing weaknesses of models at the hydrology-relevant verification stage can undermine model-based conclusions. On the other hand, peer-reviewers and referees need to review the paper objectively for its scientific methodology and congruence of inferences with the analytical outcomes, and not introduce their own biases as to how the results of the paper may be used or interpreted by others. Is the methodology appropriate and are the conclusions related to the analyses? An editor's obligation is to publish papers that advance the state of science and of understanding that science. Hydrologists and water management professionals (hydrological and hydraulic engineers) have entered the scientific debate in force, because the GCMs are being advocated for purposes they were not designed for, i.e. watershed vulnerability assessments and infrastructure design. They are now examining whether these models are suitable, using their own perfectly legitimate and peer-reviewed methods, as well as statistical tools developed over the course of a century of practical applications. They are not climate sceptics, but are sceptical of the claims of some climatologists and hydroclimatologists that these models are well-suited for water management applications.

Notwithstanding these intellectual disagreements, to put the matters in perspective, an opinion paper was also solicited from Robert L. Wilby, who recommended rejection of the original paper by Anagnostopoulos et al. (2010). In his opinion paper, Wilby (2010) reviews the principles of selection of climate models for hydrological applications. It is anticipated that publishing the paper by Anagnostopoulos et al. (2010) and the opinion paper by Wilby (2010) next to each other in HSJ can make important contributions to the scientific debate. However, we must remember to differentiate opinion from a carefully applied scientific method. The degree to which GCMs may be applied to various research questions, which then provide the foundations of peer-reviewed “best practices” and applications by water managers, is a qualitative, opinion-based issue. The methodological basis of the Anagnostoupolos et al. (2010) paper is purely a scientific issue. There is no doubt that a plurality of opinions and criticality are essential for scientific progress. This allows different stances and discrepant views to be explicitly, and critically, confronted. Such a dialogue is expected to have a clarification value. The two papers are likely to be of interest to a considerable part of HSJ readership, which may subscribe to either of the two attitudes, or search for something in between. Can the controversies be reconciled?

The paper by Anagnostopoulos et al. (2010) contains a warning that one should not use GCMs simulations uncritically, taking them at face value, as advocated by many climatologists and accepted by some hydrologists. Likewise, we should not apply watershed models uncritically as these will compound the uncertainty still further (Wilby, 2005). Such a negative result is useful, cautioning hydrologists against uncritically following this path; for without bias correction, model simulations cannot mimic reality.

Our response to the question posed in the title of this editorial is that, while they are getting better, climate models are not (up to) ready for “prime time” yet, at least for direct application to water management problems. Much more research needs to be done, and models need to be improved considerably before they can be used effectively for adaptation planning and design: “Significant improvements of GCMs to the point where their output can be input directly into water utilities planning models without bias correction and downscaling will likely take more than a decade or two” (Water Utility Climate Alliance, 2009). We estimate that these models may begin to provide useful information, at least for vulnerability assessments, at some point in the future, but unlikely earlier than after a decade. As phrased by Stakhiv (2010), the information currently available from GCMs is inadequate for most planning and design aspects of water decisions, and certainly not useful for operational decisions related to reservoir regulation rules and short-term forecasting. In fact, water managers ask why there is a proliferation of 23 GCMs that they have to contend with, as part of their analyses, each generating countless scenarios. Why should the burden be on the user community to reconcile the disparate outcomes of 23 GCMs? The spread of outcomes in these models is often used, incorrectly, as a form of uncertainty analysis, and the average of the ensemble of model projections is often advocated as a useful representation of future climate. What is the utility of having so many models, generating hundreds of scenarios (e.g. 565 in the study by Angel & Kunkel, 2009)? How does one interpret these widely disparate results, even if these models had some useful predictive capabilities, possibly for temperature increase? The water science community has developed its own suite of hydrological models over the course of a century of research and applications. These have undergone decades of peer-review, testing and application in a wide range of watersheds all over the globe, for designing irrigation systems, levees, flood control dams and hydropower systems. The water management community has converged on a set of “best management practices” through a combination of research, peer-review and field testing before the methods and models have been adopted. The GCM projections are not directly applicable to solve important practical questions. Clearly, further work is needed in the area of GCM testing and refinement, so that climate model results can be applied, in a more persuasive way, to real problems.

Trenberth (2010) soberly assesses the transient deficiency related to model improvement: “Adding complexity to a modelled system when the real system is complex is no doubt essential for model development. It may, however, run the risk of turning a useful model into a research tool that is not yet skilful at making predictions.” Koutsoyiannis et al. (2009) expressed the opinion that the mechanisms driving the changes are poorly understood and possibly beyond our ability to model adequately.

However, reliance on the stochastics alone, as proposed by Anagnostopoulos et al. (2010), would be tantamount to incomplete use of available information. Beside the stochastics driving the climatic variability, there are clear physical mechanisms that have to be taken into account, cf. Kundzewicz et al. (2009).

Needless to state, we warmly invite discussion of either of the following two papers.