Context and capacity: The potential for performance-based agricultural water quality policy

Abstract

Current Canadian policy approaches to agricultural water quality encourage the adoption of best management practices through voluntary, incentive-based measures. Despite these measures, concerns about agricultural impacts on water quality persist. Performance-based policy approaches with incentives that are tied to defined outcomes, and not to particular practices, may have an important role in managing water quality. Five performance-based approaches to address water quality in agricultural landscapes were identified: water quality trading/permitting, differentiated payments for ecosystem services, reverse auctions; emissions charges, and cross-compliance (a hybrid measure). The purpose of this paper is to critically assess the institutional and socio-cultural context that facilitated existing performance-based policy instrument adoption. Through this analysis, three key contextual factors were identified as enablers of performance based approaches: (1) social context, (2) institutional capacity and (3) standardized, consistent and robust estimation methodologies. A framework was developed to classify performance based programs and approaches. The application of the findings from this research and the classification framework provide an organized approach to assess the feasibility of implementing performance-based approaches for agri-environmental water quality policy.

Les approches actuelles de la politique canadienne à la qualité de l’eau encouragent l’adoption des meilleures pratiques de gestion à travers les mesures volontaires et incitatives. Malgré ces mesures, des préoccupations à propos les impacts agricoles sur la qualité de l’eau persistent. Les approches politiques basées sur la performance qui sont liées aux résultats définis et non pas aux pratiques particulières puissent avoir un rôle important dans la gestion de la qualité de l’eau. Nous avons identifié cinq approches basées sur la performance pour aborder la qualité de l’eau dans les paysages agricoles : l’échange de crédits de qualité de l’eau/permettre la qualité de l’eau, les paiements différenciés pour les services d’écosystème, les enchères inversées, les frais d’émissions et l’écoconformité (une mesure hybride). Cet article vise à évaluer de façon critique le contexte institutionnel et socio-culturel qui facilitait l’adoption de l’instrument politique actuel basé sur la performance. À travers cette analyse, on a identifié trois facteurs contextuels clés comme facilitateurs des approches basées sur la performance : (1) le contexte social, (2) la capacité institutionnelle et (3) les méthodes d’estimation standardisées, logiques et robustes. Nous avons développé un outil pour classer les programmes et les approches qui sont basés sur la performance. L’application des résultats de cette recherche et l’outil de classement fournissent une approche organisée pour évaluer la faisabilité de mettre à effet les approches basées sur la performance pour la politique agroenvironnementale de la qualité de l’eau.

Introduction

There is growing public concern over the effects of agricultural activities on water quality (Coote and Gregorich 2000; Organisation for Economic Co-operation and Development [OECD] 2003). In Canada, several agricultural water contamination events and ongoing boil water advisories in some areas have exacerbated these concerns (Schuster et al. 2005; Salki 2007; Environment Canada 2010a). An often-cited example is the contamination of drinking water in Walkerton, Ontario, by bacteria from livestock manure. Seven deaths and approximately 2300 illnesses occurred due to this contamination (O’Connor 2002). Other water quality issues have served to raise the concern of national and provincial governments. For example, Agriculture and Agri-Food Canada reported that water quality in Canada decreased between 1981 and 2006, primarily as a result of increasing applications of nitrogen (N) and phosphorus (P) as fertilizers and manure (Eilers et al. 2010). Environment Canada (2010b) reported 20% of tested sites had water quality rated as “marginal” or “poor”, defined as guidelines exceeded often.

Water quality policy in Canada has focused primarily on improving management practices through voluntary measures (Weersink et al. 1998). An alternative performance-based policy approach that focuses incentives on defined outcomes rather than on inputs could ensure environmental improvements from agri-environmental programs and play an important role in managing water quality (e.g. Weinberg and Claassen 2006). However, the factors that enable or limit the ability of performance-based approaches to meet environmental objectives are poorly understood. This paper provides a critical review of existing performance-based approach examples from a range of jurisdictions and applications to: (1) identify alternative, performance-based policy approaches that have been implemented to manage water quality, (2) assess the institutional and socio-cultural context that resulted in their adoption and (3) develop a framework to categorize performance-based policy measures based on their perceived effectiveness where information is available.

Performance-based approaches to agri-environmental water quality policy

Voluntary incentive or cost-sharing programs to encourage adoption of beneficial management practices (BMPs) are the main instrument used by agencies to manage agricultural activities that affect water quality (Weersink et al. 1998). However, the link between prescribed BMPs and environmental outcomes is indirect and influenced by location, topography, weather and specific farm and farmer traits. In contrast, performance-based approaches to agri-environmental policy are designed to focus on environmental outcomes, with payments or charges tied to these outcomes. In performance-based initiatives, the environmental outcome must be well defined, and program responses are conditional on outcome delivery (Wunder et al. 2008). For example, payments can be made for each unit reduction in a specific water pollutant. The environmental outcome of a particular practice can be predicted with models (Lowell et al. 2007), indicators (Weinberg and Claassen 2006) or remote sensing (Cohen and Goward 2004), while actual performance can be measured through direct inspections (Hanley et al. 2004). Often, more than one mechanism is used to estimate performance, such as models supplemented with local data for more accurate estimates of the effect of a management change (Lowell et al. 2007).

Performance-based approaches have been shown to provide a cost-effective mechanism to provide desired environmental outcomes because landowners can meet the environmental target by adopting the least-cost strategy for the socio-economic and biophysical conditions of their farm (Wätzold and Dreschler 2005). Benefits of this approach include the efficient allocation of public funds toward practices that have a proven or predicted level of environment performance (Claassen et al. 2001). Performance-based payments can also be targeted to specific locations (Weinberg and Claassen 2006), which may be a more cost-effective approach than uniform payments (Wätzold and Dreschler 2005). However, high administration costs associated with the delivery mechanisms and environmental monitoring required for the success of performance-based approaches is commonly cited as a limiting factor (Hodge 2000; Weinberg and Claassen 2006). In addition, environmental gains from performance-based programs can be unpredictable (Jack et al. 2008) due to climate and ecological variability that is outside the control of the landowner (Hodge 2000). Physical process models used to estimate the environmental effect of a management change can be imprecise, resulting in inaccurate estimates of improvement and reduced cost-effectiveness (Keeney and Boody 2005). This can increase the economic risk to the landowner when payments are contingent on specific outcomes (Engel et al. 2008), causing landowners to be hesitant to commit to a program or to demand higher payment to offset this uncertainty and risk (Hodge 2000).

A significant body of research has shown that performance-based policy measures may facilitate meeting environmental objectives, specifically water quality, more effectively than more traditional voluntary BMP-based approaches. As a result, there is a strong incentive for the transfer of successful performance-based instruments to other locations. However, the design and effectiveness of these policy instruments is determined by a number of contextual conditions that characterize the target region. Policy transfers are often implemented in an unsystematic way, relying on anecdotal evidence and paying inadequate heed to context (Wolman 1992; Pigram 2001; Dwyer and Ellison 2009). It has been argued that the successful transfer of a policy approach from one location to another depends on the broader contextual factors that drive and enable it (Dolowitz and Marsh 2000; Mossberger and Wolman 2003; Millennium Ecosystem Assessment 2005). In the following discussion, this research gap is addressed by explicitly identifying and considering the drivers and enabling conditions that together define the context for the implementation of performance-based approaches. The insights gained from this literature review will be used to inform the classification of existing performance-based policy approaches. The classification process developed in this paper is designed to assist practitioners in evaluating potential approaches that might best fit their particular needs and contexts.

Drivers of policy measures can be based on demands for policy resulting from environmental, social and/or economic conditions, or institutional mandates for environmental improvement such as legislation and/or policy. In contrast, enabling conditions that allow the consideration of performance-based approaches include those institutional and/or legal characteristics such as the well-defined property rights of agricultural landowners, and social norms including recognition by policy makers of water quality as a valued ecosystem good. It is important to note that well-defined property rights make it evident whether the policy requires governments to employ a “beneficiary pays” principle or a “polluter pays” principle for agricultural landowners (OECD 2010b). For the purpose of analysis, Dolowitz and Marsh (2000) separated drivers and enabling conditions of a performance-based approach into economic, environmental, institutional and social factors. Where performance-based approaches have been implemented, an assessment of these factors across cases allows some generalizations to be made about those factors that facilitate implementation, and an identification of common factors in successful and/or unsuccessful approaches.

Much of the literature related to the policy transfer process discusses social and institutional context (e.g. Rose 1993; Dolowitz and Marsh 1996; Mossberger and Wolman 2003; de Jong 2009), and it is clear that social and institutional drivers and enabling conditions can be particularly important in assessing fit in water policy transfer (Dolowitz and Marsh 1996). In this case, fit is defined as the potential for implementation, considering the context in the region where the policy originates and the region considering transfer. Previous contextual analyses for potential water policy transfer have also identified social and institutional capacity or mismatch as significant barriers to a transfer (e.g. Hu 1999; Malano et al. 1999; Swainson and de Loë 2010). Where policy transfers have been evaluated post-implementation, social and institutional mismatches have often been identified as a significant cause of substandard outcomes or outright policy failure (e.g. Wolman 1992; Barnes et al. 2009).

The remainder of the paper is organized into three sections. First, a survey of performance-based approaches addressing water quality on agricultural landscapes is presented. This is accompanied by a discussion focused on the relevant institutional and social drivers and enabling conditions for each approach. The next section describes an organizational framework for performance-based approaches based on the policy survey. This framework provides a tool to categorize approaches according to their capacity to incorporate performance, the measurement/estimation capacity and the desired payment structure for regions where performance-based approaches are being considered. The final section discusses applications of the framework to inform the classification, selection and potential transfer of water quality-focused performance-based policy.

Results

Survey of performance-based approaches

Performance-based approaches were defined, for this discussion, as those approaches where there was some differentiation in payment or penalty level based on water quality outcomes, and some effort to measure or estimate those water quality outcomes. To enable an explicit comparison of the performance-based approach cases, five summary tables were developed around the four attributes described in this section (Tables 1 to 5). The “Characteristics” column provides a brief description of each performance-based approach including the method used to measure water quality outcomes and the structure of the payment or penalty scheme. Next, the identified enabling conditions and drivers for the programs, as evaluated in the literature, are summarized. Finally, where available, the outcomes are presented. Where there was more than one instance of implementation of a particular approach, the cases where the most information was available in the literature were chosen.

Table 1. Water quality trading approaches to agricultural water quality management.

	Case
	South Nation River Watershed phosphorus (P) trading program, Ontario	United States water quality trading (WQT) programs (several states)
Defining characteristic: Measurement	Modeled based on measured performance of practices in other regions, calculated per hectare or per head for livestock	Either site-specific calculations to estimate nutrient losses or reductions using variables such as soil type, slope and other input rates, or pre-determined nutrient reductions for practices that assign a nutrient reduction credit based on estimated average nutrient reductions
Defining characteristic: Payment	The South Nation Conservation group retained the funds paid for P credits and used those funds for grants to agricultural producers to implement best management practices (BMPs). Grant amounts were based on projects and not amount of P reduction	Credits based on an emission unit purchased through three mechanisms: (1) bilateral negotiations, (2) clearinghouses or (3) exchange markets
Enabling conditions: Institutional	The Ontario Ministry of Environment created water quality guidelines with maximum allowable concentrations of pollutants Regulatory measures for drinking water implemented under the Ontario Safe Water Drinking Act South Nation Conservation community group established to manage natural resources in the watershed	Previous success of sulphur dioxide trading under the US Clean Air Act US Clean Water Act gave individual states authority to regulate non-point source pollution US Environmental Protection Agency's Watershed Protection Approach created capacity to use unique approaches to achieve ecological goals
Enabling conditions: Social	Agricultural producers were well organized and effective in lobbying for policies that affirm strong property rights
Drivers: Institutional	P concentration limit enforced in watershed Program viewed as a market-based approach to mitigate impacts of wastewater emissions Removal of legal liability for agricultural producers through Statement of Roles and Responsibilities	US Environmental Protection Agency and United States Department of Agriculture have promoted water quality trading through funding initiatives and policy provisions Several state-specific regulations to encourage trading Implementation, or imminent implementation, of nutrient caps for water bodies based on total maximum daily load (TMDL) for pollutants
Drivers: Social	Stakeholder buy-in achieved through the use of a locally trusted organization, created a sense of security and trust Use of a trusted organization, the South Nation Conservation group, as a broker for credits
Outcomes	Water quality improved, estimated reduction in emissions of approximately 10,000 kg per year More cost effective than improving wastewater treatment Watershed residents perceived other benefits, including soil and livestock health benefits, reduced risk to human health, improved opinions of the South Nation Conservation group, increased property values	Economic benefits, lower cost to reduce pollutant loads Where trades have not occurred, attributed to lack of institutional drivers
References	OMOE (1994); O'Connor (2002); Allaway (2003); OFEC (2006); Boutz (2007); Selman et al. (2009); Driedger (2010); O'Grady (2011)	McGinnis (2001); Woodward et al. (2002); Woodward (2003); Fang et al. (2005); Abdalla et al. (2007); Selman et al. (2009)

To maintain a degree of consistency between the Canadian context and the contexts of other regions investigated (Mossberger and Wolman 2003), as well as to maintain a focus on agri-environmental policy and water quality in the findings from the literature review, the emphasis was on approaches that: (1) were designed specifically for application to agriculture, (2) addressed water quality, or included water quality management as a component, and (3) were deployed in member countries of the Organisation for Economic Co-operation and Development (OECD). OECD countries were chosen as comparable to Canada because: (1) agriculture in these countries is primarily practiced on private land, (2) all have a system of democratic governance and (3) all are developed countries with market economies (OECD 2010a). Five categories of approaches were ultimately identified through the literature survey, and are presented below.

Water quality trading/permits

Water quality permit trading allows large emitters of water pollution to buy emission credits from market participants who can decrease emissions for a lower cost than the value of the permit. This market-based approach establishes property rights for pollutant discharge and may be implemented where cap and trade regulations have been put in place, or where voluntary demand is great enough to warrant the policy instrument (e.g. where purchasing water quality credits is less costly than technological or operational improvements for a point source polluter) (Lal et al. 2009). There are several necessary conditions for water quality permit trading to be effective: (1) a specific environmental objective, (2) clearly defined property rights for discharging pollutants, (3) an appropriate level of incentives to ensure trades, (4) appropriate and clear trading rules, and (5) measurement and monitoring (Weersink et al. 1998; Cantin et al. 2005).

Water quality trading uses a model or set of models to estimate water quality improvements and/or transfer of reported performance from other studies to estimate the effect of specific emission-reducing practices. In the case of water quality improvement, payments are made based on the magnitude of reduction in pollutants estimated from those models. A number of cases were reviewed from the United States, and one in Canada (Table 1). These examples are generally based in small watersheds where there is a point-source polluter (often industrial) that acts as a buyer of water quality permits from several non-point-source polluters that can abate emissions at relatively low cost (often agricultural producers).

Differentiated payments for ecosystem goods and services

Performance-based payments for ecosystem goods and services encompass a range of payment types that rely on site-specific models with payments structured to reflect the degree of improvement in pollutant concentrations in water (Claassen et al. 2001). While there are a number of purported “performance-based” payments for ecosystem services (PES) programs being used in the US and abroad (Wunder et al. 2008), most payment structures are strictly tied to inputs or practices. In this paper, performance-based PES are approaches where payments are differentiated, at least to some extent, based on estimated environmental outcomes facilitated by the adoption of specific practices and not focused on offsetting pollutant loading by other sources, separating them from water quality trading. Payments can be based on estimated units of pollution reduction, or can be tiered to pay set amounts based on falling within a range of estimated outcomes. However, many of the payments for ecosystem services programs implemented in OECD countries do not have this feature, and therefore are not within the scope of this survey. Programs that met these requirements are summarized in Table 2.

Table 2. Differentiated payments for ecosystem goods and services approaches to agricultural water quality management.

	Case
	Performance-based Environmental Policies for Agriculture (PEPA), Hewitt Creek Watershed, Iowa, United States	Privately-sponsored payments for Ecosystem Services (PES), Vittel Company, France	Group payments for Ecosystem Services (PES), Cullers Run Watershed, West Virginia, United States
Defining characteristic: Measurement	Suite of indices to estimate nutrient and sediment losses calculated for individual fields	Estimated the total reduction of nitrogen emissions required to maintain acceptable water quality standard, based on site-specific research. Management practices identified that could achieve nitrogen reduction requirement	Measurement of water quality was direct, background levels of nitrogen were accounted for by measuring water quality in an additional "index watershed" that was similar in size and location but not impacted by agricultural activities
Defining characteristic: Payment	Variety of incentive payments; some are based on nutrient testing or fixed payments for specific practices, while others are incremental and increase based on degree of reduction in pollutants as estimated by indices	Contracts negotiated based on actions required from each farmer	Payments to whole group, distributed equally and based on water quality, volume
Enabling conditions: Institutional	Listed as impaired watershed under the US Clean Water Act		Impaired watershed listed under the US Clean Water Act
Enabling conditions: Social
Drivers: Institutional	Impending regulation as a result of impaired watershed listing Financial support of local farm bureaus	Company required actions at the farm level to maintain viability Political will to retain company in the region
Drivers: Social	Fear of government intervention Civic responsibility and peer pressure	Company employed many people in the region, providing a strong incentive for farmers to work with the company; stakeholder buy-in	Researcher interest in assessing feasibility of using a group-based PES approach to managing water quality
Outcomes	Social successes were considered important. Stakeholders formed cohesive structure, shared knowledge and ideas Water quality improved, estimated reduction of nitrate pollution by 23%; phosphorus pollution reduced by 14%	Water quality maintained at an acceptable level for company to continue to operate, no jobs lost Farmers negotiated contracts that have been satisfactory, no ill will	Social successes included increased interest in acquiring information about water quality, increased participation and pressure to participate from other farmers. Trust among researchers and farmers developed throughout the project
References	Morton et al. (2006); Hewitt Creek Watershed Improvement Association Inc. (2008); Morton (2008); Winsten (2009); Winsten et al. (2011)	Perrot-Maȋtre (2006)	Maille et al. (2009)

Reverse auctions

Reverse auctions are characterized by multiple sellers of ecosystem goods and services that participate in a competitive bidding system and a single buyer, usually a government agency. Three common approaches used to select successful bids from sellers in a reverse auction are: (1) based on cost where bids are ranked and winning bids are funded starting with the lowest bid; (2) based on benefits where bids are ranked according to environmental outcomes and funded starting from the bid providing the greatest benefit; and (3) based on cost effectiveness where ranking occurs with both cost and benefit considerations (Selman et al. 2008). The latter two types of auctions can be considered performance-based as some measure of benefits is used, and payments are tied to the degree of environmental outcomes in the sense that the most cost effective, or the most beneficial, plans receive payment. While the bid may not directly reflect the value of the outcome, it is sufficient payment for the landowner to achieve the outcome, and therefore may actually be more cost effective than differentiated payments. The reverse auction approach enables agricultural producers to use their knowledge of the costs of implementing specific practices to set a bid price that makes the activity financially attractive (Latacz-Lohmann and Van der Hamsvoort 1997). However, in the presence of competition among sellers with similar objectives, the participants must also act strategically to decrease the probability of a rejected bid if it exceeds the perceived value of expected environmental outcomes. This mechanism alleviates the problem of asymmetric information for funding agencies, as it reveals the landowner’s opportunity cost of providing ecosystem goods and services (Ferraro 2008).

Reverse auctions have been implemented in the US as a part of programs to manage highly erodible land and to reduce P runoff (Reichelderfer and Boggess 1988; Selman et al. 2008), and in Australia to manage water quality along with other land management issues (Eigenraam et al. 2007; Lowell et al. 2007) (Table 3).

Table 3. Reverse auction approaches to agricultural water quality management.

Characteristics	Case
	Conservation Reserve Program, United States	Conestoga Watershed, Pennsylvania, United States	Great Barrier Reef, Australia
Defining characteristic: Measurement	Use of an Environmental Benefits Index (EBI) to estimate the site-specific environmental outcomes of retiring land, and may be used to weight them according to the priorities of the program	The expected reduction in phosphorus runoff estimated using a combination of site-specific variables and research estimates of effectiveness of various best management practices (BMPs)	Used models with site-specific data to estimate the amount of agricultural emissions mitigated by each bid proposal, resulting in an environmental benefit index score
Defining characteristic: Payment	Bidding by landowners for land retirement payments. Bids selected based on EBI score	Used a cost-effectiveness strategy to select bids for specific and appropriate BMPs	From the EBI and the proposed bid amount, cost-effectiveness was determined and bids were selected on this basis
Enabling conditions: Institutional	History of land retirement programs during periods of low commodity prices; significant area of land designated as highly erodible	Watershed listed as impaired under the US Clean Water Act	Previously successful reverse auctions for conservation
Enabling conditions: Social	Desire to support agricultural landowners
Drivers: Institutional	Enactment of the Food Security Act with a legislated land base target United States Congress authorized use of cost-benefit targeting to select bids		Government concern about misallocation of resources; trial auctions were perceived as a method to better estimate opportunity costs for agricultural producers to make changes to practices and reduce potential of over-payment
Drivers: Social		Researcher interest in demonstrating the potential cost efficiencies using a performance-based approach as compared to a conventional input-based approach
Outcomes	Estimated increase in environmental benefits but no measurement Criticisms of program for lack of objectivity, inflexibility to respond to changing environmental conditions	The difference in phosphorus pollution reduction and costs were substantial: the conventional program required $26.19 lb^–1, while the reverse auction paid out an average of $5.06 lb^–1	These trials highlighted the variability in producers' costs and the importance of using a cost-effectiveness strategy in assessing bids
References	Reichelderfer and Boggess (1988); Claassen et al. (2008)	Selman et al. (2008)	Rolfe and Windle (2011)

Emission charges

Emissions charges represent increasing and incremental levies charged to agricultural producers who create more pollution than they are permitted at the individual operation level. The only OECD country that has instituted a performance-based emission charge to manage water quality is The Netherlands (Table 4). In this case, mandatory nutrient accounting, with a specific level for N and P inputs and outputs, was implemented with no payments being provided for maintaining pollutant emissions below the limit, but levies were administered where emissions were exceeded (Peerlings and Polman 2008).

Table 4. Emissions charges approach to agricultural water quality management.

Characteristics	Case: MINAS^* program, The Netherlands
Defining characteristic: Measurement	Individual farm accounting of inputs and outputs related to phosphorus
Defining characteristic: Payment	Nutrient surpluses subject to incremental levy
Enabling conditions: Institutional
Enabling conditions: Social	Intensive agriculture industry including livestock production, resulting in excess manure which contributed pollutants to water bodies
Drivers: Institutional	Failed attempts to manage agricultural sources of water contamination through input-based regulations
Drivers: Social Outcomes	Some water pollutants reduced (nitrates), but phosphorus remained a concern and actually increased. Program considered unsuccessful Eventually discontinued as a result of incompatibility with European Union directives
Reference	van Grinsven et al. (2005)

* MINeral Accounting System (MINAS).

Cross-compliance

Cross-compliance examples impose a mandatory minimum standard for environmental performance on all agricultural producers, with required adoption of activities to achieve the environmental standard to remain eligible for government support programs, loan programs and voluntary agri-environmental programs (Claassen et al. 2004; Mann 2005; OECD 2010a). Where agricultural producers choose not to achieve the minimum standard, they are excluded from most, if not all, forms of government assistance (Claassen et al. 2004). The primary example of cross-compliance application is linked to N emission objectives in nitrate-vulnerable zones in the United Kingdom (UK) (Table 5). While the objective of cross-compliance programs is an environmental outcome, compliance is determined based on the adoption of practices (Baylis et al. 2008). As such, cross-compliance may be considered a “hybrid” policy instrument, rather than strictly performance- or practice-based (Weinberg and Claassen 2006).

Table 5. Cross-compliance approach to agricultural water quality management.

	Case: Nitrate Vulnerable Zones (NVZs), United Kingdom
Defining characteristic: Measurement	No measurement of individual farm contribution to water quality. However, beyond the initial assessment of a watershed as an NVZ, reviews must be undertaken every four years to assess water quality and revise the NVZ based on direct water quality monitoring
Defining characteristic: Payment	Where producers within these zones comply with required practices, income support payments are provided in full. Otherwise, reductions to, or denial of, income support payments and agri-environmental payments occurs
Enabling conditions: Institutional	Previous water quality directives and legislation Previous zonal designations for water protection (Nitrate Sensitive Areas) and implementation of the Common Agricultural Policy reforms of 1992 created institutional capacity
Enabling conditions: Social	Growing social belief that agricultural producers should not receive payments unless they comply with basic environmental requirements
Drivers: Institutional	World Trade Organization pressure to reduce income support payments and fit payments within “green box” requirements (decoupled payments from production)
Drivers: Social Outcomes	Mixed environmental outcomes: some NVZs (31%) showed improvement in surface water quality; some remained the same; and some (17%) became more polluted over a 12-year period Focus on inputs was not an effective strategy to reduce nitrate pollution Social outcomes showed a mismatch between farmers’ attitudes and public expectation of environmental benefits. Farmers were distrustful and felt penalized. However, considered an effective way to induce a change in practices and improved environmental outcomes in the face of a lack of interest by landowners
References	Osborn and Cook (1997); Hall et al. (2004); Amin-Hanjani and Todd (2005); Macgregor and Warren (2006); Baylis et al. (2008); WTO (2008); Barnes et al. (2009); Nielsen et al. (2009); Worrall et al. (2009); European Commission (2010); McVittie et al. (2010)

General lessons drawn from the survey of performance-based approaches

The survey of performance-based approaches to agri-environmental policy revealed that, while not well developed for the management of water quality, there are examples of performance-based approaches within OECD countries. Many of the applications represent pilot projects or localized, watershed-based initiatives (particularly those that are more strongly performance based). Despite this, there are contextual similarities among many of the described cases, and some general lessons for successful transfer and implementation of performance-based approaches can be taken.

The first lesson is that social context matters. There are two facets of social context that can play an important part in the capacity to implement performance-based approaches: (1) the regional social context plays an important role in the successful implementation of a performance-based program. Specifically, public acknowledgement of the importance of environmental outcomes from agriculture created the appropriate social enabling conditions to implement a performance-based approach. Understanding the prevailing social context regarding agriculture and the environment will enable an assessment of the capacity of a region to implement a performance-based approach; (2) understanding social values and norms provides important information to program administrators enabling programs and related policy approaches to be tailored to the specific conditions of the region, leading to greater potential for success (Perrot-Maȋtre 2006). Where performance-based approaches have been used to manage water quality successfully, social buy-in has been essential (Selman et al. 2009). Researchers involved in pilot projects identified trust as the key factor in gaining local stakeholder buy-in for the program (Perrot-Maȋtre 2006; O’Grady 2011). For example, O’Grady (2011) described a point in the process of developing a watershed-based water quality trading program in the South Nation Watershed in Ontario, where progress stalled as a result of lack of trust and an inability to agree on a risk management strategy. He reports that the program would not have been successfully implemented without developing social buy-in from the community by tailoring an approach that was consistent with the social norms and values of the stakeholders. Selman et al. (2009) report that several water quality trading programs in the US have stalled short of implementation due to lack of support from key stakeholders. Lack of buy-in by agricultural producers in Scotland was identified as a limiting factor in the success of cross-compliance measures to reduce nitrate pollution, and stakeholder reluctance to participate hindered all cross-compliance efforts reviewed here. The main problem was a disparity of social values between agricultural producers, who view agriculture as an activity to produce food and fibre, and the public, with their multi-functional expectations of agriculture (Macgregor and Warren 2006; McVittie et al. 2010). The effort to understand the social values of stakeholders and the development of trust have been acknowledged as factors important to the success of watershed-based management (Conley and Moote 2003; Perrot-Maȋtre 2006; Global Water Partnership 2009; Mandarano 2009).

The second lesson is that institutional capacity appears to be a common constraint on the implementation of performance-based approaches. For example, researchers have called for specific language in legislation that creates capacity for these approaches (Selman et al. 2008; O’Grady 2011). The total maximum daily load (TMDL) standards in the US, combined with specific mandates for cost-effective agri-environmental programming, have contributed to the implementation of a major reverse auction program along with over 50 smaller water quality trading programs and several examples of PES programs (Selman et al. 2009; Claassen et al. 2001). Implementation, or forthcoming implementation, of water quality standards has been an important factor in many of the US-based programs and pilot projects, as well as examples in the UK and the South Nation Watershed in Ontario (Selman et al. 2009). The presence of standards is required to implement some of the more strongly performance-based approaches, and the introduction of TMDLs and local or regional pollution caps has driven interest in water quality trading, payments for ecosystem services, and cross-compliance approaches. Even where water quality standards have not been implemented or enforced, the fear of regulation on the basis of water quality standards has created social conditions conducive to the use of performance-based approaches. In some cases, agricultural producers recognized a potential for water quality regulation within their watershed which spurred interest in alternative mechanisms to manage water quality (Winsten 2009).

The third lesson is that standardized, consistent and robust estimation methodologies are necessary for the successful implementation of performance-based approaches (Guiling and St. John 2007; Selman et al. 2008, 2009). Guiling and St. John (2007) call for improved site-specific research to improve estimates of environmental outcomes, a framework to monitor and ground-truth models to test accuracy, and a repository of estimation models and monitoring data for the US. Current estimation tools are often relatively simple and unlikely to be accurate (Winsten 2009), while increased accuracy and site specificity would result in an increase in cost effectiveness of payments for ecosystem goods and services (Ribaudo et al. 2001). These needs likely extend to other regions as well, especially those with less experience in implementing performance-based approaches to agri-environmental policy, such as Canada.

The performance-based measures organizational framework

The Performance-based Measures Framework (PMF) is developed in this study to provide an organizational scheme that can be used to compare performance-based approaches. The PMF uses methods of measurement of water quality and the structure of payments or penalties to categorize approaches on a scale from strongly performance-based to weakly performance-based. Each approach, as well as individual programs, can be categorized based on performance capacity (Figure 1). For example, the strongest approach in terms of performance incorporates site-based direct measurements of performance and variable payments, while the weakest approach uses very weak measures of performance with payments based on inputs and no varying the degree of incentive or disincentive based on the degree of outcome. The labeled arrows in Figure 1 represent benchmarks reflecting changes in measurement method and payment structure along the ranking system. The measurement method and payment structure of the performance-based approaches are not necessarily linked, though they may occupy a similar category.

Figure 1. The performance-based measures framework (PMF) for organization of performance-based approaches to agricultural water quality management.

The mechanisms to categorize policy approaches using the PMF can be demonstrated. For example, a strongly performance-based approach is one that uses direct water quality measurements to determine the magnitude of the potential payment or levy. A key strength of a strongly performance-based approach is the efficiency and effectiveness of payments/penalties; direct measurement of environmental quality conducted and payments are made only for measured improvements, enabling a high degree of accuracy. However, there are several significant weaknesses to strongly performance-based approaches, including high administrative costs and, potentially, significant risk assumed by the producer due to, for example, variable and unpredictable environmental performance due to weather events, and other management choices made by landowners that may preclude the environmental improvement. Perceived unfair distribution of risk has been shown, in the programs reviewed, to lead to stakeholder refusal to participate.

A moderately performance-based approach estimates performance rather than measuring performance directly. As a result, accuracy and cost effectiveness decrease for the program provider but there is a potential for increased landowner adoption of the program components due to decreased private risk, since the landowner is less likely to receive no payment based on inadequate water quality improvements. Water quality estimates that include site-specific conditions (e.g. slope, distance to surface water, crop grown, soil nutrient levels) are higher on the PMF than regional estimates, where only regional conditions are considered. Payment types can be variable and based on the expected outcome from models, or else fixed and based on model outcomes (e.g. reaching a set standard of water quality to be eligible for government funding). These types of approaches afford landowners greater security in receiving funding if actions are carried out, regardless of the variability of environmental outcomes due to weather and other conditions beyond their control.

Where approaches are only very weakly performance-based, outcomes are assessed based on research results representing average responses to a specific management practice. The programs that incorporate this type of assessment are prescriptive and allow no flexibility for the landowner to adopt practices appropriate to his or her particular biophysical and socio-economic situation. While a weak attempt is made to assess whether there will be a positive environmental outcome, in general, outcomes are assumed to be linked to inputs or management practices. Payment structures for these approaches are usually fixed, paid as a cost-share or subsidy for inputs, and measurement of environmental improvement is not included. The strengths of these approaches include low administrative costs with no need to develop models or measure site-specific or regional conditions, and broad applicability within a region, and there is no risk to landowners as the receipt of payments is based entirely on inputs. However, there is only an indirect link between environmental performance and payment structure, and the environmental outcomes are likely lower than with the other approaches discussed.

Applying the PMF to policy examples

Many of the categories of approaches identified in the literature survey (i.e. water quality trading, payments for ecosystem services, reverse auctions, emissions charges and cross-compliance) varied in terms of the degree of performance for each case. Further, the implementation was often specific to the particular case, resulting in differing capacity for performance within the categories, as well as among them, as shown in Figure 2. While the categories of approaches varied in their performance capacity, they can be qualitatively categorized based on the scope of two dimensions of performance: measurement and payment/penalty structure.

Figure 2. Performance classifications of policy approaches to manage water quality for agricultural landscapes.

Depending on the implementation context for the performance-based approaches, categorization on the scale may differ (Figure 2). For example, if a cross-compliance program was based on achieving water quality standards, rather than assuming water quality standard achievement with the adoption of specific practices, then reliable, site-specific models would be required and payments could be based more strongly on environmental performance. This change would alter the position of the cross-compliance approach from very weak to moderately performance-based. The context for implementation of these approaches, which influences the capability for measurement of water quality and payment/penalty structure, is critical to the degree of cost effectiveness achieved by performance-based approaches.

Applications of the PMF

A single category of performance-based approaches can occupy more than a single point on the PMF, as shown by water quality trading and differentiated payments for ecosystem services. The methods used to measure performance and to pay or penalize agricultural producers can vary amongst specific cases, creating a range, or scope, of potential for incorporating elements of performance (Figure 2). This identifies opportunities to improve the degree of performance of a policy instrument where necessary drivers and enabling conditions are present (Rose 1993; Hospers and Beugelsdijk 2002). For example, in its current application in the UK, cross-compliance for nutrient-vulnerable zones is based almost entirely on agricultural producers within the zone adopting specific practices. This makes this form of cross-compliance a very weakly performance-based approach from an individual producer perspective, but stronger from a whole watershed-based perspective. Basing compliance on flexible site-specific management practices, where physical process models are available to provide reliable estimates of the environmental outcomes of those practices, would create a stronger performance-based cross-compliance approach at the individual producer level. The PMF provides insight into what is needed to achieve a stronger performance element in water quality programs for agriculture.

The PMF has further potential to guide a general approach to categorizing performance-based approaches, and may be useful in other research or policy studies to enable the selection of appropriate instruments based on measurement or payment structure capabilities, or where a specific level of environmental performance is desired. Using the PMF, a region can assess its potential to measure environmental outcomes and the social and institutional capacity to implement varying degrees of differentiated payments. Based on these two measures of potential, the region can determine where they are positioned on the PMF to inform policy approaches that are consistent with their position within the framework.

Success and the PMF

In addition to the organizational framework provided by the PMF, an investigation was performed to assess the linkage between a greater potential for incorporating performance in the usage of the approach, and the reported success of the specific programs. In this analysis, the definitions of success and reports of success are based on available information from the literature and, therefore, no assertions can be made beyond the acknowledgement of a general trend. This potential relationship between degree of performance and degree of success warrants further study with quantifiable metrics to measure reported successes.

Reported policy success was mostly situated within the social and environmental contexts. Social success was often reported as stakeholder buy-in, development of trust and high rates of participation by agricultural producers (e.g. Perrot-Maȋtre 2006; Morton 2008; O’Grady 2011). Environmental success was evaluated based on the measured or estimated improvements to water quality as a result of the implementation of a program (e.g. Perrot-Maȋtre 2006; Winsten 2009). The environmental outcomes often varied depending on the pollutant under consideration. For example, in the MINeral Accounting System (MINAS) program, N was reduced in water bodies, but P was not. In a few cases, economic successes were also noted. The Vittel water company’s PES program identified economic success for the company as evidenced by the ability to continue to operate with assurances of adequate water quality, for the program participants, based on payments to farmers that were negotiated individually, and also to the broader community with many community members being employed by Vittel (Perrot-Maȋtre 2006).

In the literature survey, there appeared to be a weak correlation between reported successes of programs employing performance-based approaches and increasing capacity for performance. For example, where a weakly performance-based cross-compliance approach was used in the UK, there were reported mixed successes and failures, while some of the more strongly performance-based approaches (particularly those that used differentiated payments for ecosystem services) reported high levels of success in several areas (economic, environmental and social) and no failures. Further evidence of this weak trend was the MINAS program, which failed to reduce P emissions and was eventually discontinued (van Grinsven et al. 2005).

An important factor in interpreting the reported success of these approaches may be the temporal and physical scales of the programs when reporting successes. Many of the pilot and small watershed-based programs reported greater successes than larger, established regional or national programs. This may be due in part to: (1) the scale of the project: small vs. large scale, and (2) the amount of time available to assess program performance and success: time and effort devoted by researchers to small pilot projects vs. resources available to governments in larger-scale efforts. From the approaches assessed in this study, it appears that where researchers were involved, even in situations where the approach was government-led, there were more in-depth data regarding social successes in particular.

Risk and the PMF

A common theme identified in the reviewed programs was the importance of the “conditionality” aspect of performance-based approaches (Wunder et al. 2008) and the associated distribution of risk. Approaches that tie payments or penalties to the achievement of water quality outcomes produce, or increase, the inherent risk of achieving the desired outcome, primarily due to factors such as variable weather that can directly or indirectly influence water quality beyond the participants’ control. This risk must be borne either by the program participant(s) or by the program provider, or shared by these and other stakeholder groups. As the degree of performance incorporated in the payment/penalty structure increases, risk in receiving payments for water quality increases for program participants. For example, a program with a low position on the PMF, such as a practice-based payment program, will provide a payment regardless of environmental outcome, so risk to the participant is limited. Conversely, payments for incremental improvements to water quality that are measured directly will impose substantial risk due to conditions, beyond the control of the agricultural producer, having a significant impact on water quality outcomes and associated payments. A high degree of risk was identified in several of the programs as an issue to be resolved in order to implement a performance-based approach (e.g. differentiated payments for ecosystem services, water quality trading and cross-compliance).

Balancing risk allocation for stakeholder buy-in was and is an important consideration in implementing performance-based approaches. Often, the issue was addressed by transferring risk from the agricultural producer (participant) to the program provider or other bodies through guaranteed payments for actions. This created a substantial weakening effect of the performance-based aspect of the programs. For example, in the South Nation water quality trading program, a Statement of Roles and Responsibilities was created that shifted the liability for achieving water quality outcomes from landowners to purchasers of water quality credits. This statement created the necessary stakeholder buy-in to proceed with the program (O’Grady 2011). Further, Wunder (2006) reported that most existing payments for ecosystem services schemes do not adhere to the “conditionality” requirement to maintain relationships with farmers. The inference in Wunder’s report is that conducting PES programs with non-payment for undelivered ecological goods and services would likely have negative impacts on relationships between governments and farmers. Tools to manage the effects of environmental variability on environmental outcomes have been proposed, including a combination of fixed and variable payments that reduces the payment risk borne by agricultural producers (Meijerink 2008). The assurance of a payment increases participation, and the variable payment provides added incentive to produce positive environmental outcomes. Another tool to reduce the negative impact of risk on program participation is a relative performance evaluation, where the performances of agricultural producers are compared to one another, rather than actual measurements or estimates of environmental outcome production (Zabel and Roe 2009).

Conclusions

The vast majority of OECD member countries have not used performance-based approaches in agri-environmental policies related to water quality (Latacz-Lohmann and Van der Hamsvoort 1997; Vojtech 2010). This review is limited by the relative newness of the use of performance-based approaches for water quality and agriculture, as well as limited published literature related to these approaches. However, it is evident from OECD documents that all member countries have several agri-environmental regulations, policies, and programs to manage water pollution that are based on inputs or BMP adoption (Vojtech 2010). Where performance-based approaches have been used, they generally have been piloted but not implemented into longer-term programs. The US, Australia and Canada are the main countries that have applied instruments that use performance or outcomes rather than inputs as measures for improvements to water quality and to calculate payment levels. Other OECD countries have used market-based policy instruments for a number of other issues, but not specifically to manage water quality (Vojtech 2010).

The contextual conditions of regions where performance-based approaches have been implemented show some similarities, particularly in the need for a physical driver, institutional capacity and favourable social conditions. There were also noticeable differences among cases with performance-based approaches where unique conditions such as data availability, or private company initiatives, created opportunities for specific approaches. However, general lessons can be drawn based on the similarities among cases.

Contextual factors were employed to categorize each performance-based approach on the Performance-based Measures Framework (PMF), which describes their performance capacity using measurement method and payment/penalty structure as the metrics. It is evident from the application of this tool that substantial variability exists in terms of the potential for incorporating performance into agri-environmental policy approaches. The PMF may also be applied as a tool to identify a region’s capacity to implement a performance-based approach based on the same metrics of measurement and payment/penalty structure.

Acknowledgements

Financial support for this research came from the Social Sciences and Humanities Research Council of Canada, the University of Saskatchewan, the Linking Agriculture and Environment Research Network, the Sustainable Prosperity Network and the Institute for Environmental Sustainability, Mount Royal University. We also thank the editors and reviewers for their thoughtful and insightful feedback.