Evaluating the effectiveness of land and water integrative practices for achieving water sustainability within the Colorado River Basin: perceptions and indicators

ABSTRACT

Principle 12 of the OECD Principles on Water Governance calls for the evaluation of sustainable water policies, which to be effective must be salient, credible and legitimate. Using a 2021 survey of land and water managers within the Colorado River Basin, we examine the salience, credibility and legitimacy of two approaches to evaluate policies that integrate water and land management: using practitioners’ perceptions of policies effectiveness to achieve water sustainability goals, and the importance of indicators to assess the water sustainability of a community. Results show perceptions of policy effectiveness and indicator importance vary with the context of organization and place.

KEYWORDS:

• Land and water integration
• water management
• land planning
• water sustainability indicators
• policy evaluation
• Colorado River Basin

Introduction

Access to clean and relatively inexpensive water has been a key factor in the growth of urban and rural communities in the United States (Quay, 2012). Technology over the last 100 years has created water systems that have reliably delivered water to the vast majority of communities. Most people assume that water will flow when they turn on their faucet and likely have little knowledge of where the water is coming from and how it reached their sink. Water sustainability has been at best a secondary issue of concern. However, in the Colorado River Basin, this soon will not be the case for many communities across the US West (Arizona, California, Colorado, Nevada, New Mexico, Utah and Wyoming) and Mexico as water demand is projected to exceed the available surface water (McCabe & Wolock, 2021; Miller et al., 2021; Sampson et al., 2016). In 2021, the United States Bureau of Reclamation declared the first shortfage of water allocation on the Colorado River since the 1922 Colorado River Compact created the law for river rights management (Goodland, 2021; Sakas, 2021). Trends of growth, economy, politics and climate are creating challenges for communities to maintain sustainable water systems (Hondula et al., 2019). The strategies and policies used over the last century to store water during wet years in large federal reservoirs and release and deliver water from reservoirs in dry years through a system of ditches and canals will likely not be enough to achieve water sustainability in the future (Gober et al., 2016; Udall, 2020). New strategies will be needed.

Water sustainability has been an evolving concept ranging from the simple concept of supply being balanced with demand to more complex and broader concepts of social justice and long-term processes. Hufschmidt and Tejwani (1993, p. 3) defined water sustainability as ‘a set of activities that ensures that the value of the services provided by a given water resource system will satisfy present objectives of society without compromising the ability of the system to satisfy the objectives of future generations’. Modelled after the Brundtland Commission report (World Commission on Environment and Development (WCED), 1987) this is a very anthropogenic view point (Jones, 2011) and later definitions included natural or environmental needs as well as human needs such as the 2013 California Water Plan definition: ‘Water sustainability is the dynamic state of water use and supply that meets today’s needs without compromising the long term capacity of the natural and human aspects of the water system to meet the needs of future generations’ (Department of Water Resources, 2013, p. 5–8). For the purposes of this research, we follow Wiek and Larson’s (2012) definition of water sustainability as:

Having sufficient water to meet the community’s economic, social, and environmental goals, without compromising the ability of future generations to meet their own water needs and community goals. (paraphrased by current authors)

The integration of the activities of land-use management – including planning and regulation – with the activities of water management – including operations and planning – has recently been cited as one way to increase the sustainability of community water systems (Fedak et al., 2018; Quay et al., 2018a; Räsänen et al., 2018). This is also a key element of Principle 3 of the OECD Principles on Water Governance (OECD, 2015), which ‘Encourages policy coherence through effective cross-sectoral co-ordination, especially between policies for water and the environment, health, energy, agriculture, industry, spatial planning and land use’ (emphasis added). For example, including a review of how much water is available to serve proposed development during the planning review of the development process helps to avoid over-committing available water supplies. Additionally, including water efficiency requirements as part of a development review can help to reduce future water demand. Several institutions, such as the American Planning Association, the American Water Works Association, the Babbitt Center for Land and Water Policy (which was created by the Lincoln Institute of Land Policy), the Colorado Water Conservation Board and the Water Research Foundation, are now supporting efforts to expand these activities (Hansman, 2021), yet there is currently little information about the status of such activities or evidence to understand how these activities will actually help achieve sustainability.

The OECD Principles on Water Governance include ‘Principle 12: Promote regular monitoring and evaluation of water policy and governance where appropriate, share the results with the public and make adjustments when needed’. To implement this principle with regard to integrated land and water management would require some evidence, that is, some method to evaluate these policies. One way to provide such evidence would be to assess the current state of water sustainability and practices of integrated land and water management in terms of effectiveness. The OECD emphasizes effectiveness as an important dimension of water governance and defines it as ‘the contribution of governance to define clear sustainable water policy goals and targets at all levels of government, to implement those policy goals, and to meet expected targets’ (OECD, 2015, p. 3). Thus, the extent that the implementation of policies to achieve targets of sustainability would be one assessment of effectiveness. This assessment could be based on either professional opinion of effectiveness or more objective indicators of water sustainability that are credible and salient to water and land managers.

To address this gap in the evidence and to explore these aspects of assessment, the Babbitt Center for Land and Water Policy and the Decision Center for a Desert City at Arizona State University developed and administered a survey for the spring of 2021 to assess the current state of integration between land and water management activities within communities of the Colorado River Basin. In our efforts to design the survey to evaluate these policies, there was much discussion on how to do that. We realized that such an evaluation must be outcome based and needed to be credible, salient and legitimate in order to influence policy, but how to achieve such was unclear. We concluded there were two approaches, either based on practitioners’ perceptions of outcomes or based on more objective indicators that measured outcomes. In our discussion we saw the pros and cons of both approaches, thus we designed the survey to also explore these questions. The intent of this article is to report on these questions using our survey as a case study.

Survey methodology

The abovementioned survey was developed based on several social science research frameworks: integrated policymaking (Meijers & Stead, 2004), values and goals – normative (Carter et al., 2005) and discourse (Lebel et al., 2005), water governance, more specifically, the management and transition framework (Pahl-Wostl et al., 2010), vulnerability (Larson et al., 2013), relevant dominant uncertainty (RDU) for policy and management applications (Thorne et al., 2018), water sustainability assessment framework (Godwyll et al., 2018), and integrated analysis of water governance (Quay et al., 2018b; Wiek & Larson, 2012). The survey was designed to address several research questions, including two related to practices and indicators:

• What are land and water managers’ perceptions of the effectiveness of integrated policies and practices to achieve sustainability goals?

• What factors are important to land and water managers to assess the current state of achieving sustainability goals and targets?

The first research question helped to examine the credibility and legitimacy of using practitioner perceptions as a general evaluation of the impacts water governance policies have on community water sustainability. The second research question helped to examine the credibility and salience of using water-related indicators to assess over time the impact that water governance policies may have on community water sustainability.

The survey had 17 questions with 119 descriptive items to rate. Of the 17 questions, seven were designed to address the two research questions above. Survey questions 4–9 were based on a list of 34 potential integrated land and water practices grouped into six categories and were developed based on a list of 268 unique water and land practices developed by the Babbitt Center and other research (Quay et al., 2018b). Table 1 includes all six effectiveness survey questions, 34 practice items, both relevant OECD Principles, and which practice each principle pertains to. This list of practices reflects specific aspects of two OECD principles of water governance focused on enhancing the effectiveness of water governance including: (1) Principle 1, clearly allocate and distinguish roles and responsibilities including priority setting and strategic planning, financing and budgeting, infrastructure investment, and regulation and enforcement; and (2) Principle 3, encourage policy coherence through effective cross-sectoral coordination, especially between policies for water and the environment, health, energy, agriculture, industry, spatial planning and land use, including encouraging coordination mechanisms to facilitate coherent policies across ministries, public agencies and levels of government; and coordinated management of the use of water resources, taking into account policies that affect water availability.

Table 1. Survey practice questions with the OECD Principles on Water Governance

Question		OECD Principle
Q4	How effective are water supply management practices for your organization’s efforts to achieve water sustainability?
	Acquiring new water supplies: activities acquiring and/or developing new water supplies such as desalination or ground and surface water rights	1
	Water banking/recovery: activities recharging water into an aquifer, where it is stored until recovered through ground water pumping	1
	Water reuse: activities related to the reuse of wastewater as a water supply, such as treatment and distribution	2
	Water transfer and sharing agreements: activities developing and/or using agreements between two organizations to transfer or share water rights and/or supplies	1
	Water and wastewater infrastructure: activities related to the planning, building and operation of facilities to treat and distribute water or collect and treat wastewater	2
	Water quality: activities managing the quality of water delivered to customers or discharged to the environment	2
	Water resource management planning: activities related to planning water supplies and operations needed to meet future water demand, including water resource plans	1
	Water access: activities to ensure adequate access to water is available to meet community economic and social needs	2
Q5	How effective are water rates/financing practices for your organization’s efforts to achieve water sustainability?
	Development fees: activities developing and/or using fees for new development to fund new infrastructure or supplies to serve the development	1
	Conservation incentive rate structure: activities developing and/or using water rates that charge high water users more than low water users	1
	Water affordability: activities related to policies and programmes that keep water affordable for all customers	1
	Rates based on water budgets: activities developing and/or using tiered water rates tied to water budgets reflecting expected efficient customer water use	1
Q6	How effective are water demand and conservation practices for your organization’s efforts to achieve water sustainability?
	Public education on water conservation: activities that educate the public on how to use water more efficiently	2
	Water demand scenario planning: activities related to developing multiple forecasts of water demand based on different future factors affecting water demand such as land use	1
	Water conservation planning: activities, planning programmes and policies intended over time to increase water use efficiency	2
	Drought response planning: activities that develop and/or use programmes and policies applied during drought conditions to reduce water demand or increase water supplies	1
	Water efficiency incentive programmes: activities developing and/or using policies and programmes that provide incentives to customers, financial or social, to increase water efficiency	2
Q7	How effective are land-use planning practices for your organization’s efforts to achieve water sustainability?
	Water in comprehensive or general plan: activities related to including water information and issues in a comprehensive plan	1
	Density/infill/clustered development: activities developing and/or using urban design policies or regulations intended to increase water use efficiency	2
	Growth boundaries or annexation policies: activities that develop and use policies or regulations spatially limiting development to areas with available water access and supply	1
Q8	How effective are land and water collaboration practices for your organization’s efforts to achieve water sustainability?
	Joint water and land-use agency education programmes: activities in which water and land-use management agencies are jointly developing and/or implementing education programmes	2
	Population and growth projections align: activities that coordinate the forecasts of population and growth used for planning activities in land-use and water agencies	1
	Coordination of development review: activities that coordinate the development review processes conducted by land-use and water management agencies	2
	Collaboration on development code: activities in which land and water agencies are collaboratively developing regulations for land-use management and development review	2
	Coordination of water policy and procedures: activities coordinating water management policies and procedures between different water or land management agencies	2
	Integrating water and land-use planning in capital improvement plans: activities that coordinate water and land management related projects and issues within capital improvement planning processes	1
Q9	How effective are practices to integrate water into land regulations and policies for your organization’s efforts to achieve water sustainability?
	Demonstrate sufficient water supply before approving development: regulations or policies that require evidence that sufficient water supply exists to serve new development before it is approved	2
	Net-zero or water demand offset policy: regulations or policies that require new development to offset any new demand it creates by reducing existing demand	2
	Post-occupancy enforcement on water policies by a land-use authority: regulations or policies requiring land-use agencies to enforce water-related development requirements during development occupancy	2
	Water allotment: regulations or policies allocating access to water supplies for new development based on level of benefit, such as jobs or quality of life	2
	Incentives for developers based on development water efficiency: regulations or policies that provide special development incentives for projects that increase water efficiency	2
	Water efficiency requirements: regulations or policies requiring new development to use water efficient design, such as native landscapes or water harvesting	2
	Landscaping/irrigation policies and regulations: regulations or policies that increase the efficiency of landscape water use	2
	Building/plumbing policies and codes: building construction regulations or policies requiring water efficient fixtures and practices	2

To rate these questions, respondents were asked to use a Likert scale of (1) Not at all effective, (2) A little effective, (3) Somewhat effective, (4) Very effective, (5) Extremely effective and (6) Don’t know.

Survey question 14: ‘How important are sustainability indicators in your organization’s assessment of water sustainability’, was based on water sustainability indicators developed by Godwyll et al. (2018), which used a Likert scale ranging from ‘Not at all important’, ‘A little important’, ‘Somewhat important’, ‘Very important’ to ‘Extremely important’. The 12 indicators included:

• Balance between groundwater pumping and recharge/replenishment.

• Balance between supply and demand.

• Meeting basic economic needs for water today and in the future.

• Meeting quality-of-life needs for water, such as health, comfort and recreation.

• Level of wastewater reuse.

• Level of water loss.

• Communication and coordination between local, regional, state, and federal managers and planners.

• Capacity to continue increasing water efficiency.

• Preparing for future water shortages.

• Adequate stream flows to sustain environmental integrity and ecosystem functions.

• Meeting agricultural needs for water.

• Equitable distribution of impact and cost of potable water service.

The target population of the survey was the senior land and water managers for cities and counties with a population over 40,000 within the seven states included in the Colorado River Basin: Arizona, California, Colorado, Nevada, New Mexico, Utah and Wyoming. This included 283 cities and 107 counties representing 566 possible water and land managers from cities and 214 water and land managers from counties. A search for contact information for these managers resulted in 597 unique cohorts (a cohort is a senior land and water manager per city or county). The survey was deployed using Qualtrics; an invitation email was sent to each cohort, which resulted in the completion of 236 surveys.

Practices and policies can be described in general and theoretical terms, but they are implemented by specific people within specific organizations within specific places. Though we often collectively describe these organizations, people and places as society, they each have their own unique attributes which creates a specific context for policy implementation. Understanding this context can be important when assessing cases of such implementation (Gibbons, 2000; Larson et al., 2009; Volkery & Ribeiro, 2009). To explore the context of perceptions of effectiveness and importance, secondary data were collected for each cohort to capture the context of organization and place. Organizational context was represented by identifying the manager type of each respondent (127 land managers and 109 water managers). Place context was represented by census data (average household income, and population) for the place where each respondent worked and the type of government (city/county) for whom they worked. A k-means classification was applied to all the cities’ census data and place type data to create four census groups: Small poor city (41%), large moderate wealth city/county (1%), medium county mixed wealth (30%) and small prosperous city (28%).

Results

Effectiveness

Table 2 shows the frequencies of responses for questions 4–9 by individual practices. Six of the 34 practices were rated ‘extremely effective’ by at least 30% of the respondents, which include: ‘Acquiring new water supplies’ (34%), ‘Water and wastewater infrastructure’ (39%), ‘Water quality’ (44%), ‘Water resource management planning’ (43%), ‘Water access’ (40%), and ‘Demonstrate sufficient water supply before approving development’ (33%). In addition to these six, 13 of the remaining practices had at least 40% of the respondents rate them ‘Very’ or ‘Extremely’ effective, which included at least one practice for each question group, all the practices under question 4: ‘Water supply management’, at least half of the practices under question 5: ‘Water demand’ and question 9: ‘Water into land regulations and policies’. This suggests that over half of the practices were considered effective.

Table 2. Frequencies and mean practice effectiveness results

Practice	Not at all (1)	A little (2)	Somewhat (3)	Very (4)	Extremely (5)	Don’t know	Frequency N^a	Mean^b,c	Mean N^b	SD	Skewness
Q4: Water supply management
Acquiring new water supplies	11%	10%	17%	29%	34%	0%	168	3.65	168	1.32	–0.73
Water banking/recovery	27%	12%	22%	17%	23%	0%	155	2.96	155	1.51	–0.01
Water reuse	14%	17%	23%	22%	23%	0%	175	3.25	175	1.35	–0.22
Water transfer and sharing agreements	14%	14%	25%	18%	28%	0%	154	3.31	154	1.39	–0.26
Water and wastewater infrastructure	5%	7%	18%	30%	39%	0%	183	3.91	183	1.16	–0.95
Water quality	4%	4%	16%	32%	44%	0%	180	4.07	180	1.06	−1.16
Water resource management planning	4%	7%	15%	30%	43%	0%	184	4.01	184	1.12	−1.06
Water access	5%	8%	18%	29%	40%	0%	184	3.90	184	1.17	–0.091
Q5: Water rates and financing
Development fees	6%	8%	20%	23%	20%	23%	220	3.54	169	1.21	–0.54
Conservation incentive rate structure	10%	11%	13%	21%	13%	32%	219	3.25	148	1.33	–0.33
Water affordability	8%	10%	16%	19%	11%	35%	219	3.24	142	1.27	–0.31
Rates based on water budgets	13%	8%	15%	14%	9%	42%	219	2.96	128	1.38	–0.09
Q5: Water demand and conservation
Public education on water conservation	3%	13%	28%	24%	16%	16%	216	3.46	181	1.07	–0.20
Water demand scenario planning	7%	9%	22%	21%	14%	26%	214	3.35	158	1.21	–0.36
Water conservation planning	3%	10%	25%	27%	18%	18%	215	3.57	177	1.07	–0.36
Drought response planning	4%	11%	20%	24%	19%	21%	217	3.56	171	1.16	–0.45
Water efficiency incentive programmes	7%	13%	20%	18%	15%	26%	215	3.28	159	1.26	–0.21
Q7: Land-use planning
Water in comprehensive or general plan	5%	17%	25%	25%	15%	13%	216	3.33	188	1.14	–0.17
Density/infill/clustered development	7%	12%	32%	23%	8%	17%	214	3.16	177	1.06	–0.26
Growth boundaries or annexation policies	13%	8%	20%	23%	13%	23%	216	3.21	166	1.31	–0.37
Q8: Land and water collaboration
Joint water and land-use agency education programmes	11%	15%	20%	10%	5%	37%	210	2.73	132	1.19	–0.18
Population and growth projections align	3%	13%	24%	24%	17%	18%	210	3.47	172	1.12	–0.26
Coordination of development review	5%	10%	22%	31%	19%	12%	210	3.54	184	1.13	–0.53
Collaboration on development code	8%	15%	29%	22%	12%	14%	210	3.17	180	1.16	–0.16
Coordination of water policy and procedures	7%	16%	27%	22%	6%	22%	210	3.05	164	1.08	–0.17
Integrating water and land-use planning in capital improvement plans	9%	14%	20%	20%	11%	25%	210	3.13	157	1.24	–0.18
Q9: Water into land regulations and policies
Demonstrate sufficient water supply before approving development	6%	9%	16%	26%	33%	9%	204	3.79	185	1.24	–0.82
Net-zero or water demand offset policy	27%	7%	13%	5%	4%	43%	204	2.15	117	1.30	0.75
Post-occupancy enforcement on water policies by a land-use authority	24%	11%	16%	8%	4%	37%	204	2.31	128	1.27	0.51
Water allotment	26%	8%	9%	6%	5%	45%	202	2.20	111	1.39	0.79
Incentives for developers based on development water efficiency	18%	16%	15%	12%	3%	37%	204	2.46	129	1.22	0.32
Water efficiency requirements	7%	15%	21%	29%	13%	15%	204	3.29	174	1.18	–0.35
Landscaping/irrigation policies and regulations	7%	12%	27%	26%	21%	6%	204	3.46	191	1.19	–0.39
Building/plumbing policies and codes	4%	8%	23%	32%	21%	12%	204	3.67	179	1.07	–0.63

Note: ^aIncludes ‘Don’t know’.

^bDoes not include ‘Don’t know’.

^cFigures in bold font indicate practices that had a mean above the mean average across practices of 3.3.

No practice had more than 30% of the respondents rate it ‘Not at all effective’ and the four practices with the most respondents indicating ‘Not at all effective’ were ‘Water banking/recovery’ (27%), ‘Net-zero or water demand offset policy’ (27%), ‘Post-occupancy enforcement on water policies by a land use authority’ (24%) and ‘Water allotment’ (26%). This suggests that only a few of the practices were considered not effective.

A few practices had high rates of ‘Don’t know’, including ‘Rates based on water budgets’ (42%), ‘Net-zero or water demand offset policy’ (43%) and ‘Water allotment’ (45%), suggesting there was less experience with these practices.

The variance of the means for these results is not large, but most are skewed towards extremely effective except for ‘Net-zero or water demand offset policy’, ‘Post-occupancy enforcement on water policies by a land-use authority’, ‘Water allotment’ and ‘Incentives for developers based on development water efficiency’ which are skewed towards ‘Not at All’ effective.

There are significant differences in these means between different groups of contexts. Table 3 shows the results of an analysis of variance (ANOVA) test for differences between land and water manager effectiveness responses; and Table 4 shows the results for differences between census groups. A total of 50% of the practices showed a statistically significant difference between effectiveness ratings of land managers and water managers. In all but one case for the practices with a statistically significant difference, water managers rated effectiveness higher than land managers. The one exception was ‘Growth boundaries or annexation policies’ which was rated among the water managers’ lowest practices and among the land managers’ highest practices.

Table 3. Analysis of variance (ANOVA) results for the test of difference between land and water manager responses of effectiveness

Practice	Mean		One-way ANOVA
	Land	Water	Test	Test value	DF1	DF2	Sig.
Acquiring new water supplies	3.12	4.03	F	33.99	1	166	0.000*
Water banking/recovery	2.72	3.13	F	6.41	1	153	0.094
Water reuse	3.11	3.35	F	2.50	1	173	0.243
Water transfer and sharing agreements	3.06	3.48	F	6.34	1	152	0.070
Water and wastewater infrastructure	3.59	4.20	F	17.08	1	181	0.000*
Water quality	3.79	4.33	Welch	11.90	1	159	0.001*
Water resource management planning	3.56	4.38	Welch	26.33	1	144	0.000*
Water access	3.59	4.18	F	15.78	1	182	0.001*
Development fees	3.18	3.85	Welch	13.10	1	145	0.000*
Conservation incentive rate structure	2.82	3.55	F	18.95	1	146	0.001*
Water affordability	2.71	3.58	F	25.50	1	140	0.000*
Rates based on water budgets	2.69	3.14	F	6.40	1	126	0.067
Public education on water conservation	3.27	3.62	F	5.35	1	179	0.030*
Water demand scenario planning	3.08	3.58	Welch	7.15	1	156	0.008*
Water conservation planning	3.29	3.83	F	12.50	1	175	0.001*
Drought response planning	3.26	3.79	F	11.70	1	169	0.003*
Water efficiency incentive programmes	2.94	3.55	Welch	9.65	1	156	0.002*
Water in comprehensive or general plan	3.29	3.38	F	0.34	1	186	0.610
Density/infill/clustered development	3.27	3.00	Welch	2.89	1	163	0.091
Growth boundaries or annexation policies	3.41	2.94	F	9.02	1	164	0.021*
Joint water and land-use agency education programmes	2.73	2.72	F	0.01	1	130	0.947
Population and growth projections align	3.31	3.64	F	4.79	1	170	0.050*
Coordination of development review	3.59	3.47	F	0.70	1	182	0.460
Collaboration on development code	3.22	3.10	F	0.60	1	178	0.505
Coordination of water policy and procedures	2.92	3.20	Welch	2.92	1	159	0.089
Integrating water and land-use planning in capital improvement plans	3.07	3.21	F	0.82	1	155	0.466
Demonstrate sufficient water supply before approving development	3.75	3.84	F	0.33	1	183	0.641
Net-zero or water demand offset policy	2.05	2.24	F	1.06	1	115	0.431
Post-occupancy enforcement on water policies by a land-use authority	2.37	2.25	F	0.46	1	126	0.596
Water allotment	1.87	2.49	F	10.84	1	109	0.017*
Incentives for developers based on development water efficiency	2.18	2.77	F	11.35	1	127	0.005*
Water efficiency requirements	3.23	3.37	F	0.90	1	172	0.421
Landscaping/irrigation policies and regulations	3.36	3.57	F	2.04	1	189	0.229
Building/plumbing policies and codes	3.69	3.65	F	0.09	1	177	0.783

Note: *Significance < 0.05

Table 4. Analysis of variance (ANOVA) results for the test of difference between census group manager responses of effectiveness

Practice	Census group				One-way ANOVA
	Small poor city	Large moderate wealth city/county	Medium county mixed wealth	Small prosperous city	Test	Test value	DF1	DF2	p-value*
Acquiring new water supplies	3.88	4.50	3.21	3.79	Welch	2.621	3	5	0.158
Water banking/recovery	3.06	4.50	2.70	3.05	Welch	3.138	3	5	0.118
Water reuse	3.31	5.00	2.75	3.62	F	25.601	3	166	0.003*
Water transfer and sharing agreements	3.40	3.00	3.06	3.50	F	5.240	3	147	0.448
Water and wastewater infrastructure	4.08	4.33	3.33	4.21	Welch	4.615	3	11	0.026*
Water quality	4.26	4.00	3.51	4.35	Welch	4.874	3	10	0.025*
Water resource management planning	4.29	5.00	3.38	4.24	F	32.541	3	173	0.000*
Water access	4.04	3.00	3.20	4.40	F	40.149	3	173	0.000*
Development fees	3.74	3.00	2.91	3.87	F	25.942	3	160	0.000*
Conservation incentive rate structure	3.30	5.00	2.79	3.65	F	17.627	3	140	0.018*
Water affordability	3.33	0.00	2.70	3.71	Welch	6.697	2	82	0.002*
Rates based on water budgets	3.06	0.00	2.47	3.45	F	16.942	2	120	0.012*
Public education on water conservation	3.45	4.00	3.04	3.94	F	19.986	3	173	0.000*
Water demand scenario planning	3.41	3.50	2.93	3.66	F	11.892	3	151	0.041*
Water conservation planning	3.59	5.00	3.10	3.96	F	20.043	3	170	0.000*
Drought response planning	3.56	4.50	3.10	4.00	Welch	4.720	3	5	0.062
Water efficiency incentive programmes	3.31	5.00	2.64	3.71	F	26.787	3	150	0.001*
Water in comprehensive or general plan	3.29	3.00	3.14	3.52	F	4.063	3	178	0.373
Density/infill/clustered development	3.18	3.50	2.73	3.57	F	17.963	3	168	0.001*
Growth boundaries or annexation policies	3.21	2.00	3.02	3.40	F	4.825	3	158	0.427
Joint water and land-use agency education programmes	2.77	2.00	2.66	2.70	F	0.800	3	124	0.907
Population and growth projections align	3.52	4.00	3.00	3.75	F	14.992	3	163	0.006*
Coordination of development review	3.57	4.00	3.14	3.82	Welch	2.605	3	5	0.165
Collaboration on development code	3.11	5.00	2.88	3.43	F	10.757	3	170	0.042*
Coordination of water policy and procedures	3.09	4.00	2.80	3.24	F	5.668	3	155	0.175
Integrating water and land-use planning in capital improvement plans	3.17	4.00	2.55	3.70	F	29.499	3	148	0.000*
Demonstrate sufficient water supply before approving development	3.78	2.50	3.55	4.12	Welch	3.356	3	5	0.109
Net-zero or water demand offset policy	2.00	3.00	2.00	2.58	F	8.431	3	110	0.176
Post-occupancy enforcement on water policies by a land-use authority	2.27	0.00	2.03	2.72	F	9.452	2	122	0.055
Water allotment	2.02	0.00	1.94	2.88	F	14.775	2	104	0.020*
Incentives for developers based on development water efficiency	2.30	0.00	2.44	2.71	Welch	1.505	2	72	0.229
Water efficiency requirements	3.36	3.50	2.74	3.67	Welch	4.554	3	5	0.066
Landscaping/irrigation policies and regulations	3.52	3.50	2.83	3.92	Welch	6.999	3	5	0.029*
Building/plumbing policies and codes	3.78	4.00	3.05	4.06	Welch	5.683	3	5	0.047*

Note: *Significance < 0.05

More than half of the practices showed statistically significant differences in effectiveness between different census groups. The pattern of these differences was a bit mixed across all the practices. In general, ‘Small prosperous city’ groups rated effectiveness higher than the other groups and ‘Medium county mixed wealth’ group rated effectiveness lower than the other groups. The pattern for the ‘Small poor city’ and ‘Large moderate wealth city/county’ was mixed with some practices being higher than other groups and some being lower than other groups. However, these two groups rarely agreed on practice, if one rated in at the higher end the other would rate it at the lower end.

Water sustainability indicators

Survey question 14: How important are the following sustainability indicators in your organization’s assessment of water sustainability? had land and water managers rate the importance of metrics that could be used to measure some aspect of water sustainability. Table 5 shows the results for land and water managers’ ratings of the water sustainability indicators listed in rank order based on mean importance. The distributions of ratings for all indicators except three were somewhat normally distributed, though all were skewed slightly to ‘Extremely’. ‘Balance supply demand’, ‘Economic needs met’ and ‘Water shortage’ were all very strongly skewed towards ‘Extremely’. ‘Safe yield’ was bimodal with ratings clustered around ‘None’ and ‘Very’, though less around ‘None’. Three had 20% or more of their ratings of ‘None’ or ‘A little’: ‘Safe yield’ (22%), ‘Water reuse’ (26%) and ‘Agricultural needs met’ (36%). Three had over 80% of their ratings ‘Very’ or ‘Extremely’: ‘Balance supply demand’ (83%), ‘Economic needs met’ (83%) and ‘Quality-of-life met’ (80%). Four are clearly at the top, ‘Economic needs met’, ‘Balance supply demand’, ‘Quality-of-life met’ and ‘Water shortage’. This suggests that water and land managers are focused on ensuring adequate supplies to meet the economic and quality-of-life water needs of their communities. Two indicators were at the bottom of the list: ‘Water reuse’ and ‘Agricultural needs met’.

Table 5. Frequencies (%) of ratings for importance of water sustainability indicators

Sustainability indicator	Not at all (1)	A little (2)	Somewhat (3)	Very (4)	Extremely (5)	N	Mean	SD	Skewness
Balance supply demand	2%	4%	11%	39%	44%	199	4.2	0.9	−1.33
Economic needs met	1%	3%	13%	41%	43%	199	4.2	0.9	−1.12
Quality-of-life met	2%	3%	16%	43%	37%	198	4.1	0.9	−1.03
Water shortage	1%	8%	15%	36%	41%	198	4.1	1.0	−0.93
Increasing water efficiency	3%	6%	20%	45%	26%	198	3.9	1.0	−0.84
Water loss	5%	9%	20%	39%	28%	197	3.8	1.1	0.78
Governmental coordination	2%	7%	30%	39%	23%	199	3.7	0.9	−0.43
Equitable distribution	4%	7%	26%	40%	24%	194	3.7	1.0	−0.68
Safe yield	15%	7%	14%	34%	30%	194	3.6	1.4	−0.75
Environmental stream integrity	9%	8%	29%	33%	22%	197	3.5	1.2	−0.6
Water reuse	9%	18%	27%	24%	23%	197	3.3	1.3	−0.24
Agricultural needs met	17%	20%	20%	26%	18%	198	3.1	1.4	−0.12

Managers’ responses are skewed towards ‘Extremely important’ for all the indicators and the skew is quite large for ‘Balance supply–demand’, ‘Economic needs met’ and ‘Quality-of-life met’. There are significant differences in these means between different groups of contexts. Table 6 shows the results of an ANOVA test for differences between land and water manager importance responses; and Table 7 presents the results for differences between census groups. Differences between land and water managers were notable with 75% of the indicators showing statistically significant differences at the 95% level. Water managers consistently reported higher levels of importance than land managers. Indicators with differences between responses by census group were not as numerous with only 33% showing statistically significance at the 95% level. The pattern of responses among census groups for the different indicators was somewhat mixed. For the four indicators with a statistically significant difference, ‘Small prosperous cities’ generally ranked the indicators higher than the other groups. Responses in the ‘Medium county mixed wealth’ group were generally lower than the other groups. One exception was ‘Agriculture needs met’, which ranked the highest of all the groups. This indicator was interesting because it had some of the lowest importance ratings of all the indicators (‘Large moderate wealth city/county’ = 2.73 and ‘Medium county mixed wealth’ = 2.67) and the largest difference between the highest and lowest importance ratings of all the indicators with a statistically significant difference.

Table 6. Analysis of variance (ANOVA) results for the test of difference between land and water manager responses of importance of water sustainability indicators

Practice	Mean		One-way ANOVA
	Land	Water	Test	Test value	DF1	DF2	Sig.
Safe yield	3.48	3.68	Welch	0.973	1	166	0.325
Balance supply demand	3.91	4.56	Welch	29.869	1	193	0.000*
Economic needs met	4.02	4.44	F	13.140	1	197	0.000*
Quality-of-life met	3.97	4.29	F	6.846	1	196	0.010*
Water reuse	3.19	3.53	F	3.518	1	195	0.062
Water loss	3.56	4.01	Welch	9.318	1	191	0.003*
Governmental coordination	3.61	3.91	Welch	5.103	1	196	0.025*
Increasing water efficiency	3.72	4.03	Welch	5.379	1	195	0.021*
Water shortage	3.78	4.44	Welch	27.447	1	193	0.000*
Environmental stream integrity	3.31	3.76	F	7.461	1	195	0.007*
Agricultural needs met	2.98	3.20	F	1.282	1	196	0.259
Equitable distribution	3.54	3.97	Welch	9.141	1	191	0.003*

Note: *Significance < 0.05

Table 7. Analysis of variance (ANOVA) results for the test of difference between managers responses by census groups of importance of water sustainability indicators

Sustainability indicator	Mean				One-way ANOVA
	Small poor city	Large moderate wealth city/county	Medium county mixed wealth	Small prosperous city	Test	Test value	DF1	DF2	Sig.
Safe yield	3.65	4.33	3.48	3.59	F	0.456	3	184	0.714
Balance supply demand	4.28	4.33	3.89	4.35	Welch	1.746	3	10	0.220
Economic needs met	4.18	4.33	3.96	4.51	Welch	3.714	3	10	0.050*
Quality-of-life met	4.12	4.00	3.76	4.51	F	7.044	3	188	0.000*
Water reuse	3.39	4.67	3.05	3.54	F	2.579	3	187	0.055
Water loss	3.89	3.33	3.47	3.88	Welch	1.445	3	10	0.289
Governmental coordination	3.73	3.67	3.70	3.90	F	0.487	3	189	0.692
Increasing water efficiency	3.92	3.67	3.52	4.14	Welch	3.190	3	10	0.073
Water shortage	4.12	4.00	3.84	4.30	F	2.062	3	188	0.107
Environmental stream integrity	3.49	3.33	3.54	3.68	F	0.325	3	187	0.807
Agricultural needs met	2.73	2.67	3.73	3.10	F	6.764	3	188	0.000*
Equitable distribution	3.85	4.00	3.35	3.96	F	4.084	3	184	0.008*

Note: *Significance < 0.05

These differences between land and water managers and census groups for perceptions of effectiveness and importance of indicators show a statistically significant absolute difference but do not indicate the relative differences. Figure 1 shows the mean effectiveness of each practice by land managers and water managers, which corresponds to the data in Table 2. Though there are in some cases large differences between land and water managers for certain practices, the general pattern of which practices are rated higher, and which are rated lower than others, appear to be the same for land and water managers. Table 8 shows the results of a Pearson correlation coefficient, which indicates there is a statistically significant correlation (95%) of perceived practice effectiveness means between land and water managers.

Table 8. Pearson correlation of means of practice effectiveness between land and water managers

N	34
Pearson coefficient	0.781
t-value	7.074
p-value	0.000

Figure 1. Mean effectiveness by practice and manager type.

Figure 2 shows a similar pattern for the census groups; and Table 9 shows Pearson correlation coefficients for each pair of the different census groups. All these correlations were statistically significant but the correlations with large moderate wealth city/county and all three of the other census groups were not that strong.

Table 9. Pearson correlation means of practice effectiveness between census groups

	N = 34	Small poor city	Large moderate wealth city/county	Medium county mixed wealth
Large moderate wealth city/county	Pearson coefficient	0.566
	t-value	3.883
	p-value	0.000
Medium county mixed wealth	Pearson coefficient	0.914	0.527
	t-value	12.728	3.509
	p-value	0.000	0.001
Small prosperous city	Pearson coefficient	0.939	0.510	0.820
	t-value	15.391	3.357	8.105
	p-value	0.000	0.002	0.000

Figure 2. Mean effectiveness by practice and census group.

Similar patterns were found for means of indicator importance. Figure 3 shows the importance means by indicator and manager type. Though there are large differences in the absolute perceptions of importance for several indicators, with water mangers consistently rating indicators higher in importance than land managers, the general pattern of high and low perceptions seems the same for land and water managers. Table 10 provides the results of a Pearson correlation analysis, which shows a strong statistically significant correlation between land and water manager perceptions.

Figure 3. Mean importance by indicator and manager type.

Table 10. Pearson correlation of means of indicator importance between land and water managers

N	12
Pearson coefficient	0.946
t-value	9.260
p-value	0.000

However, these patterns do not bear out for the census groups regarding importance of indicators. Figure 4 shows the mean importance by indicator and census groups. Though there appear to be some patterns that suggest a correlation exists in some cases, there are also some patterns that show discord. Table 11 presents the results of a Pearson correlation analysis for the different census group pairs. Only one pair, ‘Small prosperous city’ and ‘Small poor city’, show a strong statistically significant correlation at the 95% level, and in one case, the ‘Large moderate wealth city/county’ and ‘Medium county mixed wealth’ pair had a negative coefficient.

Figure 4. Mean importance by indicator and census group.

Table 11. Pearson correlation of indicator importance means between census groups

	N = 12	Small poor city	Large moderate wealth city/county	Medium county mixed wealth
Large moderate wealth city/county	Pearson coefficient	0.561
	t-value	2.143
	p-value	0.058
Medium county mixed wealth	Pearson coefficient	0.403	−0.141
	t-value	1.393	−0.452
	p-value	0.194	0.661
Small prosperous city	Pearson coefficient	0.947	0.470	0.536
	t-value	9.350	1.686	2.006
	p-value	0.000	0.123	0.073

Discussion

Principle 12 of the OECD Principles on Water Governance calls for the evaluation of sustainable water policies. Policies and practices that integrate land and water management are a key component of Principle 3 within the OECD Principles. This research examined two approaches to evaluate these policies: (1) using water and land managers, their perception of effectiveness in achieving organizational water sustainability goals; and (2) using more objective indicators to assess changes in communities’ water sustainability resulting from the implementation of practices over time. The results of this survey suggest both approaches may be useful, but each is likely influenced by organizational and place-based characteristics.

These survey results suggest that using policy managers’ perceptions of effectiveness of integrated land and water management practices to achieve water sustainability appear to be an internally valid methodology. The variance of responses for specific practices was low. Most practices have results close to normal, while others had varying degrees of skewness towards higher and lower effectiveness. Only a few had a binomial distribution. The range of effectiveness across all practices ranged from very high to relatively low (Table 2). This all suggests that managers’ perceptions of effectiveness tend to be disbursed in a normal distribution but represent a thoughtful range of opinions.

Organizational and placed-based context does seem to influence managers’ perceptions of absolute effectiveness. On the organizational side, land managers generally rate the effectiveness of individual water-focused practices lower than water managers, and land managers rated some planning focused activities higher than water managers (Table 3). But despite these absolute differences in perceptions of effectiveness, land and water managers did generally agree on the relative effectiveness (Figure 1 and Table 8). These consistent patterns among manager types were not as distinct regarding the placed-based context of the census groups based on population size, wealth and city/county government. Significant differences in perceived effectiveness were found between the census groups with land and water managers from mixed wealth counties rating practice effectiveness the lowest, followed by small poor cities and then small prosperous cities (Table 4). There were also correlations between census groups in terms of the lowest and highest rated practices, though this relationship was not as strong as that between land and water managers (Figure 2 and Table 9).

These differences in context suggest that managers are considering the aspects of their work focus and the characteristics of their community when assessing the effectiveness of practices. They also seem to agree as to which are the lower and higher effective practices within their own context. This suggests periodic surveys to measure progress towards water sustainability over time using managers’ perspectives of effectiveness towards achieving water sustainability would be legitimate within each specific place and organizational context. But it also suggests that when reporting comparisons of effectiveness spatially or temporally, ranges that represent different contexts should be used.

These survey results also suggest that practitioners view the use of more objective indicators of water sustainability as salient (important) metrics to help them manage water sustainability. The variance in responses from managers for specific practices was generally low, even somewhat lower than for variance in the effectiveness of practices. This suggests a high level of agreement about importance of indicators. However, almost half of the indicators had large amounts of skew towards the ‘extremely important’ levels, which suggests that some indicators are of high importance (Table 5). Among these were ‘Balance supply–demand’, ‘Economic needs met’, ‘Quality-of-life met’, ‘Increasing water efficiency’ and ‘Water shortage’, which represent a core set of indicators highly valued for importance. Interestingly, two of these, ‘Economic needs met’ and ‘Quality-of-life met’, are less often part of the discussion about water and land management and do not currently have traditional or standard ways to measure them.

The contexts of organization and place also appear to be influencing perceptions of indictor importance. Just as with perceptions of effectiveness, water managers rated the importance of all indicators higher than land managers (Table 6). In this case, there also was a high correlation between water and land managers’ perceptions of high and low importance, with general agreement on which were more important than others (Figure 3 and Table 9). However, context of place, as represented by the census groups, had different results. In general, the pattern in perceptions of importance was similar to that of effectiveness perceptions, mixed wealth counties rated indicator importance the lowest, followed by small poor cities and then small prosperous cities. But in this case, there were some indicators that did not follow this pattern where importance was rated the same or importance was flipped with one group of practices higher and the other group lower.

These patterns of differences in perceptions of indicator importance based on organizational and place context suggest a few implications. The correlation between land and water managers’ perceptions of indicator importance suggests that within an organization, either a city or a county, some agreement can be reached on a single set of indicators used by water and land managers. However, the lack of correlation among the census groups suggests that this set of common organizational indicators might be different from one city to another, and one county to another. There are also some indicators that share a common vector of importance between census groups, which would indicate that there is an opportunity for a few indicators to provide a common comparison across different places.

Conclusions

Principle 12 from the OECD Principles on Water Governance calls for the evaluation of policies’ impact on water sustainability and its effect on change in the policies based on this evaluation. This will require some method of measuring the effectiveness of policies. Measuring water sustainability in a manner that is salient to practitioners is not straightforward because salience is heavily influenced by the context of place and organization. The organizational context was represented by identifying the manager type of each respondent (land and water managers) and place context was represented by census data (average household income, and population) for the place where each respondent worked and the type of government (city/county) for whom they worked. However, using perceptions of water sustainability is a valid approach to measuring sustainability in the current timeframe and useful for the reflection on current practices. Although this can be repeated over time to assess changes, perceptions of water sustainability are subject to some variance and are influenced by the context, such as characteristics of place and organization. If these change over time as well, this may introduce changes in perceptions. Any scheme to develop a standard method of assessing the effectiveness of practices of integrated land and water policy to impact water sustainability must do so with an understanding of the influences of context such as organization and place.

Water and land managers have strong perceptions of the importance of using specific indicators to assess levels of a community’s water sustainability in the present and over time. Managers’ perceptions of indicator importance also appear to be influenced by the context of organization and place. This seems strongest regarding place. Any scheme to develop and track indicators should recognize that different indicators will be salient to different places, though there are some that will be important to all places.

More research is needed to identify how other factors of context may influence perceptions of effectiveness and importance of indicators, such as water sources, landforms, climate, politics and institutional frameworks. Ultimately, we need research to connect perceptions of effectiveness and more objective indicators so we can assess exactly how and why effective practices do (or do not) contribute to the broader picture of water sustainability. It will only be through this evidence of evaluation that institutions will be able to effectively adjust the course of their policies.

Author contribution statement

The Land Wager Survey (LWS) project was initiated in 2019. RQ was the project manager for ASU and FS was the project manager for Lincoln. The project was conceived in a workshop with RQ, FS, RA, DW, JH, ZS, and ER2. Questions were initially developed by RA and finalized in a workshop with RQ, FS, RA, DW, JH, ZS, and ER2. ER2 and RQ developed the list of practices, RQ and ZS developed the list of indicators, AB and ER1 developed the secondary data. RQ, ER1 and AB developed the list of targeted survey cohorts and managed the survey deployment. RQ, ER1, and DW designed and conducted the statistical analysis. RQ, FS, RA, ER1, AB, DW, JH, ZS, and ER2 reviewed results. RQ and FS created the initial article draft which was reviewed by ER1, RA, AB, DW, JH, ZS, and ER2 before submission, RQ and FS managed the submission and peer and editor review process.

Geolocation

This research focused on the Colorado River Basin states of Arizona, California, Colorado, Nevada, New Mexico, and Wyoming within the United States of America.

Acknowledgments

This work was funded, in part, by the Babbitt Center for Land and Water Policy, Lincoln Institute of Land Policy and by the National Science Foundation SES-0951366.

The authors report there are no competing interests to declare.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Lincoln Institute of Land Policy [V10R010-VAS050720]; NSF [SES 0951366].