Are We There Yet?

Abstract

Problem, research strategy, and findings

In 2020, Berke and Conroy demonstrated that plan policies did not promote sustainability principles strongly or in a balanced fashion. Has this changed in the intervening years? We conducted a longitudinal analysis of the policies in the updated versions of the plans in Berke and Conroy using their coding framework, comparing scores with the original plans. We found that most of the updated plans from the original study locations have an overarching sustainability vision, though this did not result in consistently higher sustainability scores. Although there were no significant differences in the strength with which sustainability principles were promoted in the updated plans, there were descriptive shifts in the promoted principles. In the updated plans, we found the same imbalance in principles promoted by present policies as in the original plans, making policy integration between principles a key consideration. The progress of policy adoption to promote sustainability principles may be slow; therefore, as more communities frame their comprehensive plans around sustainability, longitudinal studies are required to continue to track the effects of the progress.

Takeaway for practice

Sustainability scores for comprehensive plans have not significantly changed from 20 years ago, though more are focused on the concept of sustainability as an overarching vision. Evaluating scores over time provides insight into the nature of progress in supporting that vision.

Keywords:

• comprehensive plan
• content analysis
• longitudinal analysis
• sustainable development

More than 20 years ago, Berke and Conroy (2000) analyzed whether planning practices adhered to the United Nations Conference on Environment and Development Earth Summit directive for sustainable development. The study was the first attempt to measure how the concept was being used in the primary community planning document, the comprehensive plan, and has more than 1,000 academic and nonacademic citations. It received the Journal of the American Planning Association Article of the Year award in 2000 and has been incorporated into sustainability-related syllabi across graduate and undergraduate curriculum in planning, public policy, public health, and business in the United States and internationally, leaving a lasting mark as a seminal work for the profession at large. In the study, the authors established six sustainability principles (livable built environment, harmony with nature, place-based economy, equity, polluters pay, and responsible regionalism), and used them to evaluate the policies of 20 high-quality (not integrated) and 10 explicitly sustainability-focused (integrated) comprehensive plans implemented across the United States. They concluded the two sets of plans were not significantly different in how well they promoted the principles. Both sets of plans supported one principle (livable built environment) more than the others, establishing an unbalanced promotion of the concept in practice (Berke & Conroy, 2000).

Berke and Conroy (2000) evaluated the strength with which comprehensive plan policies promoted sustainability principles; this approach remains a key component of the plan evaluation literature (Berke & Godschalk, 2009). The literature in the intervening years has grown to capture communicative processes (Norton, 2008), detailed internal and external plan characteristics for best practices (Berke & Godschalk, 2009), and anticipatory governance (Berke et al., 2014). The target of plan evaluations has also expanded to include diverse topics such as affordable housing and climate change. However, since the Berke and Conroy publication, most works have evaluated plans from a singular point in time. There have been few studies that examined plans from multiple time frames (Brody, 2003a; Lukasiewicz et al., 2020; Stevens & Senbel, 2017), and none determined how plan sustainability has potentially changed over time despite continued attention to the concept.

We now add to the longitudinal plan evaluation literature by returning to the first evaluation of plan policy sustainability and ask: Do the policies of updated plans for each community from the Berke and Conroy (2000) study promote sustainability principles in a stronger and/or more balanced manner than their original counterparts? To address this question, we begin with a review of the literature focusing on essential considerations of sustainability in plan evaluation in the past decade and of longitudinal plan evaluation studies. We then describe the methods used in the original study. Finally, we return to the original 30 communities and comparatively evaluate how strongly updated plan policies promote the six sustainability principles. We make three primary conclusions: 1) more comprehensive plans are organized around sustainability as an overarching vision than there were in the original study; 2) there is no significant change in the strength of plan policies that promote sustainability principles in the updated plans, although there is a descriptive difference in the promoted principles; and 3) balance in the promotion of the principles via policies remains elusive.

Plan Sustainability Considerations

Berke and Conroy (2000) sought to measure how well policies in sustainability-focused and high-quality comprehensive plans promoted the concept of sustainable development. To do so, they established a literature-grounded definition of sustainable development based on a set of four key characteristics: reproduction, balance, linking local and global concerns, and dynamic processes. These characteristics support the proposed definition that “sustainable development is a dynamic process, in which communities anticipate and accommodate the needs of current and future generations in ways that reproduce and balance local social, economic, and ecological systems, and link local actions to global concerns” (Berke & Conroy, 2000, p. 23). The principles developed from this definition are presented in Figure 1 and were intended to bridge an abstract definition to actionable practice. Sustainability as a planning paradigm continues to evolve, though we contend the evolution has retained essential connections to the principles of harmony with nature, livable built environment, place-based economy, equity, polluters pay, and responsible regionalism.

Figure 1. Original principles from Berke and Conroy (2000), pp. 23–24.

To understand how the concept has transformed and what elements of a new and/or expanded planning paradigm must be considered in an updated evaluation, we used SCOPUS to search planning-related journals, limiting the focus areas to plan (covering planning, plans, etc.), sustain (covering sustainability, sustainable), and paradigm. Of the resulting 302 articles reviewed, we determined 57 studies published since 2010 focused on the theory and application of sustainability as a concept in planning. We distilled three themes from these studies that supported the continued relevance of the Berke and Conroy (2000) principles. First, although there have been updates in the past 2 decades, the concept itself has remained foundational as a touchstone for practice, relevant given its continued focus for plans and planning research, and aspirational in that it has continued to be a stated goal to work toward. In the early 2010s, for example, the APA established sustainability as the foundation of its “best practice standards” for comprehensive planning (Godschalk & Rouse, 2015, p. 10). Godschalk and Anderson’s (2012) Planning Advisory Service report was the product of the APA’s Sustaining Places Task Force, which was charged to “define the role of comprehensive planning in addressing the sustainability of human settlement” (Godschalk & Anderson, 2012, p. 1). Campbell and Zellner’s (2020) manuscript in the Vanderbilt Law Review examined planning for sustainable development as a 21st-century wicked problem, considering the relationship between complexity, urban systems, and the role of appropriate planning strategies that account for complexity. The authors asserted that, although it remained a contested ideal, “sustainability retains its hold at the center of the field” (Campbell & Zellner, 2020, p. 1654).

Second, most of the reviewed studies focused on applying the concept to key areas such as housing (Gilbert & Gurran, 2022; Sorrentino et al., 2014), land use regulations (Garde, 2004; Garde & Kim, 2017; Kim & Larsen, 2017; Mohammed et al., 2016), and infrastructure (Grant et al., 2018; Lindsey, 2003; Ramani & Zietsman, 2016; Sanders et al., 2015). These application approaches are often how plans and policies are asserted to be sustainable. However, these works have rarely examined sustainable development as an evolving concept, instead using it as a framework to structure their evaluations.

Third, there are studies that extended the concept of sustainability to issues and practices not fully mature in planning literature 20 years ago, including climate action/adaptation planning (Bassett & Shandas, 2010; Lyles et al., 2018; Mendez, 2015; Tang et al., 2009; Tozer, 2018) and resilience (Jun & Conroy, 2014; Meerow et al., 2016; Saunders & Becker, 2015; Yaman Galantini & Tezer, 2018). These are mutually reinforcing issues garnering increasing attention from the planning community because of climate change projections and the related physical and social impacts (De Jong et al., 2015; Holling, 2001). Urban resilience, for example, can be considered in terms of engineering as well as social–ecological systems (K. Davidson et al., 2019). Berke and Conroy’s (2000) reproduction characteristic acknowledged that both of these systems responded to dynamic change. In Jun and Conroy’s (2014) analysis of 46 Ohio township plans, the authors linked sustainability and resilience, arguing that preparedness frameworks such as community comprehensive plans provided ways in which communities can respond to exogenous shocks. Further, “a comprehensive plan that promotes sustainable development enhances redundancy and resourcefulness of the community, which are the mean states toward community resilience” (Jun & Conroy, 2014, p. 909). They supported the argument with a correlation between changed housing value (before and immediately after the housing crisis system shock) and plan sustainability scores, finding a positive relationship between scores and housing value change, as well as a lower standard deviation of housing value change in higher scoring plans. In addition, K. Davidson et al. (2019) conducted a comparative case study of the OneNYC and Plan Melbourne plans to “investigate the addition of social-ecological resilience to sustainability within the urban policy discourse of metropolitan plans” (p. 4). They asserted, “We are progressively observing resilience as a component of sustainability as the dominant organizing frame in the field of urban planning” (K. Davidson et al., 2019, p. 1).

Are the Original Principles Still Relevant?

Our focus on determining the continued relevance of Berke and Conroy’s (2000) principles for plan policy evaluation was grounded in the above review. Berke and Conroy’s sustainability principles were based on the characteristics of reproduction, balance, linking local to global concerns, and dynamic processes, where each retained a connection to the location, shape, scale, and quality of human settlements. The characteristics were intended to be broad, referring to clusters of qualities, yet “each principle had a common basis and may be measured in a common way” (Berke & Conroy, 2000, p. 23). These principles attempted to be foundational and were intended to specifically address considerations of human settlements, rather than provide a broad interpretation of sustainability or even associated processes and practices. Over the years, more sustainability-related principles have been proposed in the planning literature based on those of harmony with nature, livable built environment, equity, responsible regionalism, place-based economy, and polluter pays.

We found 9 of the 57 reviewed studies focused on the evolution of the concept of sustainable development, primarily through the addition, modification, or elimination of prior principles and characteristics. Principles noted in these studies are summarized in Table 1. They were assessed as either an essential core of the concept (foundational) or focused specifically on practice (operational), highlighting considerations ranging from philosophical notions of inclusiveness (Spiliotopoulou & Roseland, 2020) to practical proposals for lowering vehicle miles traveled (Talen, 2014).

Table 1. Summary of proposed sustainability principles in the literature.

	Operational principles
Foundational principles	Practice-based	Process-based
Provision^a	Provide more transportation choices^b	Authentic participation^c
Equity^a	Promote equitable, affordable housing^b	Support existing communities^b
Resilience^a	Enhance economic competitiveness^b	Value communities and neighborhoods^b
Biophysical limits^a	Coordinate and leverage federal policies and investment^b	Provide choices^b
Livable built environment^b,c	Design on a human scale^b	Think and act systemically^b
Harmony with nature^b,c	Encourage mixed-use development^b	Instill resiliency^b
Resilient economy^b,c	Preserve urban centers^b	Foster innovation^b
Interwoven equity^b,c	Vary transportation options^b	Redefine progress^b
Responsible regionalism^b,c	Build vibrant public spaces^b	Live within means^b
Healthy community^b,c	Create a neighborhood identity^b	Cultivate collaboration^b
Design matters^b	Protect environmental resources^b	Ensure equity^b
Quality of life^d	Conserve landscapes^b	Embrace diversity^b
Livability^e	Concentrate development and mix uses^b	Inspire leadership^b
Livelihood sufficiency and opportunity^f	Protect land and ecosystems^b	Continuously improve^b
Intra-generational and intergenerational equity^f	Use natural resources wisely^b	Advance equity^b
Precaution and adaptation^f	Expand housing opportunities^b	Make efficient decisions^b
Resource maintenance and efficiency^f	Provide transportation choice^b	Plan regionally^b
Equity, peace, justice, inclusiveness^g	Increase job and business opportunities^b	Systems-based decision making and implementation^g
Sustainability as dynamic system^g	Promote clean energy^b
Precautionary principle^g	Institutions^h
Polluters pay^g	Lower vehicle miles traveled^i Lower energy costs^i Limit damage to natural environments^i

Note: Bold indicates original/updated original foundational principle. Sources: a. Campbell and Zellner, 2020. b. Feiden and Hamin, 2011. c. Godschalk and Anderson, 2012. d. Fischer and Amekudzi, 2011. e. Gough, 2015. f. Roseland and Spiliotopoulou, 2016. g. Spiliotopoulou and Roseland, 2020. h. Foley et al., 2016. i. Talen, 2014.

Further review of sustainability principles divided operational principles into two subgroups focused on process and practice. Operational principles are descriptive elements of the foundational principles; that is, depending on context, foundational principles, such as equity or a livable built environment, would link to practice principles such as expanding housing opportunities. Therefore, we considered the foundational principles essential building blocks for the evolution of the concept.

There are 21 foundational principles listed in Table 1. A comparison of these principles with those of Berke and Conroy (2000) highlighted several overlaps and similarities. Godschalk and Anderson (2012) included five of the original six principles, dropping polluters pay, and updated both equity and place-based economy. Spiliotopoulou and Roseland’s (2020) “critical review of the theoretical roots, conceptual influences, major debates, limitations, and current trends in community and urban sustainability” (p. 7245) included the polluters pay principle. Harmony with nature is found in Campbell and Zellner’s (2020) provision and biophysical limits principles and Roseland and Spiliotopoulou’s (2016) precaution and adaptation, as well as resource maintenance and efficiency principles, because they “support the essential cycles and life support functions of ecosystems” (Berke & Conroy, 2000, p. 23).

A livable built environment is arguably one of the most noted sustainability principles in the literature. Fischer and Amekudzi (2011) examined quality of life in civil infrastructure decision making for sustainable development as expanding choices in three transportation infrastructure cases in the United States, Scandinavia, and France. Gough (2015) explored the “complementarity of livability and sustainability at a theoretical level” and applied it through “the analysis of comprehensive plans in fourteen jurisdictions in the Mississippi Gulf Coast” (p. 145). Both quality of life and notions of livability are essential for the ideas of community cohesion and a sense of place. Healthy communities is also a relevant updated principle encompassed by livable built environments and the principle of equity (Godschalk & Anderson, 2012), emphasizing activity, health care, justice, safety, and access to healthy food. Finally, that a livable built environment supports “protection of any special physical characteristics of urban forms that support community identity and attachment” (Berke & Conroy, 2000, p. 23) also captures the American Institute of Architects’ “design matters” principle, as indicated by Feiden and Hamin (2011).

A place-based economy captured Campbell and Zellner’s (2020) biophysical limits, because the “local economy should strive to operate within natural system limits” (Berke & Conroy, 2000, p. 23). Godschalk and Anderson’s (2012) update to incorporate resilience issues, noting potential positive and negative changes in community economic health, was less directly related to the definition of the original principle. However, because consideration of resilience has been integrated into the overall concept of sustainability, it is essentially present in the interpretations of all foundational principles.

Berke and Conroy’s (2000) principle of equity highlighted basic human dignity, environmental health, and access to resources. This captures the considerations of inter- and intragenerational equity (Roseland & Spiliotopoulou, 2016) and interwoven equity (Godschalk & Anderson, 2012). Although temporal considerations were not explicit in the principle, the definition upon which the principles were based included references to current and future generations. Spiliotopoulou and Roseland’s (2020) principle of equity, peace, justice, and inclusiveness was not fully captured by the principle of equity, mainly because of the consideration of peace. This consideration transcends comprehensive planning, and we acknowledge it is not addressed. Finally, although Berke and Conroy did not present a principle focused on sustainability as a dynamic system (Spiliotopoulou & Roseland, 2020), the dynamic nature of the concept is a key characteristic that supports its definition and the principles derived from the definition.

Our review supports the continued relevance and validity of the original principles. Although many additional and updated principles exist, those from Berke and Conroy (2000) have retained their connection to the foundations of planning human settlements and can capture the essence of the maturing considerations of adaptation and resilience. We contend the principles can continue to serve as a basis to evaluate the strength by which updated plan policies promote sustainable development.

The Case for a Longitudinal Approach to Plan Evaluation

Evaluations of comprehensive and other specialty plans have been a significant focus of the planning literature for decades. Fundamental plan quality components typically have focused on establishing a fact base, goals and objectives, and policies, strategies, and tools (Kaiser et al., 1995). Brody (2003b) enhanced those components with the inclusion of interorganizational coordination and capabilities and implementation. These have formed the foundation of plan evaluations focused on a range of diverse issues including climate change, natural hazards, ecosystem services, college campus planning, and affordable housing (e.g., Brody, 2003a; Gilbert & Gurran, 2022; Stevens & Senbel, 2017; White, 2014; Woodruff & BenDor, 2016). In addition to the common evaluative components, most plan evaluation studies have provided a cross-sectional assessment of plans. Cross-sectional analyses provide insight into the state of practice at a particular point in time. Such an approach does not provide an understanding of how communities have reacted to change over time or the shifts in priorities that have resulted from internal and external influences.

Longitudinal plan evaluation studies have given scholars a chance to “help identify factors that contribute to improving the quality of existing plans” (Stevens & Senbel, 2017, p. 2). The approach has provided practitioners with the opportunity to learn in an experimental fashion, assessing the impact of prior policies on existing outcomes (Brody, 2003a). However, there have been very few longitudinal plan evaluation studies. Brody (2003a) initiated the call for a longitudinal approach to assess adaptive management, looking at hazard mitigation components from 60 local comprehensive plans in Florida and Washington in 1999, compared with a 1991 study that evaluated those same plans. The intent was to “determine the extent to which their plans’ hazard mitigation components changed and to identify factors driving communities to adopt stronger policies” (Brody, 2003a, p. 191). The analysis highlighted improved scores in the later plans, indicating policy learning had taken place.

Stevens and Senbel’s (2017) analysis of climate change considerations in 39 municipal plans from British Columbia (Canada) was the next to take a longitudinal plan evaluation approach. The authors examined updated versions of plans that had been the focus of a 2014 evaluation, following the same plan evaluation protocol. They found that although “mean total plan score did not improve over time…[there were] significant improvements in both the fact base and implementation principles” (Stevens & Senbel, 2017, p. 6). The authors acknowledged that the short period of time between plan evaluations was a limitation given plans “were unlikely to have undergone more than one significant revision during that time period, which means that the amount of progress we detect in plans might be limited” (Stevens & Senbel, 2017, p. 6).

Our review found only one other longitudinal type of plan evaluation study, by Lukasiewicz et al. (2020), which focused on evaluating regional environmental plans in Queensland (Australia). The authors examined 30 natural resource management–related plans in three temporal sets across a 13-year period; some locations had plans in multiple sets, but it was not a requirement of the methodology. Evaluations were based on the time of a plan’s passage (its temporal set) rather than tracking individual plans across the three temporal sets. They found that “the quality of NRM [natural resource management] planning has fallen since it was introduced in the early 2000s” (Lukasiewicz et al., 2020, p. 13).

Our review of the plan evaluation literature found only three longitudinal studies since 2003 despite the potential for municipalities to use resulting insights for an adaptive management approach to comprehensive planning. The outcomes of the studies were mixed, showing both gains and losses in plan scores over time. The evaluation span of these studies ranged from 4 to 13 years. Major comprehensive plan updates may not be captured in those gaps given traditional practice guidelines of 7 to 10 years between major updates (Kaiser & Godschalk, 1995). The longitudinal studies addressed natural hazard, climate change, and resource management issues, which are important elements of planning for sustainable development but are more specialized than comprehensive. Our study addresses these gaps by revisiting the locations of the first plan evaluation study focused on sustainable development by Berke and Conroy (2000).

Measuring Plan Policy Sustainability

Given the continued relevance of Berke and Conroy’s (2000) sustainability principles and the dearth of longitudinal plan evaluations, we returned to the methodology established in the original study. The replication process consisted of two primary steps: plan collection and review and plan policy coding and analysis.

Plan Collection and Review

The original set of 30 comprehensive plans from Berke and Conroy (2000) included 10 plans organized with sustainable development as an overarching framework in the plan overview, introduction, and/or vision (integrated), with 20 additional plans identified as high quality (e.g., APA comprehensive plan award winners) that did not overtly set the concept as their organizing framework (not integrated). We retrieved plans from the community websites for those 30 communities for our updated review. Of the 30 original plans, one from each group had not been updated at the time of collection: Kansas City (MO; integrated) and Bethel (ME; not integrated). We excluded these two locations from further analysis.

The 10 original integrated plans were passed between 1994 and 1997; the nine updates were passed between 2011 and 2016, representing an average gap of 21 years. The original 20 not integrated plans were passed between 1985 and 1998; the 19 updated not integrated plans were passed between 2001 and 2017, representing an average gap of 20 years. The update gaps ranged from 10 years for Loudoun County (VA) to 33 years for Madison County (WI). These highlighted the potential for significant community changes in demographics, physical and economic conditions, and associated community priorities.

Community comprehensive plans are the statement of the physical vision of their future and a touchstone for planning practice (Kaiser & Godschalk, 1995). However, they are not the only local planning document that may provide insight into sustainability-related policies and initiatives in a community. Climate action plans, for example, may provide initiatives that link emission reductions to land use choices that support multiple principles of sustainable development. We reviewed the websites of the 28 communities for climate, resilience, and sustainability-related plans through keyword searches. Although we cannot be certain we found all potential sustainability-related plans, we believe those provided in Table 2 are primary documents. Some communities, such as Jacksonville (FL), integrated climate action strategies into their comprehensive plan per amendments to the state’s Growth Management Act in 2008. Others, such as Bucks County (PA), have a climate action plan in progress. Most of the adopted alternative sustainability plans were not based in the community’s planning department; they were more typically out of a mayor’s office. Such high-level buy-in can provide enthusiasm for sustainability initiatives, though it may also risk a disconnect with practice-based efforts connected to implementation departments such as planning. Therefore, although we chose to retain the focus on community comprehensive plans for this longitudinal analysis, we recognize the limitation that we may not have a complete picture of sustainable development activities for the communities.

Table 2. Alternate sustainability-oriented plans by community.

Community	Plan(s)	Year(s)	Program/office/department
Sustainable development integrated in plan
Burlington, VT	Climate action plan	2016	Sustainability Program/Mayor’s Office
	Legacy action plan	2000
Chattanooga, TN	Climate action plan	2023	Office of Sustainability
Lincoln, NE	Climate action plan	2021	Mayor’s Office
Livingston, MT	—	—	—
Portland, OR	Climate action plan	2015	Bureau of Planning & Sustainability
San Antonio, TX	Climate ready, action & adaptation plan	2019	Office of Sustainability
San Jose, CA	Climate Smart San Jose	2018	Environmental Services
Seattle, WA	Climate action plan	2013	Office of Sustainability & Environment
Teton County, WY	—	—	—
Sustainable development not integrated in plan
Anchorage, AK	Climate action plan	2019	Anchorage Assembly
Annapolis, MD	Sustainable Annapolis community action plan	2010	Department of Neighborhood and Environmental Programs
Arlington, TX	—	—	—
Bozeman, MT	Climate plan	2020	Sustainability Board/Strategic Services
Bucks County, PA	—	—	—
Champaign, IL	Growing Greener sustainability plan	2013	Sustainability Team (departmental representatives)
Charleston, SC	Climate action plan	2021	Mayor’s Office of Resilience and Sustainability
Cleveland, OH	Climate action plan	2018	Mayor’s Office of Sustainability
	Sustainable Cleveland municipal action plan	2013
Davis, CA	Climate action and adaptation plan	2023	Community Development & Sustainability
Georgetown, TX	—	—	—
Honey Brook, PA	—	—	—
Howard County, MD	—	—	—
Jacksonville, FL	—	—	—
Loudoun County, VA	—	—	—
Madison, WI	—	—	—
Nantucket, MA	—	—	—
Pittsburgh, PA	Climate action plan	2018	Sustainability & Resilience (Department of Planning)
Wilmington, DE	Resilient Wilmington plan	2020	Department of Public Works
Windsor, CT	—	—	—

Plan Policy Coding

The plan coding approach used by Berke and Conroy (2000) assessed plan policies based on how strongly they used development management techniques to promote the six sustainability principles. The policies were scored based on the strength with which they promoted a principle, scoring 1 if they used suggested language, such as may, or 2 if they used required language, such as must. For example, a policy stating that developers may receive a density bonus for offering a portion of their units at below market rates would score a 1, whereas a policy requiring developers to dedicate a portion of the units at affordable rates in return for increased densities would score a 2. Policies were classified according to their primary alignment with one of the six sustainability principles, categorized under one of seven plan elements (housing, transportation, environment, energy, urban design, economic development, and public facilities), and aligned with 1 of 27 development management techniques. If a specific combination of sustainability principle–plan element–development management technique had no associated policy, it was scored as 0; multiple policies supporting a specific combination only counted as the highest strength score (1 or 2). The highest scoring policy served as a specific combination score, which meant a perfect plan would have received a value of 2 for every sustainability principle–element–development management technique combination. Following the original Berke and Conroy protocol, we summed scores for each of the seven plan elements under each principle, and then standardized by dividing by the perfect score and multiplying by 10. This protocol favors the breadth of sustainability policies across principles, elements, and techniques rather than a comprehensive consideration of a particular principle, for example.

The original coding framework has since been updated to provide both policy clarifications in certain categories (e.g., setback buffers under land use regulations) and new categories from the plan evaluation literature: implementation and monitoring and fact base (Brody & Highfield, 2005; Jun & Conroy, 2013) Although these updates reflect an advance in the state of plan evaluation practice, we adhered to the original coding framework, as was the protocol in the Stevens and Senbel (2017) and Lukasiewicz et al. (2020) studies. Berke and Conroy (2000) recognized that “the focus on policies presented here represents a critical dimension of plans in that policies are the part of plans that guide day to day and long range decision making about land use and urban form” (p. 32). As with all plan evaluation studies, we recognize that progress in plan policies promoting sustainability does not guarantee progress in community sustainability outcomes.

We also wanted to capture the potential shortcomings faced by coders and consider the updated fit of policies to this framework. The original coding rubric, which focused on development management techniques, may not be the most effective measure for updated comprehensive plan policies. Kaiser and Godschalk (1995) discussed four distinct plan types: traditional land use design plans, policy plans, land classification plans, and development management plans. They recognized that “[t]he rationality of practice has integrated the useful parts of each of the separate prototypes reviewed here into contemporary hybrid plans that not only map and classify land use in both specific and general ways but also propose policies and management measures” (Kaiser & Godschalk, 1995, p. 375). They even typified this hybrid model with the earliest sustainability plan, Toward a Sustainable Seattle (1994). Though development management plans were dominant in the original study, the evolution of hybrid plans had already begun. The traditional land use focus of plans had given way to a broader consideration of the content appropriate for comprehensive plans. This shift led to an acknowledged limitation of the study in that we only measured what we set out to measure, and policies supportive of sustainability principles but outside the scope of development management (e.g., more procedural) were not captured. Therefore, while adhering to the original coding framework for comparability, we recognize it may be ill fitted to perform a truly accurate assessment of the sustainability strength of comprehensive plan policies.

Intercoder Reliability

Intercoder reliability provides a measure of the level of agreement between coders in content analysis studies. Reporting intercoder reliability scores is important to establish consistency and trust in the content analysis method and findings, though based on a review by Lyles and Stevens (2014), publication of intercoder reliability scores in plan evaluation studies is not prevalent. Percentage agreement score is the most common reliability measure in plan evaluation literature of studies reporting intercoder assessments. Another approach, measuring Krippendorff’s alpha, captures reliability assessment for more than two coders and accounts for potential chance agreements but has its own set of challenges in a plan coding context (Dyckman & Conroy, 2020).

We and two research assistants independently coded and compared three plans. We calculated the average pairwise percentage agreement scores (99% and 93%, respectively); the scores were well above the accepted 80% minimum agreement suggested in the literature (Miles & Huberman, 1994; Stevens et al., 2014). We then used the online tool ReCal (Freelon, 2010) to calculate Krippendorff’s alpha (α = .075 and α = .226). Stevens et al. (2014) suggested a α ≥ 0.70 minimum and α ≥ .58 for “tentative conclusions,” taking into account the “relative difficulty of evaluating plans” (p. 87). Our low alpha values were likely due to the number of 0 scores assigned by the coders (e.g., the lack of policies with matching principle–element–development management techniques within the comprehensive plan). We contend that the absences are meaningful to the research questions and should not be disregarded by this chance-corrected alpha score.

We Found That the More Things Change…

The principle scores (harmony with nature, livable built environment, place-based economy, equity, polluters pay, and responsible regionalism) for the updated comprehensive plans are shown in Table 3. Although overall variations exist, there was not a consistent higher or lower trend when compared with scores from the original integrated and not integrated sets. The bolded or italicized values show whether a score was higher or lower than the original, but according to t test comparisons of the principle-based scores for each subset and overall, no differences were statistically significant. The results imply that plan policies are not promoting sustainability principles any more strongly than they were 20 years ago. However, some descriptive observations were of interest in terms of principles and communities.

Table 3. Scores for promoting sustainable development principles by community.

Community	Harmony with nature	Livable built environment	Place-based economy	Equity	Polluters pay	Responsible regionalism	Total
Sustainable development integrated in plan
Burlington, VT	3.3	6.3	1.1	1.4	0.3	0.0	12.5
Chattanooga, TN	0.0	1.1	0.1	0.0	0.0	0.3	1.6
Lincoln, NE	2.4	11.6	0.7	0.9	0.0	0.0	15.7
Livingston, MT	0.0	2.5	0.5	0.0	0.0	0.7	3.8
Portland, OR	5.5	10.9	3.8	9.8	0.3	2.4	32.9
San Antonio, TX	3.3	5.5	1.8	5.0	0.0	3.5	19.2
San Jose, CA	10.9	20.3	6.3	2.7	0.3	0.7	41.4
Seattle, WA	5.7	10.1	5.0	4.6	0.3	0.9	26.8
Teton County, WY	2.4	5.7	0.7	0.0	0.5	0.0	9.4
Sustainable development not integrated in plan
Anchorage, AK	0.5	6.1	0.3	2.0	0.0	0.0	9.0
Annapolis, MD	1.3	4.2	1.3	0.3	0.0	0.5	7.7
Arlington, TX	0.3	4.2	1.3	0.1	0.0	0.0	6.1
Bozeman, MT	2.2	10.1	0.3	1.4	0.0	1.3	15.5
Bucks County, PA	4.2	11.6	2.5	1.8	1.3	1.1	22.7
Champaign, IL	0.0	2.7	0.0	0.5	0.0	0.0	3.3
Charleston, SC	2.5	5.5	0.0	0.0	0.5	0.3	9.0
Cleveland, OH	0.0	5.1	1.1	2.5	0.0	0.0	8.8
Davis, CA	4.6	23.7	5.3	10.9	3.5	1.3	49.4
Georgetown, TX	0.0	4.4	0.0	0.0	0.0	0.0	4.4
Honey Brook, PA	4.2	4.4	1.8	0.3	0.0	0.3	11.2
Howard Co., MD	2.2	6.4	2.5	3.8	0.1	1.1	16.4
Jacksonville, FL	9.4	27.9	5.9	7.4	5.5	2.5	58.8
Loudoun Co., VA	7.4	19.4	2.0	5.7	0.0	10.5	45.1
Madison, WI	1.8	3.8	1.3	3.3	0.0	0.3	10.7
Nantucket, MA	2.5	5.5	0.3	1.1	0.0	0.0	9.6
Pittsburgh, PA	0.1	0.1	0.1	1.1	0.0	0.0	1.6
Wilmington, NC	1.4	5.5	2.0	2.5	0.0	0.1	11.8
Windsor, CT	1.3	7.9	4.4	1.6	0.0	0.5	15.9

Notes: Values of .0 range from 0 to 1. Communities in bold are ones whose updated plans are in the alternate plan group from their original. Bold numbers indicate values higher than Berke and Conroy (2000) by >.1. Italic numbers indicate values lower than Berke and Conroy (2000) by >.1.

Among the integrated plans (organized with sustainable development as an overarching framework), compared with their original scores, the scores of Chattanooga (TN) and Teton County (WY) were lower or stagnant for all principles, suggesting those communities have possibly lessened their commitment to promoting sustainability principles either with fewer policies or with fewer required policies. The scores for San Antonio (TX) were higher or stagnant for all principles, suggesting more and/or stronger policies promoting sustainability in the updated plan. The remaining integrated plans portrayed a mix of higher and lower scores across the principles. Average differences by principle in this group compared with original plan scores highlighted the greatest gains in livable built environment and place-based economy, whereas polluters pay and responsible regionalism were the only two principles to lose points on average.

The not integrated plans (high-quality plans that did not overtly set sustainable development as an organizing framework) also presented a mix of score differences, although there were five consistently lower or stagnant scoring plans (Annapolis [MD], Arlington [TX], Champaign [IL], Georgetown [TX], and Nantucket [MA]) and two consistently higher or stagnant scoring plans (Bozeman [MT], Windsor [CT]). The largest gains in the not integrated group were from equity and responsible regionalism, whereas the greatest losses were from harmony with nature.

Overall, most plans (20 of 28) portrayed consistently lower scores for harmony with nature than those in the original study. This was especially true for the not integrated set of plans, in which only 4 of the 19 had updated principle scores higher than the original scores. It was the only principle that was on average lower scoring than its original counterpart. Consistently lower scores for harmony with nature were unexpected given the increasing attention to climate change and resilience in planning practice. The shift away from policies supporting harmony with nature may reflect the lower priority of environmental issues in the years in which these plans were passed. Environmentalism was the lowest public priority from 2008 through 2016; economy, jobs, and the deficit were all of higher priority (Pew Research Center, 2020). This may have been related to the Great Recession and strong economic concerns across the country. M. Davidson and Ward’s (2022) analysis of changes in municipal budgets and governance between 2006 and 2016 highlighted diverse local responses to the Great Recession that defy universal characterizations but did generally focus on economic issues over environmental or social governance considerations.

Although the livable built environment principle, on average, had the largest gains among all principles, equity presented the second largest increase overall and was higher for the not integrated group than for the integrated group. The increased attention paid to equity issues for not integrated plans may also be related to the timing of the updated plans. On average, updated not integrated plans were passed in 2011, as the United States was pulling out of the Great Recession; recovery indicators did not return to prerecession levels until 2014 (Bernanke, 2015). The Great Recession resulted in widespread loss of homes and economic security across the United States. Varied local responses included attention to affordable housing, community resilience, and economic development initiatives to help with recovery and to improve conditions of low-income communities and communities of color (Myers et al., 2021; Wolf-Powers, 2014). However, although communities may have adopted equity-based policies, attempts to revise accompanying social injustices were generally not effective due to conflicts between restoring norms, restoring markets, and reversing injustice that impeded “practitioners and policy-makers from delivering a greater measure of relief to affected neighborhoods and households” (Wolf-Powers, 2014, p. 214).

A final descriptive consideration in the updated plan scores was that fewer than half of the plans (11 of 28) received total scores higher than their original counterparts. The proportion of the increase in the total score was the largest for the integrated plans (4 of 9). Although the change was not significant, it supported continued attention to sustainability principles though the strength of the support may have waned. Lower scores may mean plans had more policy suggestions versus mandates or policies were focused on fewer areas of attention. This may provide evidence that intentionally focusing on the concept as an overarching theme increases principle scores more so than as a byproduct of “good planning” (Conroy & Berke, 2004, p. 1394). Another potential contribution to the increase in some scores is that, unlike the original plans, the updated plans were likely developed by planners exposed to the considerations of planning for sustainable development during their educational and professional preparation (Conroy & Berke, 2004).

Updated Plans and Updated Groups

Though comparing the original plans with the updated plans provides useful insights, we also reassessed the plan groups (integrated and not integrated) to see whether a community’s updated comprehensive plan would still fit the original categorization of integrated or not integrated. The communities in bold font in Table 3 indicate those that changed groups when plans were updated. Of the original set of integrated plans, one (Livingston [MT]) no longer set sustainability as an overarching theme. However, 14 of the original set of the not integrated plans would now be classified as integrated plans, leaving only 6 of the 28 updated plans in what had originally been the larger of the two groups. The shift to majority integrated plans reflects the acceptance of planning for sustainable development as an overarching goal or vision.

As with the original grouping, we ran t tests to compare the principle and overall scores of the new grouping of integrated and not integrated plans. There remained no statistically significant difference between the groups. However, an additional assessment of how the updated plans of each group compare across the principles by elements is shown in Table 4 and indicates more significant differences when the groups were realigned. This is especially true for the energy and economic development elements, where the scores were significantly different for the harmony with nature, livable built environment, place-based economy, and equity principles. Though the realigned groups present mixed results in terms of the significance of organizing a plan around a sustainable development framework on sustainability scores, it may indicate that a shift is taking place.

Table 4. Comparison of mean and total scores for promoting sustainable development principles by plan element.

Plan element^a	Harmony with nature	Livable built environment	Place-based economy	Equity	Polluters pay	Responsible regionalism
Sustainable development principles integrated (not integrated): Original groups
Housing	0.2 (0.1)	1.1 (1.0)	0.1 (0.1)	1.1 (1.0)	0.0 (0.0)	0.0 (0.1)
Transportation	0.4 (0.1)^b	1.3 (1.4)	0.3 (0.2)	0.4 (0.3)	0.0 (0.0)	0.2 (0.2)
Environment	1.3 (1.1)	1.6 (1.3)	0.2 (0.2)	0.2 (0.0)	0.0 (0.1)	0.0 (0.1)^c
Energy	0.3 (0.0)	0.3 (0.2)	0.1 (0.0)^c	0.0 (0.0)	0.0 (0.0)	0.0 (0.0)
Urban design	0.9 (0.7)	2.1 (2.1)	0.6 (0.4)	0.4 (0.3)	0.0 (0.1)	0.3 (0.2)
Economic development	0.1 (0.0)	0.6 (0.6)	0.6 (0.6)	0.1 (0.1)	0.0 (0.0)	0.0 (0.1)
Public facilities	0.3 (0.2)	1.0 (1.5)	0.2 (0.1)	0.3 (0.3)	0.0 (0.1)	0.2 (0.1)
Principle totals	3.7 (2.4)	8.2 (8.4)	2.2 (1.7)	2.7 (2.4)	0.2 (0.0)	0.9 (1.0)
Sustainable development principles integrated (not integrated): Updated groups
Housing	0.1 (0.1)	0.9 (1.4)	0.1 (0.0)^b	1.1 (0.7)	0.0 (0.0)	0.1 (0.0)
Transportation	0.2 (0.2)	1.4 (1.4)	0.2 (0.4)	0.3 (0.4)	0.0 (0.2)	0.2 (0.2)
Environment	1.2 (0.9)	1.4 (1.4)	0.2 (0.2)	0.1 (0.0)^c	0.1 (0.4)	0.1 (0.0)
Energy	0.1 (0.0)^b	0.3 (0.0)^b	0.0 (0.0)^c	0.0 (0.1)	0.0 (0.0)	0.0 (0.0)
Urban design	0.8 (0.5)	2.1 (2.0)	0.5 (0.3)	0.4 (0.3)	0.1 (0.5)	0.3 (0.2)
Economic development	0.1 (0.0)^b	0.6 (0.8)	0.7 (0.2)^b	0.2 (0.0)^b	0.0 (0.0)	0.1 (0.0)
Public facilities	0.2 (0.4)	1.4 (1.2)	0.1 (0.1)	0.4 (0.1)	0.0 (0.4)	0.1 (0.1)
Principle totals	3.0 (2.3)	8.3 (8.5)	2.0 (1.4)	2.7 (1.8)	0.3 (1.5)	1.1 (0.6)

Notes: Values of .0 range from 0 to 1. a. Among the integrated and not integrated groups: F-values are significantly different (p < .01) for energy, economic development, and public facilities. Comparison of mean scores of integrated and not integrated plans for each principle by each plan element and principle totals show t values that are significantly different. b. p < 0.05. c. p < 0.1.

Balance of Sustainability Principles

In previous studies, notions of balance have been touted as meaningful components of sustainable development (Berke, 2002; Campbell, 2016; Wheeler, 2013). These notions are connected to both resilience and adaptation in that lack of balance hinders a system’s ability to respond and adapt to shock (Holling, 2001). Although debates continue as to what constitutes balance from the standpoint of sustainability, the prevailing sense is that no one goal or characteristic of the concept should dominate the rest. Following the same balance assessment approach from Berke and Conroy (2000), in Table 5 we combined the two groups of plans to determine the highest-scoring principle and evaluated that score against the remaining scores as a consideration of balance. Berke and Conroy noted an unbalanced promotion of the principles in the plans, and this pattern continued 20 years later.

Table 5. Mean scores of highest-scoring principle versus remaining principles for updated plans.

Plan element: Highest-scoring principle	Highest-scoring principle	Remaining principles	t Test
Housing: Livable built environment	1.08 (1.09)	0.28 (0.72)	3.7^a
Transportation: Livable built environment	1.44 (1.16)	0.23 (0.39)	5.45^a
Environment: Livable built environment	1.42 (1.43)	0.37 (0.70)	3.81^a
Energy: Livable built environment	0.26 (0.36)	0.05 (0.18)	3.06^b
Urban design: Livable built environment	2.14 (1.48)	0.42 (0.66)	6.06^a
Economic development: Place-based economy	0.66 (0.73)	0.21 (0.45)	3.13^b
Public facilities: Livable built environment	1.37 (1.32)	0.22 (0.41)	4.59^a

Notes: a. p < .001. b. p < .05.

All seven plan elements (housing, transportation, environment, energy, urban design, economic development, and public facilities) significantly promoted a livable built environment more than any other principle. Livable built environment, which was also dominant in the original study, is aligned with considerations of livability (Fischer & Amekudzi, 2011) and quality of life (Gough, 2015). Therefore, the prevalence of the principle as a focus of plan policies is not surprising. Its significant promotion to the exclusion of other principles is problematic from a sustainability perspective, however, because it suggests that communities are missing opportunities for policies to support an integrative consideration of the concept. That is, a policy promoting new housing codes for diverse opportunities that, without further detail, would likely be categorized as a livable built environment–principled policy could consider language that would highlight ancillary uses for disadvantaged populations (equity) and perhaps align with subsidies for clustered housing that protects ecosystem service functions (harmony with nature). For example, Seattle 2035 policy LU 7.11 states

Permit, through Council or administrative conditional use approval, variations from established standards for planned large developments in single-family areas to promote high quality design that:

• is compatible with the character of the area,

• enhances and preserves natural features and functions,

• encourages the construction of affordable housing,

• allows for development and design flexibility, and

• protects environmentally critical areas.

Such developments should not be considered as sole evidence of changed circumstances to justify future rezones of the site or adjacent properties [City of Seattle, 2018, pp. 52–53; emphasis added].

This policy promotes the principles of livable built environment, equity, and harmony with nature through the housing element and permitted use development management technique. Although such multifaceted policies are not always possible, they highlight one approach to improving the balance among the principles.

Are We There Yet?

We took a longitudinal approach to assessing how comprehensive plan policies promoted principles of sustainable development in the same communities 20 years after Berke and Conroy evaluated them. Specifically, we asked: Do the updated plans for each original community promote sustainability principles in a stronger and/or more balanced manner than their original counterparts? Investigating the question led to three conclusions.

First, there has been a shift in the number of plans from the original study that now focus on sustainability as an overarching vision. This shift is notable given the plans were originally selected for being notable and high quality in some way. It may be an acknowledgment that initial steps are being taken to explicitly focus community plans on a sustainable future. Subscribing to that vision is a necessary condition for adopting policies that will support it. Since T. J. Kent’s 1964 book The Urban General Plan, planners have recognized the political context within which they operate. Without the support of a city council to adhere to sustainability as a goal, for example, it is unlikely there would be corresponding commitment to policy mandates or funding of associated projects.

Second, whereas the updated plans showed some variation regarding the strength with which they promote sustainability principles (compared with their original scores), there were no statistically significant differences in the scores. Neither set of updated plans had universally higher or lower scores than their original counterparts. Therefore, similar to prior longitudinal plan evaluation studies, the analysis shows any progress toward stronger plan sustainability scores is slow and not linear. This may be due to shifting attitudes influencing adoption of fewer required policies and/or more focused policy efforts on particular principles.

Third, the evaluation of elements, based on the highest-scoring principles versus the remaining ones, demonstrated that Berke and Conroy’s (2000) claim of imbalance between principles remained in the updated plans. The livable built environment principle was still statistically dominant among the plan elements. This imbalance suggests that most development policies under specific elements focus on enhancing considerations of community cohesion, access, and sense of place to the exclusion of the goals of other principles. A balanced plan would not need to equally promote the principles but would instead provide opportunities for policies to support multiple principles through integrative considerations.

Sustainability is a complex, interdependent concept encompassing diverse issues of climate, transportation, economic prosperity, affordability, and access; thus, it poses practical challenges for planners and planning. What is being done is often the focus of analysis, as in this work, rather than how it is being done. In 2016, Berke noted that “there is evidence that municipalities are not using the concept as an overarching paradigm…[highlighting a disconnect] between the academic/practice approach and its implementation” (Berke, 2016, p. 382), though scholars have called for planners to integrate diverse perspectives and specialties (Godschalk & Anderson, 2012). We have found that updated plans are now dominated by those that integrate sustainability, though they remain unbalanced in how policies promote sustainability principles and their scores still are not significantly different than other high-quality plans. There is progress on adopting the vision of sustainability but not as much on implementing strong supportive policies in the comprehensive plan. That is what remains the enigma for practice. Some changes create policies that bring communities closer to a sustainability ideal (evidenced by higher scores), whereas others do not. Adaptiveness remains a critical consideration as communities respond to challenges, whether pandemics or droughts, and attempt to draft a better future. Therefore, is planning for sustainable development a case in which the more things change, the more they stay the same? Perhaps. Returning to evaluate plans—not just once but at regular intervals—may provide a key to understanding the nature of the changes.

Additional information

Notes on contributors

Maria Manta Conroy

MARIA MANTA CONROY ([email protected]) is an associate professor of city and regional planning at The Ohio State University.

Jessica Pagan Wilson

JESSICA PAGAN WILSON ([email protected]) is a researcher specializing in sustainability and water issues.