Governance for sustainable urban development: the double function of SDG indicators

ABSTRACT

At the core of the 2030 Agenda for Sustainable Development is the imperative of monitoring progress and holding policy-makers accountable. For this purpose, 17 Sustainable Development Goals (SDGs) and 230 global indicators were established with the double function of being a report card and a management tool. In the light of a pilot study about the experiences of local planning officials in the City of Gothenburg, Sweden, in relation to the suggested indicators for SDG 11, ‘Make cities and human settlements inclusive, safe, resilient and sustainable’, and a comparison with findings in four other cities, the paper argues for a need to reprioritize indicator criteria to serve better governance for sustainable development in diverse urban contexts worldwide.

可持续的城市发展治理：

可持续发展目标指标的双重功能. Area Development and Policy.《2030年可持续发展议程》的核心强调监测进展和让决策者肩负责任的重要性。为此，设立了具有作为工作报告和管理工具双重功能的17项可持续发展目标（SDGs）和230个全球指标。根据一项对哥德堡市当地规划官员关于目标11’建设包容、安全、有抵御灾害能力和可持续的城市和人类住区’相应指标的经验研究，以及与4个其他城市研究结果的对比，本文认为为更好地服务于世界不同城市背景下的可持续发展治理，需要对指标标准的优先顺序进行重新调整。

Gobernabilidad para el desarrollo urbano sostenible

la doble función de los indicadores de los ODS. Area Development and Policy. En el centro de la Agenda 2030 para el Desarrollo Sostenible está la exigencia de llevar un seguimiento de los avances y hacer que los responsables de formular las políticas rindan cuentas. Para este propósito, se establecieron 17 objetivos de desarrollo sostenible (ODS) y 230 indicadores globales con la doble función de que sirvieran como libreta de calificaciones y como herramienta de gestión. Teniendo en cuenta un estudio piloto sobre las experiencias de los encargados de la planificación local en la ciudad de Gotemburgo con relación a los indicadores sugeridos para el ODS 11, ‘Conseguir que las ciudades sean integradoras, seguras, resilientes y sostenibles’, y una comparación con los resultados de otras cuatro ciudades, en este artículo se argumenta la necesidad de reorganizar las prioridades de los criterios de los indicadores a fin de contribuir mejor a la gobernabilidad para el desarrollo sostenible en diversos contextos urbanos de todo el mundo.

Управление в интересах устойчивого развития городов: двойная функция показателей ЦУР. Area Development and Policy. В основе повестки в области устойчивого развития на период до 2030 года лежит императив мониторинга прогресса и обеспечения подотчетности директивных органов. С этой целью были сформулированы 17 Целей устойчивого развития (ЦУР) и 230 глобальных показателей с двойной функцией отчетности и инструмента управления. На основе экспериментального изучения опыта планирования на местном уровне в городе Гетеборг относительно показателя ЦУР 11 «Сделать города открытыми, безопасными, жизнестойкими и устойчивыми» и сравнения с выводами в четырех других городах, аргументируется необходимость переоценки критериев этого показателя для лучшего обслуживания системы управления устойчивым развитием в различных городах по всему миру.

KEYWORDS:

• Sustainable Development Goals (SDGs)
• 2030 Agenda
• indicators
• sustainable governance
• urban sustainability

可持续发展目标（SDG）；:

• 《2030年可持续发展议程》
• 指标
• 可持续治理
• 城市可持续发展

• Objetivos de Desarrollo Sostenible (ODS)
• Agenda 2030
• indicadores
• gobernabilidad sostenible
• sostenibilidad urbana

• Цели устойчивого развития (ЦУР)
• повестка на период до 2030 года
• индикаторы
• устойчивое управление
• устойчивость городов

INTRODUCTION

The 2030 Agenda for Sustainable Development, adopted by the United Nations General Assembly on 25 September 2015, aims to transform our world through an unprecedented ‘plan of action for people, planet and prosperity’ (UN, 2015). Never before has such a broad and universal declaration been agreed upon by world leaders. Seventeen Sustainable Development Goals (SDGs) and 169 targets have been formulated and incorporated into the agenda, and in March 2016, 230 global indicators to monitor the SDGs were agreed by the UN Statistical Commission (UN, 2016).

This set of 230 global indicators will serve the double function of being a report card, measuring progress and performance, and a management tool that aims to ‘help countries develop implementation strategies and allocate resources’ (SDSN, 2015, p. 2). From studies of the Millennium Development Goals (MDGs), the predecessors of the SDGs, we have learned that the indicators helped mobilize resources and political will, but that they often were less useful in demonstrating what was actually achieved (Fukuda-Parr & Greenstein, 2010; Welle, 2014). Used in a traditional performance management way, the indicators sometimes reduced complex issues into questions of one-dimensional measurement (Unterhalter, 2014; Yamin & Boulanger, 2014), reinforced silo effects and created perverse incentives (Fukuda-Parr & Yamin, 2013; Kabeer, 2015; Sen & Mukherjee, 2014; Unterhalter, 2014). In order for the SDG indicators to contribute to sustainability, the relationship between the report card and the management functions needs to be taken into account, as together they need to be able to handle complexity rather than reduce it, contribute to integration rather than fragmentation, and create incentives for innovation in order to grapple with future challenges rather than for picking low hanging fruit (Kemp, Parto, & Gibson, 2005; Loorbach, 2010).

From a rationalist perspective on indicators as tools of ‘informational managerialism’ in prevailing performance management systems, their function is to provide feedback on the performance of policies, strategies and implementation to allow for improvement (Mol, 2006, p. 506). In contrast, from a governmentality perspective, which pays attention to the governmental logics of policies and practices, that is, how certain aspects of behaviour have been problematized and made objects of government according to certain logics (Dean, 1999; Foucault, 1991), indicators are analyzed not simply as technical devices for improving policy and outcome but rather as normative systems for control, intervention, inclusion and exclusion. As such, indicators function as a technology of government through the ‘apparent objectivity and neutrality of numbers’ (Rose, 1996, p. 57) that allow for a comparison of performance over time and provide systems of accountability that shape legitimate ways of being and acting. More specifically, as technologies of performance, indicators are indirect means of regulating agencies by providing the conditions for calculation; targeting, implementation, monitoring and evaluation with the purpose of optimization of conduct, as they shape how actors make choices (Dean, 1999). In the case of the SDG indicators, their conditioning force is further strengthened by global comparability (Löwenheim, 2008; Neumann & Sending, 2010). Considering the variety of modes of governance and governance capacity in a plural world as well as the interaction between scales in shaping development, the main topics of Area Development and Policy (Dunford et al., 2016), it is essential to pay attention to how global indicators for sustainable development, as report cards and management tools, are shaped by and shape local governance. What the global indicators will come to imply in local and national contexts will depend on how the conditioning force of global indicators and comparability co-articulate with local and national politics and existing plans and programmes.

Taking the MDG lessons into account when designing indicator sets for sustainability would imply designing them not only so as to avoid silo effects, but also to help break down the silos. The MDG lessons have to some extent been taken into account in the development of criteria for the SDG indicators. However, this paper will show that promoting sustainable development – which was not an explicit objective of the MDGs – requires further critical reflection on how criteria that respond to both the report card and management tool functions co-articulate in ways that shape governance for sustainable development with effects and consequences for the appropriate design of global indicators. Governance for sustainable development here refers to how formal and informal arrangements involving a multiplicity of actors are structured so as to promote sustainable development. It includes how relationships between actors are facilitated and how decision-making takes shape (Holman, 2009; Kemp et al., 2005).

The arguments articulated in this paper stem from the results of a pilot study focusing on the relevance and measurability of the proposed targets and indicators for SDG 11, the innovative urban goal aiming to make cities inclusive, safe, resilient and sustainable, conducted in early 2015 by Mistra Urban Futures, an international urban sustainability research centre in Gothenburg, Sweden.¹

The study builds theoretical insights from a review of literature on the MDG experience and existing sustainable development indicators. These, in turn, inform an empirical study comprising reviews of policy documents and semi-structured interviews undertaken in 2015 with 34 public officials in relevant departments in the City of Gothenburg, and other local and regional actors.² The collection and qualitative analysis of the material focused on how the indicators fit into the local policy context, their usefulness and contribution to ongoing municipal monitoring and evaluation processes, whether they are possible to measure, what additional efforts would be required in order to measure them, and how they would shape policy and practice (for details, see Simon et al., 2016). By investigating the relevance of the proposed indicators in a specific context, as well as the extent to which they can actually be measured and what effects such measurement may have on policy and practice, we highlight the tension between the report card and the management tool functions. We examine how the report card function and the management function of specific suggested indicators for SDG 11 are articulated in existing policies and strategies as well as in the interview material, that is, how they are given contextual meaning as they are related to other ongoing processes. In so doing, we pay specific attention to how the suggested indicators are treated and understood in policy documents and in interviews with regard to the four lessons from the MDGs. The four lessons were identified in the literature review of the MDG experience as particularly important in terms of how they have shaped governance in ways that are not conducive to sustainability. Against the backdrop of a theoretical understanding of the governmental function of indicators, namely how the report card function and the management function co-articulate to shape policy and practice, two main indicator criteria emerge from the material as important for designing indicator sets conducive to governance for sustainable development. These criteria imply that indicators should be (1) innovative and aim to change behaviour; and (2) integrated into sets of mutually co-articulating indicator sets. This evidence further implies that other report card criteria such as measurability, practicability and comparability may need to be decentred in order to enable a prioritization of other criteria such as the ones suggested in this text.

The case of the city of Gothenburg is used as an example, not with the purpose of indicating what effects the SDG indicators will have in any given context, but to problematize the (still prevailing) instrumental view on global indicators as direct input to specific decisions (Rametsteiner, Pülzl, Aikan-Olsson, & Frederiksen, 2011). This also enables one to highlight the importance of understanding how the indicators co-articulate with local urban politics and planning, and to point out how a reprioritization of indicator criteria may increase the potential to create indicator sets that are conducive to governance for sustainable development rather than being used performatively with easy targets and indicators to maximize the impression of making progress. We should recall that Gothenburg is not representative of all cities worldwide because of its high institutional capacity and level of resourcing, although the specific lessons derived there may be relevant to a number of other cities.

The paper is structured as follows. The next section summarizes some of the lessons from the MDGs on how the indicators shaped agendas and outcomes. In the subsequent section on the SDGs as a management tool, we demonstrate why and how certain criteria need to be prioritized in order to shift from the performance-based MDG indicators towards indicators that contribute to governance for sustainable development. The governmental function of sustainability indicators is discussed and the tension between the report card and management functions highlighted, before a detailed exploration of this tension in the Gothenburg case study. Through the case study and comparative perspectives derived from the other cities in the study, namely Greater Manchester, Cape Town, Kisumu and Bengaluru (Bangalore), the paper suggests how indicators need to be designed in order to change behaviour in specific contexts, and how the indicators should be designed to co-articulate within indicator sets in order to serve as tools to achieve sustainability.

INDICATORS AS REPORT CARDS AND MANAGEMENT TOOLS: LESSONS FROM THE MDGs

The MDGs, aimed at galvanizing efforts to address extreme poverty, are considered to have been unique in their quantified targets as reflected in their simplicity, measurability, time-boundedness and focus on outcomes, as well as in how they communicated the urgency of priorities and mobilization of resources. There has been an abundance of both praise for and critique of the MDGs. On the one hand, they have been criticized for being top down, too ambitious, poorly specified, disregarding inequality and showing inconsistency between some goals, targets and indicators (Fukuda-Parr & Yamin, 2013; Langford, 2016; UN, 2013). On the other hand, there is still a general consensus regarding their positive effects on poverty alleviation globally (report card function), despite difficulties discerning the specific contributions of the MDGs (Miller-Dawkins, 2014). More specifically in terms of indicators as management tools, Manning (2009) suggests that the MDGs have redirected statistical capacity away from national priorities, while the UN declares that they have led to the improvement of statistical methodologies and information systems as a basis for decision-making (UN, 2013). Despite disagreement over the achievements of the MDGs, useful lessons from experience with them can be applicable to SDG indicator selection and local implementation.

According to Vandemoortele (2011a), one of their co-architects, the MDGs represent an effort to rescue the Millennium Declaration from oblivion. The subsequent goals and targets corresponded to those deemed politically feasible as well as measurable with as robust data as it is possible to produce (Vandemoortele, 2011a, p. 4). The quantified aspects of the MDGs – praised by some – are also a focus of critique because of the complexity and multifaceted nature of human development (Fukuda-Parr & Yamin, 2013). Vandemoortele (2011a), however, suggests that this critique is based on a misunderstanding of the goals, since they were formulated as tools for politicians, policy-makers and journalists to communicate the link between national priorities and the MDG agenda, rather than as technical tools and decision-making devices for development practitioners and civil servants.

Despite the intended purpose formulated by Vandemoortele (2011a, p. 8) and his call to use the goals in creative and communicative ways, the MDGs gained prominence as hard priorities with significant effects on development planning and the allocation of resources. They did so not necessarily due to their inherent qualities but because of contextual circumstances as they articulated with other processes at the time. These include the momentum of poverty-reduction strategies within the dominant development agenda, the prevailing emphasis on results and performance-based new public management (NPM), and strong support from key institutional actors (White & Black, 2004).

Drawing on this brief discussion, as well as a number of evaluations of the MDGs, four lessons stand out, namely: the possibilities and limitations of the report card function, the reductionism inherent in the goals, the risk of silo effects and the tendency to pervert incentives. These lessons echo much of Mol’s critique of ‘informational managerialism’ (Lehtonen, Sébastien, & Bauler, 2006, 2016).

First, the report card function of the MDGs helped mobilize resources and political will but – perhaps surprisingly – was less useful in demonstrating what was actually achieved (Fukuda-Parr & Greenstein, 2010; Welle, 2014). This stems in part from the specific baseline years selected for the various targets.³ In addition, statistical methods and quality evolved during the 15-year life of the MDGs. In particular cases,⁴ new statistical methods were debated politically as their implementation would reduce the perceived progress achieved and therefore reflect unfavourably on governments in power (Welle, 2014). Contextual circumstances such as population growth also had a significant impact on whether targets could be achieved with available resources, thus limiting comparability and accountability (Easterly, 2009; Hansson, 2014). Although the MDGs helped develop statistical capacity, the data gaps remain vast and the indicators are not remotely comparable internationally (Committee for Development Policy (CDP), 2015; Jerven, 2016; Vandemoortele, 2011b). Therefore, the experience with the MDGs shows that while the report card function is important for mobilization and as a tool for communication, its actual informational value may be limited and the information may even be misleading in cross-country comparisons.

Second, the simplicity of the MDG indicators was considered necessary for measurability, but has had the effect of oversimplifying complex agendas, sidelining other objectives and potentially informative variables. For example, the ‘Education for All’ agenda (World Education Forum (WEF), 2000) was reduced to MDG 2 on universal primary education (Unterhalter, 2014; Yamin & Boulanger, 2014). As a consequence of the MDG indicators being cast within a results-based management framework and placed at the centre of sectoral development programmes, they were invested with the power of informing the allocation of resources (Hansson, 2014). The strong conditioning force of benchmarking as a tool for accountability, in relation to both the population and donors, therefore had the effect of shaping evolving national agendas significantly in favour of simplicity rather than complexity (Maxwell, 2004, p. 30). As complexity tends to be sacrificed for the sake of measurability the simplicity criterion risks distorting agendas for change in a reductionist direction.

Third, the performance-based management structures have had silo effects as targets were dealt with individually within their respective sectors, with only limited cross-sector cooperation (Fukuda-Parr & Yamin, 2013). This was the case within national planning structures as well as within the development community, where different actors lobbied for their specific sectoral cause. The Paris Declaration on Effectiveness in Aid and the shift to sector-wide approaches (SWAPs) additionally resulted in bilateral donors focusing their efforts on specific sectors. Another reason is that slim organizations and results-based management create incentives for resource effective measures with little room for cross-sectoral work, which by default is resource consuming. As a consequence, the interlinkages among different targets and indicators have been insufficiently dealt with. For instance, MDG 3 has been criticized for having inadequately integrated gender equality with a range of other goals such as poverty reduction, education and health (Kabeer, 2015; Sen & Mukherjee, 2014). Breaking with the silo approach is therefore crucial for the 2030 Agenda to be successful.

Fourth, the MDGs have been criticized for causing perverse incentives, for example, to set the bar too low in terms of quality of service provided. A common example is the target for primary education, which set a minimalist target as a satisfactory standard (Fukuda-Parr & Yamin, 2013). Together with other targets, it has also been criticized for focusing on readily quantifiable variables, including education infrastructure such as classrooms, rather than quality of education and availability of certified teachers. A tendency to ‘pick the low hanging fruit’ has also been identified in certain cases, that is, to invest in areas in which it is easier to show quick and measurable progress than in those in which action is most urgently needed (Unterhalter, 2014) and which may be more costly or challenging.

The Sustainable Development Solutions Network’s (SDSN)⁵ indicator report emphasizes the importance of reflecting the lessons learned from the MDGs in designing the indicators and monitoring framework for the SDGs (SDSN, 2015). In 2016, the SDGs replaced the MDGs as the agenda for global development. Although sustainability was not a central theme of the MDGs, the lessons learned are important for understanding how the SDG indicators shape governance for sustainability. We argue below that the four lessons summarized above are particularly pertinent with regard to the SDGs because sustainability requires using indicators in decision-making that enables innovation, integration, participation and flexibility.

SUSTAINABILITY INDICATORS AS BOTH REPORT CARDS AND MANAGEMENT TOOLS: THE SDG CONTEXT

Evidence from the literature

The literature on sustainability indicators is vast. Lessons from the MDG context have already been surveyed above, so here we concentrate on their role in governance, that is, how they contribute to shaping decision-making arrangements. Holman (2009) argues that studies of sustainability indicators have focused either on the production of relevant and rational indicators or on their soft impacts, while there has been less focus on their governance function. One main theme in the literature concerns the primacy of ‘sound science’ (e.g., Holland, 1997; Jesinghaus, 1999; Levett, 1998; Pastille, 2002), complemented over time by an increasing engagement with the complexities of indicator systems (e.g., Bagliani, Galli, Niccolucci, & Marchetti, 2008; Mayer, 2008; Niemeijer & de Groot, 2008; Pulselli, Ciampalini, Leipert, & Tiezzi, 2008). For example, it has been suggested that systems emphasizing stand-alone indicators should be replaced by interacting indicator sets to address complexity better (Niemeijer & de Groot, 2008). A concern with ‘soft’ indicator impacts such as learning, integration, empowerment and capacity-building, and the proposal of participatory approaches to indicator development and use, is another theme (e.g., Becker, 2005; Bell & Morse, 2008; Fraser, Dougill, Mabee, & Reed, 2006; Gahin, Veleva, & Hart, 2003; Reed, Fraser, Morse, & Dougill, 2005). Recent literature has focused increasingly on the function of indicators in governance and policy processes (Hood, 2007; Rydin, 2007; Sébastien, Bauler, & Lehtonen, 2014), distinguishing between the use and influence of sustainability indicators in order to problematize the assumption that sustainability indicators automatically enhance sustainability (Lehtonen et al., 2016). For example, Holman (2009, p. 27) points at specific dynamics of governance tied to the use of sustainability indicators, such as their role in establishing communications between departments, communities and parties, as well as their function as a technology of visibility bringing certain aspects of sustainability to the fore.

Rydin (2007) has used a governmentality perspective to analyze how sustainability indicators in local governance have had responsibilizing effects (i.e., promoting self-governing behaviour), particularly in central–local relations, although her study shows that the role of agency in determining effects should not be underestimated. Other studies have shown how indicators have the effect of opening up dialogue and shaping networks (Alstleithner, Hamedinger, Holman, & Rydin, 2004; Journel, Duchene, Coanus, & Marinais, 2003), for example, as boundary objects (Lehtonen et al., 2016). Others again have highlighted governmental effects, how the use of indicators change the objects and methods of governing, namely, how mentalities of government shift into ‘management by numbers’ through targeting and ranking (Hood, 2007). While the above-mentioned studies show the effects of indicators on management, Sébastien et al. (2014, p. 335) found that the characteristics and quality of indicators were of limited importance relative to the characteristics of policy actors and the policy context in determining how indicators were used. Hence, they concluded that indicators must be used with awareness of their effects on governance, and be negotiated among policy actors, ‘explicitly addressing the politics inherent in the processes of indicator development, use and influence’, in a spirit of reflexive governance (p. 343).

In addition, studies have pointed to the underestimation of the complexity of information transfer, particularly as a result of difficulties understanding the indicator concept, that is, the different uses of indicators, and lack of distinction between the informative aspects of indicators and their policy use (Yli-Viikari, 2009). Gudmundsson (2003) distinguishes between instrumental use of indicators, that is, as information to shape design of policy, which is more frequent where changes are ‘relatively uncontroversial and small’, enlightenment use, that ‘help raise awareness’, and legitimization, where indicators are used to justify doing what is already planned. These distinctions are useful for understanding the risk of indicators having reductionist and fragmenting effects as well as producing perverse incentives, as discussed in relation to the MDGs above. Against this backdrop, we take up Holman’s (2009) call for increasing attention to how sustainability indicators ‘are able to facilitate relationships between actors and act as a catalyst around which various contested meanings of sustainability can be evaluated’ (p. 366), and how such insights can be foregrounded in indicator selection.

Design of the SDGs

Notwithstanding the insights provided above and the task, assigned to the Inter-Agency Expert Group on the SDG indicators (IAEG-SDG), of analyzing in depth the many interlinkages between indicators, there is a continued tendency to focus on producing the most informative indicator in terms of measurability and comparability (report card function). The statistical sciences tend to dominate the development of sustainability indicators, obscuring the politics involved in selecting and defining indicators and their effects on governance. Proposed indicator criteria for the SDGs have included both simplicity and efforts to grasp complexity. For example, ‘The Task Team on Lessons Learned from MDG Monitoring’⁶ defined criteria to guide the formulation of the SDG indicators focusing on relevance, methodological soundness, measurability and understandability (UN, 2013). The SDSN adds that robust Global Monitoring Indicators (GMIs) must be simple, universal, science based, forward-looking, focus on a single variable at a time, able to generate straightforward policy implications, allow for high-frequency monitoring and constitute a proxy for broader issues or conditions (SDSN, 2015). In addition, the MDG Inter-Agency Expert Group (IAEG-MDG) emphasizes coherence, consistency and complementarity, a limited number of indicators, focusing on outcomes and the need to complement the list by process indicators (UN, 2013). While these lists also include more complex criteria such as coherence and the need to be forward-looking, in the proposed indicator sets the simplicity and measurability criteria tend to dominate over the more complex ones (Rickels et al., 2016).

Drawing on the above-described insights into the governance effects of indicators, we explored how the proposed indicators for SDG 11 fit into the local policy context, their usefulness for both policy (report card function) and officials (management tool function) and what they contribute to ongoing processes, in order to highlight the tension between the report card and the management tool functions. From our analysis of interviews and policy documents which was undertaken with the lessons of the MDGs as well as the literature on the governmental function of sustainability indicators in mind, two types of indicator criteria emerged as particularly relevant for sustainability governance, namely, that indicators are innovative and designed to change behaviour, and that individual indicators are integrated in co-articulating indicator sets. The two types of criteria will be introduced briefly below before we turn to the Gothenburg context.

In order for indicators to be feasible, they need to be integrated into existing local planning practices, and methods to assess them need to be available (White, 2004, p. 48; Simon et al., 2016). As mentioned above, the SDSN (2015) suggests that indicators should be science based and constructed from well-established sources. However, there is a risk that these criteria and the factors that are considered important for instrumental use of indicators, such as institutionalization, consensus on data, that conceptual frameworks already exist, tend to have conservatizing effects (Lehtonen et al., 2016, p. 6). In particular, such criteria may encourage the selection of indicators that measure what we already know, which is partly a result of existing power structures that have not been conducive to governance for sustainable development (Fukuda-Parr & Yamin, 2013), rather than aiming for innovative ways to encourage sustainable development. Instead indicators can be designed in ways that require a change in decision-making behaviour by public authorities or changes in behaviour that directly affect sustainability, such as transport decisions. In the first case, the indicator can, for example, be formulated so as to require wider participation, innovation and integration in decision-making; in the second, it can be formulated so as to require interventions that contribute to structural and individual behavioural change that directly affect sustainability.

Furthermore, sustainable development is a ‘wicked problem’ in the sense that it is so complex ‘that traditional disciplinary and policy boundaries fail to deal with it. that it defies traditional disciplinary and policy boundaries’ (Loorbach, (Rittel & Webber, 1973). Sustainability governance thus requires integration across administrative sectors and dialogue among actors, particularly to avoid silo effects and perverse incentives. In contrast, the indicator criteria proposed by IAEG-MDG (UN, 2013) and the SDSN (2015) focused on individual and separate indicators that should be simple, single variable, measurable in a cost-effective way and have straightforward policy implications. In each specific sector, indicators were thus developed in order to provide the best possible and most reliable information to policy decision-making. This is partly the outcome of negotiations between sectors and interests over inclusion of specific issues in SDG targets and indicators. Nevertheless, it has been argued that significant efforts were made to create the SDG goals and targets as integrated and indivisible wholes that balance the three dimensions of sustainable development (Griggs et al., 2014; Le Blanc, 2015; UN, 2015). The IAEG-MDG also emphasized coherence and complementarity as indicator criteria (UN, 2013). In terms of the management function it is therefore particularly important that the indicators are designed in ways to support integrated decision-making in order not to counteract the purpose of integrated goals. As Griggs et al. (2014) argue, integration is needed in order to maximize synergies and manage trade-offs in the implementation of the SDGs. Failure to do so runs the risk of reproducing negative effects of the MDGs in the shape of incoherent policies, adverse impact on other sectors and outcomes that diverge from the intended goals (Le Blanc, 2015). Criticism of the lack of integration among the very large number of SDG indicators and the consequent lack of policy guidance have already emerged, while additional composite indicators have been proposed (Loewe & Rippin, 2015; Rickels et al., 2016).

In the following we analyze the suggested indicators for SDG 11 in the context of Gothenburg to see what they meant in a particular context, how they articulated with existing plans, goals and politics, and how they were aimed at changing behaviour and contributing to decision-making for sustainable development. We then situate these outcomes in a comparative context so as to consider the relevance of the findings to the diverse range of cities worldwide, since many face similar challenges, albeit in different combinations and with different degrees of relative importance.

SUSTAINABILITY INDICATORS IN CONTEXT: THE GOTHENBURG CASE

The Gothenburg case addressed in this section provides an interesting illustration of how little effort went into actually piloting the indicators and their relevance and usefulness before their universal acceptance. In Gothenburg, the public officials interviewed highlighted several reservations and concerns as to how the proposed SDG targets would be measured and what the function or relevance of the indicators should or could be in the light of existing sustainability policies and practices. These concerns should be understood in relation to the overall vagueness, at the time of the pilot study, about exactly how SDG 11 would be reported and what the role of local authorities might include. While the interviewees considered most targets appropriate and valuable, several indicators were deemed either irrelevant in relation to planning practices and processes within the City of Gothenburg or too complex to measure and report on annually. Some of the indicators were thought to constitute reporting burdens rather than actually measuring urban sustainability, while, in relation to others, the purpose of the indicators was questioned or simply considered too abstract or unclear. The discussion that follows will centre on two main indicator criteria: indicators as aimed at changing behaviour; and integrated indicator sets, as important for avoiding the pitfalls of the MDGs, namely producing data of little informational value, being reductionist, producing silo-effects and perverse incentives, and contributing to governance for sustainable development through integration, participation and innovation.

Indicators aimed at changing behaviour

In Gothenburg, the officials deemed several SDG 11 indicators as insufficiently promoting innovation and behavioural change, especially two indicators related to target 11.2 on public transit and 11.6 on waste management. The first part of indicator 11.2.1, ‘percentage of people living within 0.5 km of public transit [running at least every 20 minutes]’, is already an existing measure in the Gothenburg Urban Development Plan, on which the Environment Department has the responsibility to report. However, the frequency of 20 min is not included in their measure, and the officials were unsure whether such data could be produced. Our findings showed that the policy analysis authority, Trafikanalys, could produce such information but had not yet established relations with the municipal Traffic and Public Transport Authority (TPTA). The indicator is thus measurable but, as formulated at the time, it had little informational value and provided little incentive for further development.

Owing to the extensive work done with measures of accessibility to a range of service points in the Gothenburg region (Stadsbyggnadskontoret, 2009), the 11.2.1 indicator was considered of little relevance by city officials. They concluded that service frequency might add an interesting dimension but the timeframe needed to be clearly defined. In order to produce relevant and comparable information (in similar contexts), a definition of the relevant geographical unit would also be necessary. More importantly, the officials emphasized that the indicator measures the infrastructure, but not whether people actually use the public transport available. Consequently, as a single indicator, it is not considered useful for measuring behaviour or for planning purposes.

Two important findings emerged from the Gothenburg study. First, the officials at the TPTA stressed that the most important determinant of people’s decision to choose public transport in the Swedish context, is the car/public transport time ratio. A system with public transit within 0,5 km but with long travel times thus has little effect on changing behaviour. In this light the 11.2.1 indicator is overly reductionist and would thus not provide a useful tool to address current transport issues in the Gothenburg region. An indicator that measures the time ratio would be a better tool for traffic and overall development planning and may require a different perspective on spatial planning. Nevertheless, such a measure would be costly to develop and collect data for, and it would have little comparative value.

Second, TPTA officials emphasized how the indicator’s focus on 0.5 km distance to public transit did not indicate actual physical accessibility. Instead they suggested the need to include ‘real’ accessibility, that is, to measure the actual time taken to reach public transport. In Gothenburg, as elsewhere, the concept of walkability (Choi, 2012) has been discussed. It entails taking into account topography, physical obstacles and safety issues such as lighting and open paths. Although difficult to measure, it is considered a more useful tool for public transport planning, and to produce incentives for further development in ways that shape and change behaviour. The use of a walkability indicator would further require dialogue between officials across departments, to counter silo-effects, and dialogue with citizens to determine what walkability means in particular geographical contexts. Walkability is also almost impossible to compare globally as global comparison needs to take account of the cultural acceptability of walking for different social groups and also of the ways in which diverse built environments affect walkability and the possibility of changing that behaviour.

In terms of waste management, draft indicator 11.6.1, ‘percentage of urban solid waste regularly collected and recycled’, proved more complicated than at first thought since it involved two measures: the total amount of urban solid waste and the proportion collected and recycled. In Sweden, there are no local data on the total amount of waste generated. In cities with limited waste collection and recycling, such as Kisumu in Kenya, one of the other cities covered by this study, where only about 20–25% of solid waste is collected (Simon et al., 2016), the indicator may encourage improvement. However, where the waste system is more advanced, as is the case in Gothenburg, it adds little in terms of incentives or information. Gothenburg officials considered that the indicator had weak potential to change behaviour. The formulation of the indicator suggested that increasing recycling is the desirable behavioural change. One official at the City Executive Office (Stadsledningskontoret) argued that, depending on what counted as ‘regularly collected and recycled’, this indicator would most likely show that Sweden is already relatively successful at recycling and thus has less need to alter the behaviour of people, companies and authorities. She also argued that the indicator disregarded what type of recycling is used. For instance, 60% of Gothenburg’s waste is incinerated, and the ash used in road construction, which is the worst form of recycling. The indicator also disregarded the total amount of waste produced, hence losing the link with consumption. Accordingly, the indicator had little informational value and provided few incentives for increasing sustainability in the Gothenburg context.

The City of Gothenburg reports on this goal with the objective that waste generated per inhabitant should be less in 2020 than in 2008, in line with the European Union Waste Framework Directive (2008/98/EG). Additionally, the Swedish Waste Management Association has set two goals in their vision for 2020: to disconnect the increasing production of waste from economic growth; and to safeguard a strong movement up the waste hierarchy. The waste hierarchy prioritizes minimization of waste, followed by reuse, recycling, incineration and dumping, respectively (EU Waste Framework Directive (2008/98/EG)). By contrast, a relevant indicator aimed at changing behaviour in Gothenburg would indicate the position on the waste ladder. Such an indicator requires more resources and capacity to measure, which makes it problematic as a global indicator, and would simply not reflect priorities in Kisumu or even parts of Cape Town, among the other cities in our pilot study, where the SDG indicator remained pertinent and would be a challenge to meet. Still, this point clearly highlights the need for a discussion on how to create indicators which are science based, forward looking, policy relevant and aimed at changing behaviour in local contexts. However, doing this will greatly reduce the comparability of proposed indicators in the very different contexts worldwide. As this example shows, prioritizing measurability may redirect focus from the complexity of diverse local conditions and interpretations of sustainability.

Cost-effective measurability seemed to be the prioritized criterion for the two indicators discussed above. Indicator 11.2.1 was based on research that identified 0.5 km as the relevant maximum distance for a public transport stop, information that can be produced with a geographical information system. However, research indicates that other factors are significantly more important in post-industrial cities such as Gothenburg. Instead of being innovative in how the use of public transport can be increased, the indicator was limited to existing ways of measuring. A time ratio or walkability indicator would instead provide an opportunity for integrated planning between relevant authorities, negotiation and innovation to achieve sustainable transport. In the case of waste, the indicator did not take into account the generation of waste, nor the actual sustainability of different recycling patterns and was not promoting innovation in how it addressed unsustainable behaviour. In both cases, the bar was set too low for well-resourced and efficiently managed cities, thereby unintentionally providing incentives to lower the pace of change and even redirect the focus from potential technological leaps that aim to disconnect economic growth from waste production. A contradiction therefore emerged between measurability and comparability, on the one hand, and usefulness for planning and transformation that involves integration, negotiation and innovation and aims at changing behaviour, on the other.

In the next section, we suggest that not all indicators need to fulfil every criterion separately and that since indicators fulfil different roles within the whole indicator set, attention should focus on how they co-articulate and therefore how indicator sets can be designed as integrated wholes.

The need for integration

The findings from the Gothenburg study support the argument for the interdependence of indicators. The study illustrates that it is difficult to establish a straightforward relationship between simple, single-variable indicators and associated policy implications. Furthermore, the link between a single indicator and a more general and encompassing goal may be difficult to identify, even in poorer cities where basic needs are not yet universally met and urban services are not yet universally provided (Simon et al., 2016). Rickels et al. (2016) also demonstrate that composite indicators can complement single indicators to support the assessment process. Officials in the City of Gothenburg argued that several of the proposed SDG 11 indicators were not relevant as standalone indicators and that there was an overreliance on single indicators, as well as too narrow a focus on single-variable indicators. The Gothenburg study revealed how some indicators may contradict others, and we suggest that an analysis of how specific indicators co-articulate in specific contexts can be used productively to design integrated indicator sets that address certain hierarchies of power, for example. We will illustrate this through a discussion of four of the indicators analyzed, namely: 11.2.1, ‘Proportion of people living within 0.5 km from a public transit’; 11.3.1, ‘Ratio of land consumption rate to population growth rate’; 11.4.1, ‘Percentage of budget provided for maintaining cultural and natural heritage’; and 11.7.2, ‘Proportion of residents within 0.5 km of accessible green and public space’.

When viewed in isolation, some indicators may have unintended negative consequences. For instance, in relation to target 11.3 on land use and density, the officials at the Property Management Office called for an indicator that better reflects the nuances of density and land use. Formulated as ratio of land consumption rate to population growth rate at comparable scale, indicator 11.3.1 seemed to assume a unidirectional positive relationship between density and sustainability, something which has been increasingly contested (Bramley & Power, 2009; Dempsey, Brown, & Bramley, 2012). Officials also suggested that the exact way to calculate the ratio needs to be better defined for comparison across cities. Indeed, our pilot study revealed considerable differences in this respect even across the five cities examined (Simon et al., 2016). Yet, perhaps unsurprisingly, given their local remit, the respondents considered comparability less relevant than better ways of protecting and estimating the values produced by exploitation of land.

Neither the City of Gothenburg nor the regional authorities work with the proposed ratio. Instead they measure what is built within and outside zones with walking distance from public transport, in line with principles of transit-oriented development (TOD) (Dittmar & Ohland, 2004). This is in accordance with the regional strategy for long-term sustainable growth agreed by the Gothenburg Region Association of Local Authorities (GR) and aims at continued population growth while safeguarding the possibilities of sustainable expansion of the region, that is, inclusion of more municipalities in the Gothenburg labour market, through provision of better commuting possibilities, mainly in the form of public transport (GR, 2013). On the basis of the regional context and existing strategies, two conclusions were drawn. First, the officials at GR suggested that indicator 11.3.1 should be reported and monitored at a regional level. The target therefore required cooperation between authorities across the region. Second, they pointed to the importance of integrating indicator 11.3.1 with indicator 11.2.1, ‘Proportion of people living within 0.5 km from a public transit’. The purpose of this was to take into account intra-urban residential relocation and commuter patterns that tend to be unsustainable. Recent research demonstrates the importance of taking into account particular patterns of density and sustainability. For example, since the 1960s, intra-urban residential relocation in the Gothenburg region has involved an exodus of middle-class people aged between 30 and 40 years, and their families, from the inner city to surrounding municipalities (Jörnmark, Forsemalm, & Palmås, 2016). Therefore, a measure of density is only meaningful and useful in combination with type of land use, how it interacts with other values in the city and not least with public transport planning.

Indicator 11.3.1 on its own was thus not considered a valuable contribution to local or regional planning processes by the respondents. The density indicator was considered to even constitute a threat if treated in isolation, as it pays no attention to other values in the city, such as cultural heritage and public and green space (see also Waters, 2016). This is problematic for many reasons. For instance, officials at the Property Management Office argued that the indicator did not take into account the type of land consumed and it disregarded the crucial urban–rural continuum, which cannot easily be understood with a simple ratio. An excessively strong focus on density has also been criticized because it increases pressure on services as well as on land designated for other purposes, such as green space or protection of cultural heritage (Alexander & Tomalty, 2002).

Respondents suggested that a SDG density indicator might still be useful, but only when combined with other indicators that can protect softer values in relation to expanding development and densification interests. Municipal officials with responsibilities related to cultural heritage and public green space emphasized the importance of indicators 11.4.1 on cultural heritage and 11.7.1 on green space to safeguard such interests in planning processes. Only when included in integrated planning processes might these indicators constitute a strong and useful tool. Such integration of indicators relies on targets and indicators whose purposes are clear and well understood by the actors involved, something that is difficult to achieve if measured or accounted for separately by different local authority officials. This is similar to what SDSN (2015) emphasizes in their indicator criteria, that indicators must be conceived as interdependent and thus be pursued together, since progress in one target area may contradict progress in another.

Indicators for target 11.4 on cultural and natural heritage did not make much sense to the officials in Gothenburg as separate indicators, but they still acknowledged the value when combined with others. Although indicator 11.4.1, ‘Percentage of budget provided for maintaining cultural and natural heritage’, was complicated, if not impossible to measure, and indicator 11.4.2, ‘Percentage of urban area and percentage of historical/cultural sites accorded protected status’, was of little value for comparison across cities, officials still emphasized the importance of the indicators as tools for influencing city planning processes. This, the respondents argued, is the case particularly as protection of cultural heritage tends to be overruled by other priorities such as infrastructure development. Certain indicators thus play a more important role influencing integrated policy processes than informing policy in a specific sector (Gudmundsson, 2003). Therefore, if the agenda-setting function of the indicator fulfils the purpose of protecting certain ‘soft’ values, the criteria of measurability and global comparability can legitimately be given less weight.

Similarly, officials in Gothenburg’s Park and Nature Department considered indicator 11.7.2, ‘Proportion of residents within 0.5 km of accessible green and public space’, particularly important in planning processes that aim at densification and when conflicts of interest arise. The City of Gothenburg’s ‘Green Strategy’ provides that everyone should have access to a small park or green space within 300 m of their home (Göteborgs stad, 2014). In this example, a global indicator, as part of an integrated indicator set, would accordingly constitute an important support for protecting green city planning values. Similar issues emerged across all the cities in the pilot study, although mediated by differing circumstances and institutional arrangements, such as which authorities control different categories of green and public space, how accessibility is defined and controlled, and severe inequalities in the geographical distribution of, and accessibility to, green public space. This latter applies particularly in low- and lower middle-income cities such as Kisumu, Cape Town and Bangalore.

As illustrated, public officials in Gothenburg were concerned with the possible negative effects of some of the suggested indicators in isolation. A silo approach of this kind runs the risk of being reductionist and missing the broader objective of sustainable development. The findings from the study instead pointed to the importance of understanding how certain indicators are articulated in specific contexts and therefore could be integrated into complementary indicator sets that improve the possibility of planning and management contributing to sustainable development. As discussed above, integrated indicator sets can contribute to better integrated planning that assists attainment of the overarching goal of sustainability, including across goals that may be seen as contradictory when addressed separately.

CONCLUSIONS: RETHINKING THE DOUBLE FUNCTION OF INDICATORS

As indicated above, the MDGs, both as a whole and individually, were criticized for reducing the complexity of the poverty agenda and related debates about education and gender, for example, as they involved targets and indicators that are limited and isolated rather than integrated. Their application only to the Global South also raised the criticisms that they promoted donor-driven agendas and paternalism. Addressing sustainable development on a global scale through the 17 SDGs and the far larger set of targets and indicators inevitably poses even greater challenges when it comes to both complexity of the issue and what is required in terms of governance. Still, the indicators that were proposed for SDG 11 at the time of the Mistra Urban Futures pilot study seemed to prioritize measurability and practicability. The Gothenburg case study has helped demonstrate how public officials were critical of the usefulness of the specific indicators in their report card function, considering many of the indicators as constituting mainly a reporting burden, lacking clear purpose and unnecessary to measure on an annual basis. The officials were also sceptical of their usefulness as management tools, as they set the bar too low, were reductionist and did not necessarily constitute the best measure of progress nor the best tool for planning for sustainability which requires integration, participation and innovation.

Similar issues emerged from Greater Manchester among our study cities, but not in Kisumu, Cape Town or Bangalore. We suggest, therefore, that, several interlinked factors may explain the reactions of urban officials in the Global North. First, there was some surprise and even scepticism at the novelty of being subjected for the first time to global goals, targets and indicators alongside urban areas in the Global South. Second, and consequently, there is no experience of having had to measure and report on international templates in this way. Third, despite the intended global applicability of the SDGs, many of the targets and indicators – even in the new urban Goal 11 – bear the hallmarks of having been formulated with prevailing conditions and developmental objectives in the Global South in mind. Conversely, many local authorities in the Global South continue to express concern at the number and diversity of indicators on which they are expected to have or collect data, for which they lack resources and sometimes mandates in terms of their governance remits. As in the Global North, political will is variable, and the presence of SDG ‘champions’ among officials and elected representatives will probably prove crucial.

Under these circumstances, it is highly likely that urban areas in different countries worldwide will choose, or be directed by their national reporting agencies, to report only on those targets and indicators that fulfil the criteria of ease of measurement or collection, appropriateness, convenience and relevance to prevailing conditions and national and local development policies, priorities and programmes. A minimum core of ‘compulsory’ targets and indicators, augmented by differing dashboards of options, is highly likely to emerge. Indeed, some countries in the Global North may opt to use existing locally appropriate indicator sets rather than the SDGs. This is the case in the United States, as a result of which the SDSN has produced a modified index more suited to US conditions (SDSN, 2017).

At a more conceptual global level, learning from the Gothenburg case study and the experience of its public officials, this paper has highlighted certain indicator criteria. Specifically, it has argued that indicators should be aimed at changing behaviour, and that indicator sets should be regarded as integrated wholes. These findings suggest that indicator sets may need to compromise the measurability, comparability and practicability of individual indicators. Accordingly, it is important to rethink the double function of indicators as simple report cards and management tools. An excessive focus on practicability (report card function) thus runs the risk of overshadowing more ambitious management goals, while simultaneously preventing their use for real problem solving (Miller-Dawkins, 2014). For example, the literature on international agreements shows in a variety of cases that higher ambitions and less focus on enforcement may have greater effect on outcomes since they provide domestic groups with leverage to generate change (Hafner-Burton, Victor, & Lupu, 2012; Shaffer & Ginsburg, 2012). In this sense, greater possibilities for dialogue and experimentation facilitate better national problem-solving than ‘strict implementation’ as it contributes to building trust (Andrews, Pritchett, & Woolcock, 2012).

Negative effects of simple, single-variable indicators that rarely have a straightforward relationship with policy making, may also be counteracted through a conscious design of indicator sets as integrated wholes that safeguard complementarity between desirable outcomes. In the Gothenburg example this was indicated primarily by the imperative to counterweigh the density indicator to prevent negative effects on ecological and social sustainability, a risk that has been pointed out in previous studies (Bramley & Power, 2009). As indicators may contradict one another, they should be allowed and even be designed to articulate one with another. We therefore suggest that the criteria of global measurability and comparability should be decentred and made instrumental for the broader purpose of addressing sustainability as a wicked issue. Designing and implementing an indicator set as a co-articulating whole rather than confining them to pre-existing evidence would further allow more creativity when formulating indicators with a clear purpose of promoting certain types of behaviour or behavioural change. If the governmental logic of indicator sets is made explicit, and indicator sets are designed and implemented as a whole, they furthermore require dialogue between and participation of a range of actors in integrated planning.

DISCLOSURE STATEMENT

No potential conflict of interest was reported by the authors.

Notes

1. The pilot study compared results from five cities across three continents, namely: Bangalore, India; Cape Town, South Africa; Gothenburg, Sweden; Kisumu, Kenya; and Greater Manchester, UK – and the results from the pilot study were fed into the global Urban SDG Campaign. For results from the full pilot study, see Simon et al. (2016). The present paper discusses only the results from the Gothenburg study and does not take into consideration the other case studies.

2. The Gothenburg region includes 13 municipalities and has a population of almost 1 million, of which 60% lives in the city of Gothenburg. Interviews were conducted with officials in the city of Gothenburg, neighbouring municipalities, the Gothenburg Region Association of Local Authorities (GR), Västra Götalandsregionen (VGR), the Swedish Civil Contingencies Agency (MSB), The Swedish Environmental Protection Agency, The Swedish National Heritage Board, Statistics Sweden, Trafikanalys, the Authority for Cultural Analysis, the Swedish Waste Management Association and Västtrafik, the Gothenburg municipal property company HIGAB and the municipal insurance company Göta Lejon. At the time of the study, responsibility for the SDGs had not been allocated in the politico-administrative structure of the city or the region.

3. For example, the baseline year for Target 10, ‘to halve the proportion of the population who lack access to safe drinking water’, was set at 1990. In many countries, access to water declined in the 1990s after the completion of the International Decade for Water and Sanitation (1981–90) and as a result, the actual starting point in 2000 was far below the baseline (Easterly, 2009).

4. In Niger, for example, the establishment of a statistics department at the water ministry in 2007 involved an improvement of methods for calculating the coverage rate as a geographical coverage rate was replaced by an effective access rate, as preferred by UNICEF. The new method would show a much lower access rate than previously assessed, and therefore a sudden shift of method was resisted at the political level, not least since it would under-represent the progress that had been made since 2000.

5. The UN Sustainable Development Solutions Network (SDSN) operates under the auspices of the UN Secretary General to mobilize the global scientific and technological community to develop and promote practical solutions for sustainable development. The SDSN led a major consultative effort to produce a substantive report on indicators and monitoring for the SDGs (SDSN, 2015).

6. The Task Team is part of the UN Inter-Agency and Expert Group (IAEG) on MDG indicators.