Are local climate adaptation policies credible? A conceptual and operational assessment framework

ABSTRACT

After the Paris Agreement that put stronger emphasis on the development of climate change adaptation policies and on the definition of financing mechanisms, there is a patent need to track whether actual planning efforts are proving sufficient. This entails the development of assessment methods and metrics as plans are drafted and actions implemented. To this end, this paper explores the concept of credibility as a critical issue in climate policy and develops an Adaptation Policy Credibility (APC) conceptual and operational assessment framework for helping to allocate public funding and private investments, and for implementing and catalysing climate policy. Through a pilot testing in four early-adopting cities (Copenhagen, Durban, Quito and Vancouver), a clear potential for large-n tracking and assessment exercises of local climate adaptation plans is envisaged. The APC approach might also be useful to guide individual cities that aim to improve their adaptation planning and policy-making processes.

KEYWORDS:

• Climate adaptation policy
• local climate plans
• credibility
• adaptation tracking
• adaptation metrics

Introduction

Planning for adaptation to climate change has emerged as a central component of climate policy over the last decade (Moss et al. 2013; Preston et al. 2015). The Paris Agreement set an ambitious pathway for adaptation that urges nations, regions and cities to act on climate change impacts together with other public and private stakeholders (Lesnikowski et al. 2017).

As interest in adaptation increases―including investment in adaptation policies, programmes, and actions―there is a need to track whether these are effective in reducing vulnerability and building resilience (Bours et al. 2015; Ford et al. 2015; Haasnoot et al. 2018). This requires new methods, tools and frameworks to measure the actual progress on adaptation (Ford et al. 2015; Chen et al. 2016; Magnan 2016; Magnan and Ribera 2016). An ability to track adaptation progress is highly relevant for governments at different scales looking to prioritise their investments to effectively adapt and to define how to access climate adaptation funding, especially in developing countries (Araos et al. 2015, 2016; Ford and Berrang-Ford 2015; Lwasa 2015; Sud et al. 2015). Likewise, it is fundamental to reduce policy uncertainties and provide the right guidance to the private sector, such as robust criteria to define their funding strategies (Tribbia and Moser 2008).

Efforts made to date, however, are far from being comparable to those tracking mitigation progress, and have a number of weaknesses, including the lack of consistent definitions and practices, agreed metrics, comparable baselines, standardised approaches to data collection and robust guidance (JoAnn Carmin et al. 2012; Dupuis and Biesbroek 2013; Reckien et al. 2014; Ford et al. 2015; Araos et al. 2016; Heidrich et al. 2016; Magnan 2016; Biesbroek et al. 2018). In current adaptation tracking practice, the existence and nature of climate change adaptation policies¹ has been used as an indicator of progress. Relevant studies have documented, examined and compared adaptation policies at national (Berrang-Ford et al. 2014; Austin et al. 2016; Lesnikowski et al. 2016; Pietrapertosa et al. 2017), city (Olazabal et al. 2014; Araos et al. 2016; Woodruff and Stults 2016; Reckien et al. 2018) and community scales (Pearce et al. 2011).

Nevertheless, the development of policies on its own, while indicative of adaptation policy progress, may not actually lead to vulnerability or risk reduction. There is evidence, for instance, that local authorities have often used climate mitigation policies to recodify existing strategies without having substantial impact on emissions reduction (Millard-Ball 2013). In the context of climate adaptation strategies, an ambiguous use of the concept of adaptation in policy making may also end in a general lack of concrete measures and implementation (Dupuis and Biesbroek 2013).

This paper presents a conceptual framework to assess the credibility of climate change adaptation policies, a metrics-based assessment model at the local scale, and a pilot test of the model in four cities: Copenhagen (Denmark), Durban (South Africa), Quito (Ecuador) and Vancouver (Canada). The credibility of climate adaptation policies is here defined as the likelihood that such policies will be effective in reducing or avoiding the impacts of climate change and sustained in the long-term or, at least, for the term for which they have been originally defined by responsible parties. The pilot test is used to draw conclusions on the usability of the framework and the potential and interest of a large-n experiment.

The paper is structured as follows: the next section explains the methods and data used. This is followed by a review of the main literature describing the concept of credibility in climate adaptation policy making and influencing factors. Based on this, a conceptual approach for assessing the credibility is developed and operationalised for local adaptation policies. Subsequently, the results from a pilot application of the approach in the aforementioned cities are presented. The paper finishes by reflecting on the proposed framework and its potential application.

Data and methods

The Adaptation Policy Credibility (APC) framework is developed based on a review of the literature identifying key aspects of climate adaptation planning process. The review is summarised in the next section. Likewise, indicators and metrics used to operationalise the conceptual framework have been chosen based on a review of the literature relevant to each of the components of the conceptual framework.

As the APC is aimed to be applied in large-n experiments, metrics need to be evaluated. In this paper, scoring is proposed for evaluation and comparison purposes because it is an easy-to-apply method that allows the creation of a composite credibility index (see, e.g. Preston et al. 2011; Heidrich et al. 2013; Araos et al. 2016; Lesnikowski et al. 2016; Woodruff and Stults 2016). Combining metrics might be challenging when it comes to adaptation assessment due to a lack of knowledge about the complex interactions between the components assessed (Ford and King 2015). Arguably, a composite index can here be considered a first step towards the recognition of the need to compare contexts and transfer knowledge across policies and, in this case, cities.

The conceptual and operational framework is tested using a set of local adaptation policies in four cities: Copenhagen, Durban, Quito and Vancouver. These cities have been selected because they provide examples from both developing and developed countries and they are internationally recognised for their early action on adaptation and, therefore, their efforts are well documented.

The adaptation policies are analysed as stand-alone documents motivated by the need to protect urban populations, infrastructures and other urban assets from current climate change impacts and to build adaptive capacity for future impacts. In building this framework, two choices for data collection were considered: (1) self-reporting (by city officials) based on policy documents and self-knowledge on the policy processes of the city (see, e.g. Campos et al. 2017) or (2) a systematic review process by expert analysts (see, e.g. Araos et al. 2016; Reckien et al. 2018). Although a self-reported assessment could lead to context-specific and detailed data of non-publicly available information, a systematic review process offers objective and comparable data on released and publicly available information, features that, according to these authors, provide more reliability to public policies. For this reason, the second option was chosen and a systematic review process for the selected cities was performed: each city’s municipal website was identified and scanned for climate change adaptation planning documents.

Finally, in the pilot test, the scoring procedure merits mention because of its influence on the reliability of the outcomes (Lyles and Stevens 2014). To provide a higher level of objectivity, all metrics have been coded independently by two different analysts with expertise in climate adaptation policy document analysis. The two analysts have compared scores, identified elements of disagreement and reconciled scores based on the new information considered. After this process, no conflicting scores have been found. Details on the scoring procedure and evaluation process of each of the defined metrics are included in the subsection ‘Operational framework at a local scale’, as part of the methodological development of this paper.

Credibility in adaptation policy making: a review

The concept of credibility in climate policy

For a piece of information to be credible, it should be able to ‘be believed in, justifying confidence’ (OED 2013). Credibility is a concept widely used in the policy sciences (see Drazen and Masson 1994), mainly referring to problems related to regulatory policies (Helm et al. 2003). Specifically, a large body of literature dealing with credibility is framed in the development of monetary policies (see, e.g. GomezPuig and Montalvo 1997). This literature illustrates that the credibility of a plan or policy depends not only on the plan or policy itself, but also on the context and conditions in which it is developed and implemented and on the motivations and incentives of the authorities responsible (North 1993; Drazen and Masson 1994; Helm et al. 2003). For example, Helm et al. (2003) argued that the UK’s carbon policy was not credible because it relied on the price of technology being low and decreasing over time. This involves a credibility problem around the ‘time inconsistency’ of the decision-making sequence where private-sector agents make irreversible decisions before policy makers act.

Credibility has been recognised to be an important issue in environmental and climate policy, primarily in the context of the role of science and experts in delivering ‘usable knowledge’ (Anderegg et al. 2010; Lemos et al. 2012; Ford et al. 2013b; Heink et al. 2015) but also in relation to regulations (Helm et al. 2003) or commitments (Averchenkova and Bassi 2016). With exceptions (e.g. Averchenkova and Bassi 2016 concerning the mitigation commitments under the Paris Agreement), empirical studies dealing with credibility (or any similar attribute) in the context of climate policy are scarce, especially regarding adaptation (Dupuis and Biesbroek 2013). Averchenkova and Bassi (2016) state that credibility is essential to generate the necessary flow of climate finance from private and public sectors, at different levels of governance (national, regional and local). Credibility in the context of mitigation, they maintain, ‘is vital for building trust among negotiating parties, as this will help to increase the ambition of pledges over time’ (p. 39). They identify four key determinants for policy credibility: coherent and transparent rules and procedures, dedicated and supportive players and organisations, history of norms and public opinion, and past performance.

The concept of ‘policy credibility’, as seen in the work of Averchenkova and Bassi (2016), has not been developed in and adapted to the context of adaptation. This may reflect adaptation not yet being paid sufficient attention in public negotiations across scales, due to it being less clearly recognised as a global public good, and hence the goal of building trust not yet being given much weight (Magnan 2016). Nevertheless, given the need to inform policy making, investment, and funding strategies on adaptation, and the reliance of many adaptation tracking studies on the (in)existence of adaptation policies and plans to measure adaptation progress, developing an approach to assess credibility seems an important research step in the adaptation tracking field.

Factors influencing credibility in current adaptation planning literature

Diverse approaches have been proposed in the literature to assess the abilities that a system may (or may not) have to plan for adaptation or the factors that may prevent adaptation action. The assessment of adaptive capacity (Engle 2011), for example, provides information on which factors help to build capacity to face future climate impacts. This information is key to assess whether the system has sufficient resources to adapt in terms of institutional structure, knowledge on management systems, infrastructure and technology, past experience, etc.

Other approaches focus on the assessment of barriers to adaptation, which are critical to identify potential deviations in the adaptation process. Barriers to adaptation have been theoretically discussed (Adger et al. 2009; Moser and Ekstrom 2010; Biesbroek et al. 2013; Ford and King 2015; Huitema et al. 2016), and empirically examined (Measham et al. 2011; see, e.g. Bierbaum et al. 2012; Reckien et al. 2015; Tilleard and Ford 2016; Nordgren et al. 2016). Barriers may include: lack of knowledge, uncertainty about impacts, the extended time periods involved, lack of leadership, lack of financial resources, institutional constraints (e.g. rigidity, lack of competencies), limited stakeholder engagement and participation, poor decision-making culture (not iterative or flexible), lack of public support, divergent risk perceptions and cultural attachments. Based on these findings, credible climate change adaptation policies should inspire confidence to overcome social, technical, economic and political adaptation barriers, and thereby engage better with stakeholders (e.g. funding agents, private investors) for effective action. In this sense, legitimacy has been identified as an important pillar of (perceived) successful adaptation (Adger et al. 2005), and refers to the consideration of equity and justice in policy-making and scientific processes. It includes the engagement of stakeholders and civil society in the development of the plan and the transparency of processes and information.

The assessment of adaptive capacities and barriers to adaptation both require consideration of the factors that enable or prevent current or future adaptive processes; however, these approaches do not provide information on how such processes should be built. Building on this gap, the concept of adaptation readiness was proposed to examine the adaptation process by considering what is actually being done to prepare for adaptation (Ford and King 2015; Tilleard and Ford 2016).

Other approaches related to plan evaluation have been proposed to assess how well a plan is aligned with adaptation outputs and outcomes. For example, Preston et al. (2011) argued that plan evaluation provides transparency and formal definitions of criteria and methods that can be useful for accountability in an evidence-based policy context. As part of plan evaluation research, a number of recent studies have assessed the quality of climate change mitigation and adaptation plans at the local level (Baynham and Stevens 2014; Woodruff and Stults 2016). Plan quality assessments are generally performed in order to identify aspects of plans that are considered important to achieve the objectives pursued as well as aspects that should be improved (Stevens 2013). They are more detailed than other approaches (e.g. Preston et al. 2011) and allow plans to be compared across domains (Woodruff and Stults 2016). Baynham and Stevens (2014) argued however, that the hypothesis that plan quality standards correlated with a reduction in greenhouse gas emissions and better preparedness for climate change impacts had not been evidenced so far and proved more complex for adaptation than for mitigation. Plan quality assessment should therefore not be the only aspect on which the concept of credibility relies.

In fact, as a result of the long-term nature of many adaptation strategies, the problem of credibility in adaptation policy is also tightly linked to whether a policy is intentional and substantial (Dupuis and Biesbroek 2013); that is, to whether climate change impacts originated the need for policy development and to the level of contribution of such policy to problem resolution. Nevertheless, it may be difficult to establish valid methods for measuring the outcomes of adaptation policies in a similar way to that used for mitigation policies (see, e.g. Millard-Ball 2012) as many of the impacts of climate change will be felt in the long term and therefore are not easy to measure or estimate (Ford et al. 2013a). Consequently, a focus on measuring process aspects of adaptation policy has emerged to facilitate tracking exercises (Dupuis and Biesbroek 2013).

Development of an APC framework

Conceptual framework

Building upon the review of approaches set out in the previous section, herein, the main aspects of credibility are identified and an APC framework is proposed based on: policy credibility, as defined by Averchenkova and Bassi (2016), plan evaluation (as in e.g. Preston et al. 2011; Baynham and Stevens 2014), adaptive capacity and readiness (as in Ford and King 2015), whether the plan is intentional and substantial (as in Dupuis and Biesbroek 2013) and the legitimacy of the process (as in Adger et al. 2005). A framework to assess APC should therefore look at the institutional and policy context in which the policy was developed, the resources dedicated to its creation and maintenance, the knowledge used for the decision-making process and the level of engagement with the stakeholders and the public.

Herein, this study proposes a framework that links the aforementioned aspects around a structure of three major areas: Policy and Economic Credibility, Scientific and Technical Credibility, and Legitimacy, which is common to the first two (see Figure 1). Policy and Economic Credibility is divided into three components: Resources, Reliability, and Institutional, Public and Private Support. ‘Resources’ refer to the means required for the implementation of the plan and ‘Reliability’ to past performance and current assignment of human resources for plan definition, approval and implementation, while ‘Institutional, Public and Private Support’ refers to the passive or active engagement of diverse public and private actors in the development of the plan. Scientific and Technical Credibility is divided into three components: Usable Knowledge; Monitoring, Evaluation & Reporting (MER); and Adaptive Management. ‘Usable Knowledge’ refers to the production and use of contextualised evidence (regarding climate impacts, risks and vulnerability) according to local needs and ‘MER’ to the existence of systems that assess progress and outcomes according to a set of goals, while ‘Adaptive Management’ refers to the process of learning through readjustment processes that allows revision, redefinition or change to alternative pathways. Overall, therefore, the APC comprises seven components, including ‘Legitimacy’.

Figure 1. Climate change Adaptation Policy Credibility (APC) framework.

Operational framework at a local scale

Local adaptation policies usually take the form of strategies or more concrete plans setting out a series of measures defined to reduce assessed or perceived climate risks. For the seven components identified in the conceptual framework (Figure 1), a review of the relevant literature has been performed resulting in the identification of 17 indicators and 53 assessment metrics. Table 1 lists the indicators selected and the number of metrics defined for each indicator. The complete list of metrics with an extended description and the chosen evaluation method for each metric can be found in the Appendix (Appendix 1, A1).

Table 1. Operational framework for the assessment of local adaptation policy credibility.

Major Area	Components	Indicators (i#)	No. Metrics	Description (*)
Policy and Economic Credibility	1. Resources	1. Funding	3	Funding refers to the allocation of economic resources to the overall plan, and also to each of the specific measures contained in the plan (Ford and King 2015). It is assumed that an adaptation policy that does not assign economic resources to implementation or monitoring would not be credible. Woodruff and Stults (2016) also find that plans funded by higher governmental levels (or other sources) score lower in quality. It is therefore assumed that, if plans have been self-funded, there is greater intentionality and also greater efforts will be made to ensure quality and achieve objectives.
		2. Consistency	2	This indicator aims to assess the magnitude and coherence of the adaptation economy according to what is contained in the plan and the resources of the city. With this in mind, here, a set of metrics that weight the plan budget against the city’s gross domestic product at the time of plan approval are proposed (Georgeson et al. 2016).
		3. Prioritisation and timing	3	Because of the distributive and long-term nature of climate change impacts, setting priorities is a key to progress on adaptation (Smith 1997; Füssel 2007; Lobell et al. 2008). Prioritisation is also useful to mainstream adaptation into existing policy and reduce competing interests (Champalle et al. 2015). It is thus argued that an adaptation plan that does not clearly say what is important and what is not (by, e.g. establishing prioritisation criteria among the selected measures at the time of implementation), is likely to be less effective than one that does.
	2. Reliability	4. Past performance	3	Past policy performance, regarding climate change or environmental policy matters in general, is important for the credibility of current policies (Averchenkova and Bassi 2016). This indicator takes into account the stage of the adaptation plan under analysis and performance of mitigation policies in place, assuming that mitigation has been addressed before adaptation (Reckien et al. 2014).
		5. Assigned responsibilities	3	According to Woodruff and Stults (2016), plans having been written by planning departments correlates with higher quality. Further, the provision of human resources to implement planned actions is essential to prove readiness for adaptation (Ford and King 2015) and, in this case, it also provides credibility to the plan. Adding to this, it is here argued that it is important not only whether responsible parties are assigned but also whether responsibilities are clearly specified.
	3. Institutional, Public and Private Support	6. Public opinion	1	Public awareness of climate change and perception of risk affect adaptation readiness (Ford and King 2015). Public awareness not only supports policy development and implementation (Averchenkova and Bassi 2016) but is also important in the achievement of policy objectives (Millard-Ball 2012).
		7. Legislation and regulatory nature	2	In cases where there is supporting national legislation or regulations, guidelines for the development of plans are provided (Heidrich et al. 2016). This is here understood as a factor that strengthens the credibility of local climate plans. Climate policies also need to organise action and actors to be credible, and legally binding policies would help this to become a reality (Jordan et al. 2015).
		8. Network membership	1	Considering adaptation-networking research to date, previous local adaptation tracking studies have shown the importance of international climate networks in engaging cities in climate action (De Gregorio et al. 2014; Reckien et al. 2015).
		9. Leadership and support	5	In general, politics affect the credibility of planning (Muchadenyika and Williams 2017). In addition, climate adaptation plans need strong political leadership and authority that builds external and internal legitimacy (Anguelovski et al. 2014). Additionally, Averchenkova and Bassi (2016) argue that, the existence of public and private bodies that support climate change action is essential to build credible policies. A dedicated public climate change body and the support of the upper-tiers of government and the private sector is also critical to channel action (Bulkeley and Broto 2013; Schwarze et al. 2016; Heidrich et al. 2016).
Scientific and Technical Credibility	4. Usable Knowledge	10. Impacts and vulnerability assessment	4	Independently of the approach taken to assess risks, vulnerable populations and assets and related climate impacts should be identified and assessed. Performing an analysis of (cascading) impacts would reduce potential new vulnerabilities (Adger et al. 2005) that may arise from decisions taken on the sole basis of direct observable impacts, this way, reducing the emergence of maladaptive processes (Barnett and O’Neill 2010; Juhola et al. 2016).
		11. Adaptation options assessment	4	In order to ensure that planned adaptation actions are adequate and reasonable (i.e. the best available adaptation options are selected), a preliminary list of potential options should be identified and evaluated (see, e.g. listed options by Stults and Woodruff 2016 in the US). Further, it is important that adaptation actions are connected to climate impacts and the different levels of risks identified in order to verify that planned actions are indeed adequate for expected changes.
	5. Monitoring, Evaluation and Reporting (MER)	12. Monitoring, Evaluation and Reporting (MER) processes	6	In any planning system, MER mechanisms are necessary to govern implementation processes and ensure that plan objectives will be achieved. In the evaluation of local climate adaptation plans, the existence of monitoring systems has been used as an indicator of plan quality (Woodruff and Stults 2016) and climate preparedness (Heidrich et al. 2013) and has been claimed to be a key enabling component in adaptation pathway approaches (Kingsborough et al. 2016; Haasnoot et al. 2018).
	6. Adaptive Management	13. Learning mechanisms	3	Learning and adaptive management are goals of the evaluation of climate adaptation processes and progress (Preston et al. 2011). Flexibility allows preparedness for unexpected events and efficiency of resource use. Adaptation management requires of the recognition of tipping points after which an adaptation action no longer meets the specified objectives and an alternative adaptation pathway needs to be taken (Haasnoot et al. 2013; Haasnoot et al. 2018). Through different operationalizing approaches (see, e.g. Kingsborough et al. 2016), climate adaptation policies can provide flexibility and better embrace uncertainty (Adger et al. 2005).
		14. Uncertainty awareness	1	One important factor that should be taken into account when designing and planning for adaptation is uncertainty. There are different kinds of uncertainty in a climate change context and different strategies for dealing with it in the design of projects and plans (Markandya 2014). Some strategies could entail developing a risk management process, taking into account different scenarios and the widest possible range of outcomes, evaluating the different options against various criteria, or adopting flexible management approaches beyond climate-related uncertainties.
Common Component	7. Legitimacy	15. Transparency and dialogue	5	Policy transparency is critical to raise awareness and provide legitimacy to policy processes. Social acceptance of adaptation options and trust are also important factors (Adger et al. 2005) together with clarity of the rules, availability of information, and the existence of public dialogue (Cosens 2013). Because of the cultural connotations of this, there are no universal models for creating legitimate policies (Adger et al. 2005). Notably, plans also need to be legitimate within the administrative entities responsible for the creation of the plan (this links with Indicator 9; Leadership and support see ref. Anguelovski et al. 2014).
		16. Engagement of stakeholders and civil society	3	Participation of the public, communities, organisations and businesses is an important element in adaptation decision making (Few et al. 2007; Collins and Ison 2009; Sarzynski 2015) because it can help to overcome barriers explicit to adaptation (Biesbroek et al. 2013) and because, as part of a public policy process, it helps to create legitimate plans. Further, it is important to bring into the process stakeholders with legitimate reasons to be there (Mitchell et al. 1997). Additionally, involvement should be sought from individuals with multiple different types of relevant expertise (Weichselgartner and Kasperson 2010).
		17. Equity and justice	3	Equity is a central element for successful climate change adaptation (Adger et al. 2005). Adaptation to climate change is intrinsically spatial (Shi et al. 2016); however, factors related to equity and social vulnerability, which in many cases are reflected in space, are often not taken into account in adaptation policy as much as physical factors (Hughes 2015). To achieve equitable and just adaptation opportunities, participation may be instrumental, if communities or social justice advocacy groups are engaged (Shi et al. 2016). Identifying who is directly or indirectly benefitting from the adaptation action is also relevant to achieving these objectives (Eisenack and Stecker 2012).

(*) See A1 for further details on metrics and evaluation methods

Most of the metrics are qualitative and are defined as closed questions (e.g. Yes or No) (see proposed evaluation method for each specific metric in A1). Positive responses reflect a contribution to the credibility of the policy, and are therefore awarded with 1 point, otherwise, 0 points. For a few open questions, (i.e. referring to plan budget, see metric M#5, or the number of measures contained in the plan, see metric M#6), a specific evaluation method that translates quantitative data into 1 or 0 has been defined.

A lack of information or clarity on the question under assessment either in documents or on the authority’s official websites is indicative of a low credibility; therefore, 0 points are given. Finally, to build the composite credibility index, the scores of the metrics across the indicators are summed up (sub-metrics will be equally weighted, e.g. M#11, M#15, M#28 and M#31). The maximum score for a plan, and therefore, the maximum credibility score, is equivalent to the total number of metrics, i.e. 53.

Pilot application: results and discussion

The framework has been tested using climate adaptation policies of 4 cities with population of over 1 million on different continents and representing different degrees of development, namely: Copenhagen, Durban, Quito and Vancouver. All the cities selected are internationally known for their action on adaptation and are recognized early adaptors; they all have adaptation plans approved between 2006 and 2012, meaning that the plans are all well documented and may even have been revised. Three of them were identified as extensive (Vancouver) or moderate (Quito and Durban) adaptors in the global assessment of urban adaptation by Araos et al. (2016) (while Copenhagen was not in the sample, but has been identified elsewhere as an early adopter of climate policy). As early adaptors, they have made public commitments to adaptation and have joined and submitted their plans to international climate networks (such as the Compact of Mayors or C40). Their extensive experience in adaptation policy making suggested that data sources would be available for most if not all of the APC metrics. Policies related to either climate change in general (2 cases, Durban and Quito) or adaptation to climate change (2 cases, Vancouver and Copenhagen) were identified both containing adaptation measures. Detailed results including metrics and credibility index scores and all documents and their sources (city official websites) are included in the Appendices (A1 and A2 respectively).

Figure 2 shows the final credibility index scores for each city. The best results were obtained for Quito, followed by Vancouver, Copenhagen and Durban. The average credibility index score for these four cities considered to be early and high or extensive adaptors is 29.25 (SD = 3.32) out of a total of 53. Even in the case of extensive adaptors, there are clear areas for improvement, though these may differ between the cities.

Figure 2. City credibility index scores.

As the number of indicators and metrics is not balanced across areas or components (Table 1), areas for improvement need to be analysed individually (see Figure 4). Out of 24 points, Quito and Copenhagen score better in Policy and Economic Credibility (Figure 3), which is mainly to do with the assignment of budgets (see Figure 4 for normalised scores for each indicator). Vancouver obtained the highest score in relation to Scientific and Technical Credibility (Figure 3). This is probably a result of the use of the ICLEI adaptation methodology (ICLEI 2013) that has increased the credibility of its adaptation policy in terms of adaptation options assessment, the establishment of an MER process and their focus on learning mechanisms. Regarding legitimacy, Durban and Quito have more credible participatory processes in place that engage stakeholders and civil society and greater emphasis on vulnerable groups and equity (see Figures 3 and 4).

Figure 3. Credibility scores by major area: Policy and Economic (over 24), Scientific and Technical (over 18) and Legitimacy (over 11).

Figure 4. Normalised credibility scores by indicator. Equal weights for metrics in each indicator have been considered (scores by metric can be found in A1).

Funding and consistency of planned actions with resources in the city are aspects that clearly need to be strengthened. These are two correlated indicators that provide one of the most important factors in terms of Policy and Economic Credibility. No budget assignment means no resources for implementation or no plan to acquire them. Further, in relation to Policy and Economic Credibility, developing regulatory frameworks can help instrumentalise climate adaptation in cities and mainstream it in current urban planning regulations. This would effectively channel efforts for implementation and attract public and private investments. Nevertheless, none of the cities have developed their plans in a regulatory framework or adopted legally binding measures. In terms of Scientific and Technical Credibility, as noted in earlier adaptation tracking studies (Araos et al. 2015, 2016; Stults and Woodruff 2016; Woodruff and Stults 2016), MER systems are not widely implemented and learning mechanisms to enable adaptive management are not sufficiently credible (Figure 5). Notably, credibility of adaptation policies based on their Legitimacy is stronger in developing (Durban and Quito) than in developed cities (Vancouver and Copenhagen) (see Figure 5). A preliminary hypothesis that requires further investigation is that this might be the result of a stronger culture of participation when governance structures are lacking. Although governance and democratic structures tend to be more formalised in developed countries, this does not necessarily mean that society’s interests are properly represented or fully understood by public decision makers, and hence, legitimacy processes are required to avoid unintended consequences or basing adaptation strategies on incomplete information.

Figure 5. Normalised credibility scores by component. Shaded areas define the three major areas: policy and economic credibility (grey), scientific and technical credibility (green) and legitimacy (blue). Equal weights for metrics and indicator have been considered (scores by metric can be found in A1).

In line with this, it is interesting to reflect on how the different scores of credibility may or may not be related to the different development contexts. One may conclude, for example, that many of the actions in Quito’s adaptation policy may be related to a development deficit. Sherman et al. (2016) found that the literature on adaptation in developing countries on the ground offers different interpretations of the integration of adaptation and development. They recognise that this results in different considerations regarding adaptation design, implementation, funding, monitoring and evaluation (Sherman et al. 2016). In this pilot test, any effort puts into providing more adaptive capacity has been considered an indicator of adaptation progress and thereby provide credibility to the process. Further research is required to develop indicators of the relation between adaptation and development, the definition (e.g. how many actions are considered development) and rating (positive or negative) of which will depend on the approach taken (see Sherman et al. 2016).

Finally, it is important to note that the number of metrics considered in this assessment (53) requires an intensive effort in data collection and no analysis has been performed to study potential overlapping or the risk of double counting. From a methodological perspective, reducing the number of metrics would only be possible through a larger experiment (with a large-n sample) in which statistics offer critical information regarding the determinants of credibility.

Conclusions

The implementation of adaptation actions on the ground is largely in its early stages across national and local governments, and the information needed to verify their relative degrees of success will generally only emerge in the distant future, owing to the long-term nature of climate change (Araos et al. 2016; Lesnikowski et al. 2017). Moreover, the ability to attribute observed behavioural or ecological change to specific environmental policy outputs is contested in public policy literature (Knill et al. 2012). Consequently, adaptation policy studies focus primarily on either assessing the merits of various aspects of the policy process, or on tracking changes in policy outputs.

This paper explores the concept of credibility as a critical issue in climate adaptation policy and develops an APC framework that is based on a set of policy, economic and scientific criteria and on the concept of legitimacy. The operational framework relies on 53 metrics describing resources, reliability, institutional, public and private support, usable knowledge, MER processes, adaptive management and transparency, equity and justice. On a preliminary basis, the APC framework has been tested on climate change adaptation plans of four cities around the world, namely Vancouver, Durban, Quito and Copenhagen. Specific results of this pilot application suggest that even advanced cities may find areas for improvement in their adaptation policies. Specific issues refer to funding and a rationale allocation of resources to carry out the plan, the regulatory nature of the measures, and the (lack of) MER processes together with the development of learning mechanisms, participation and equity issues. Cities in developing countries show a stronger emphasis on engagement and equity/justice. This suggests that adaptation progress goes beyond well-structured governance and democracy. Still, the relation between adaptation and development and its influence on credibility, should be further explored.

Based on these preliminary results, the APC framework seems adequate to support large-scale decisions related to prioritisation and funding allocation. This index, combined with others that address, for example, urgency to act, might be used effectively to track adaptation and thus to guide private and public investments and the international agenda.

The conceptual and operational framework proposed herein for the assessment of APC may help in understanding the strengths and weaknesses of adaptation policies and, acknowledging the low sample size conditions of the pilot testing, it has the potential to become an extremely helpful indicator for decision making, enabling regional, national and global efforts to be well targeted, funds effectively allocated, and best-practices transferred, ultimately advancing adaptation science and practice (especially as regards adaptation progress measurement efforts).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Acknowledgements

This study has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 653522 (RESIN - Climate Resilient Cities and Infrastructures project). MO acknowledges funding from the Spanish Ministry of Economy and Competitiveness (MINECO) (FPDI-2013-16631 and IJCI-2016-28835). ESM’s Postdoctoral Fellowship is supported by the Basque Government (POS_2016_1_0089).

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the European Commission [653522];Eusko Jaurlaritza [POS_2016_1_0089];Spanish Ministry of Economy and Competitiveness (MINECO) [FPDI-2013-16631,IJCI-2016-28835];

Notes on contributors

Marta Olazabal

Marta Olazabal is a research fellow and Head of the Adaptation Research Group at the Basque Centre for Climate Change (BC3). She is currently holder of a two-year AXA postdoctoral fellowship grant and a Juan de la Cierva grant by the Spanish Ministry of Economy and Competitiveness. Her current interests relate to urban climate adaptation policy and governance, knowledge co-production, systems thinking and concepts and methods to assess urban resilience and sustainability.

Ibon Galarraga

Ibon Galarraga is Ph.D. in Economics from the University of Bath (UK), M.A. Economics at the University of Essex (UK) and B.A. Economics at the University of Basque Country (Spain). He has worked as an environmental consultant for both public and private clients. He has published more than 40 research papers and 14 book chapters in the fields of environmental, energy and climate economics and policy. Ibon has actively contributed to more than 12 international research projects such ECONADAPT (FP7), RESIN (H2020), Water 2 Adapt (IWRM-net). PREEMPT (EC DG ECHO) or CONSEED (H2020).

James Ford

James Ford is a Priestley Chair in Climate Adaptation at the Priestley International Centre for Climate at the University of Leeds, UK. His research focuses on climate change impacts, adaptation, and vulnerability, with a strong focus on the Arctic and Indigenous populations. Alongside his community-based research, he has promoted the development of systematic review approaches in the human dimensions of climate change field, and made major contributions towards efforts to track adaptation policy across scales. The author of >170 articles, Professor Ford is a lead author on the IPCCs Special Report on 1.5C of warming, was a contributing lead author on the Arctic Councils Adaptation Actions for a Changing Arctic assessment, and is editor-in-chief at Regional Environmental Change.

Elisa Sainz De Murieta

Elisa Sainz De Murieta is Post-doctoral Researcher at the Basque Centre for Climate Change (BC3). She obtained a BSc. in Geological Sciences from the University of the Basque Country (2001) and a Master’s degree in Environmental Engineering and Management at the School of Industrial Organization in Madrid (2002). She is currently Visiting Fellow at the Grantham Research Institute (London School of Economics), where she is developing a three-year project on climate change adaptation and risk in coastal cities, funded by the Basque Government.

Alexandra Lesnikowski

Alexandra Lesnikowski is a PhD Candidate in the Department of Geography at McGill University. She began working with the Climate Change Adaptation Research Group in 2010 on the development of experimental frameworks for tracking adaptation policies across countries. The results of this research are published in a number of journals, including Nature Climate Change, Environmental Research Letters, and Global Environmental Change. Her doctoral research is focused on advancing these methods for assessing adaptation policy patterns among local governments. She holds a Masters of Arts (Urban Planning) from The University of British Columbia (2014) and a Bachelor of Arts (Hons.) from McGill University (2010).

Notes

1. Here, the term ‘adaptation policies’ is used to refer globally to the group of instruments, strategies and plans that are designed and implemented to achieve climate change adaptation goals.

APPENDIX 1 – TABLE 1 EXTENDED

Table A1.1 is based on Table 1 of the manuscript and includes metrics, evaluation method for each metric and the pilot application including the scores for each city.

APPENDIX 2 – LIST OF REVISED DOCUMENTS AND SOURCES

These documents are considered to be the latest update (as of March 2017) on the general adaptation policy of the city. In some cases, there are previous documents that have been taken into account (Durban and Quito) or other documents covering specific projects or initiatives emerging from the general plan (Vancouver and Copenhagen). First adaptation-related policies appear in documents published in 2006 (Durban) and 2009 (Quito). In the case of Vancouver (2012) and Copenhagen (2011), the documents revised are the first ones containing adaptation-related policies.

Table A2.1. List of revised documents and sources - Vancouver

VANCOUVER
Website	City of Vancouver official website containing information related to climate change strategies, programs or projects: http://vancouver.ca/green-vancouver/climate-change-adaptation-strategy.aspx
Documents (ADAPTATION)	The City of Vancouver Climate Change Adaptation Strategy (The Sustainability Group, 2012) NHC, 2014. City of Vancouver Coastal Flood Risk Assessment. Prepared for City of Vancouver, December, 2014.
Documents (MITIGATION)	City of Vancouver climate change mitigation strategy: Greenest City Action Plan (2011): http://vancouver.ca/green-vancouver/greenest-city-action-plan-development.aspx

Table A2.2 List of revised documents and sources – Durban

DURBAN
Website	Durban official websites containing information related to climate change strategies, programs or projects: http://www.durban.gov.za/City_Services/energyoffice/Pages/DurbanClimateChangeStrategy.aspx http://www.durban.gov.za/City_Services/energyoffice/Pages/DurbanClimateChangeStrategy.aspx http://www.durban.gov.za/City_Services/development_planning_management/environmental_planning_climate_protection/Projects/Pages/Municipal-Adaptation-Plans.aspx http://www.durban.gov.za/City_Services/development_planning_management/environmental_planning_climate_protection/Projects/Pages/Community-Adaptation-Projects.aspx
Documents (ADAPTATION)	Durban Climate Change Strategy (2015). Environmental Planning and Climate Protection Department (EPCPD) and the Energy Office (EO) of eThekwini Municipality. Community-based Adaptation to Climate Change in Durban (2011) Report Number. 11977-10286-9. Golder Associates Africa (Pty) Ltd for eThekwini Municipality Durban’s Municipal Climate Protec on Programme: CLIMATE CHANGE ADAPTATION PLANNING FOR A RESILIENT CITY. 2010/2011. Author: Meggan Lewis. Reviewed by: Debra Roberts, Environmental Planning and Climate Protection Department, EThekwini Municipality. CLIMATIC FUTURE FOR DURBAN: REVISED REPORT. March 2006. Prepared for: EThekwini Municipality. Prepared by: CSIR Environmentek. Authors: Shamini Naidu, Rob Hounsome & Kogi Iyer. Edited by: Debra Roberts, Andrew Mather & Manisha Maganlal CSIR REPORT. HEADLINE CLIMATE CHANGE ADAPTATION STRATEGY FOR DURBAN. 2006. Prepared for: Dr Debra Roberts, Deputy Head: Environmental Management Department. Ethekwini Municipality. Prepared by: Mr Rob Hounsome and Ms Kogi Iyer
Documents (MITIGATION)	Ethekwini Municipality Energy Strategy, 2008. CAN ET011 0005. Prepared by: David Mercer, Enviros Consulting Limited. Approved by: Dr Debra Roberts, Ethekwini Municipality

Table A2.3. List of revised documents and sources - Quito

QUITO
Website	Quito official website containing information related to climate change strategies, programs or projects: http://www.quitoambiente.gob.ec/ambiente/index.php/cambio-climatico/programas-y-proyectos
Documents (ADAPTATION)	Municipio del Distrito Metropolitano de Quito, Secretaria de Ambiente (2009). Estrategia Quiteña al Cambio Climático. Quito, Ecuador. Municipio del Distrito Metropolitano de Quito, Secretaria de Ambiente (2012). Plan de Acción Climático de Quito 2012-2016. Quito, Ecuador. Municipio del Distrito Metropolitano de Quito, Secretaria de Ambiente (2015). DOCUMENTO TÉCNICO SISTEMATIZACIÓN: Conocimiento de la vulnerabilidad y posibles medidas de adaptación al cambio climático en el Distrito Metropolitano de Quito. Quito, Ecuador Municipio del Distrito Metropolitano de Quito, Secretaria de Ambiente (2015). AYUDA MEMORIA: Diagnóstico local participativo para el análisis de percepciones sobre amenazas, vulnerabilidad, género y medidas de respuesta frente al cambio climático en las parroquias de la norcentralidad del DMQ. Quito, Ecuador
Documents (MITIGATION)	Not Applicable

Table A2.4. List of revised documents and sources – Copenhagen

COPENHAGEN
Website	City of Copenhagen official website containing information related to climate change strategies, programs or projects: http://international.kk.dk/artikel/climate-adaptation
Documents (ADAPTATION)	CLIMATE CHANGE ADAPTATION AND INVESTMENT STATEMENT. Part 1, October 2015 CLIMATE CHANGE ADAPTATION AND INVESTMENT STATEMENT. Part 2, October 2015 Cloudburst Management plan 2012 Copenhagen Climate Adaptation Plan - 2011
Documents (MITIGATION)	CPH 2025 Climate Plan (2011) http://international.kk.dk/artikel/carbon-neutral-capital