The impact of sustainable tourism indicators on destination competitiveness: the European Tourism Indicator System

Abstract

We aimed to evaluate the impact of sustainable tourism indicators on destination competitiveness with reference to the European Tourism Indicator System (ETIS), a scheme funded by the European Commission to address the evidence gap in tourism policy making. To do this, we evaluate the absorptive capacity of destination management organisations (DMOs) to implement and use sustainable tourism indicators to make policy decisions. We provide evidence of how DMOs have acquired knowledge about the importance of sustainable tourism indicators through ETIS, and how they have assimilated it by developing their own systems based on the principles of ETIS. However, we find that the European Commission had unrealistic expectations that DMOs, or their policies, would be transformed as a result of the use of indicators, or that indicators would be exploited to improve tourism sustainability and competitiveness. We contribute to the study of policy science by showing how absorptive capacity can be used to analyse and evaluate policy interventions, despite being a linear rational approach to explaining a complex policy context.

KEYWORDS:

• Destination management organisations
• competitiveness
• absorptive capacity
• sustainable tourism indicators
• dynamic capabilities
• policy making

Introduction

This study aims to understand better how destination management organisations (DMOs) develop dynamic capabilities to gather and act on external knowledge, in order to inform policy decisions. Rational policy making approaches suggest that policy decisions should be evidence based, but a disillusionment with attempts to evaluate the impact that scientific research has on policy decisions (Thomas & Ormerod, 2017; Xiao & Smith, 2007) has led to seeking better explanations for how decisions are actually taken (Dredge & Jamal, 2015; Dredge & Jenkins, 2007, 2011; Hall, 2008; Hall & Jenkins, 1995). While these studies contribute to unpacking the black box of the tourism policy making process and explaining the failed sustainable tourism inclusion into governments’ agendas (Farmaki et al., 2015), limited attention has been given to learning and knowledge management (Bryson et al., 2010; Hall, 2011) and specifically to the processing of sustainability performance evidence in policy making (Ruhanen, 2013; Vila et al., 2010). Such understanding is particularly relevant due to an increasing sustainability accountability for policy makers.

The theoretical contribution of this study is the use of the concept of absorptive capacity (ACAP) to respond to this need. ACAP is conceptualised as a learning journey through which organisations acquire, assimilate, transform and exploit knowledge (Todorova & Durisin, 2007; Zahra & George, 2002). While originating in the business sector, it has recently found relevance in public management (Butler & Ferlie, 2020; Harvey et al., 2010; Richards & Duxbury, 2015). As argued by Harvey et al. (2010), the application of ACAP in the public sector is significantly valuable since it helps understand the ability of public organisations to process knowledge within complex and multi-layered settings. This contestation stems from the inflexibility of the public sector in adapting its mandate, redesigning its knowledge acquisition systems, developing sensitivity towards new data sources that are often co-created by stakeholders, and responding to the faster pace of these stakeholders who expect a certain level of responsiveness on the part of the destination institutions (Fayard et al., 2017).

This study aims to explain how DMOs process knowledge from sustainable tourism indicators to inform destination governance decisions, with reference to the European Tourism Indicator System (ETIS). In doing so, it aims to evaluate the impact that ETIS has had as a policy intervention, responding to the Communication from the Commission - Agenda for a sustainable and competitive European tourism (Commission of the European Communities, 2007; Estol & Font, 2016). ETIS is a voluntary tool launched by the European Commission to manage, inform and monitor the sustainability performance of tourism destinations. It is based on 43 indicators structured in four categories: destination management, social and cultural impact, economic value, and environmental impact (see European Commission, 2016). Therefore, its design is specifically conceived to collect and analyse data to assess the impact of tourism on a destination. Its implementation process urges destinations to first raise awareness, engage stakeholders and define responsibilities and then to collect data and analyse results. This process is intended to lead destinations to establish their own sustainability indicator system. The ETIS asks DMOs and other stakeholders to think beyond traditional tourism statistics of number of tourists, length stay, tourist spending… and it provides a framework to reflect on the impact that tourism has on the local economy, community and environment. Thus, ETIS was designed to provide new insight into tourism development, offering to DMOs the chance to include new Key Performance Indicators into planning and management processes.

The article is structured as follows. We start by summarising the literature on why policy making is not necessarily informed by evidence, and analyse the logic behind using sustainable tourism indicators to inform policy decisions. We explain the shortcomings behind showing evidence of impactful research, by using the concept of absorptive capacity, to demonstrate that embedding knowledge into organisational structures takes time, and that evidencing causality is riddled with challenges. We move on to outline the mixed methods used to analyse Google scraped documents, interviews and focus groups. We then provide a historic account of ETIS, before unpacking the aspects that demonstrate the relative success of ETIS to gradually inform policy making through the acquisition, assimilation, transformation and exploitation of knowledge brought about by sustainable tourism indicators. We discuss the theoretical implications from this study, and summarise the lessons learned.

Literature review

At a time when public service organizations are expected to be responsive and co-produce solutions with stakeholders (Harvey et al., 2010), public sector agencies need organisational competencies and activities to manage policy complexity. There is increasing pressure for Destination Management Organisations (DMOs) to be seen incorporating sustainability in their policies, arguably because it is a competitive factor for tourist destinations and because there is a need to plan and manage properly future tourism and its associated impacts (McLoughlin & Hanrahan, 2019). In doing so they are expected to collect sustainability data in the form of indicators and to find ways to use this data for critical policy information (Torres-Delgado & Saarinen, 2014).

An indicator is “a measure of the existence of some issue or phenomenon of interest, used to describe an aspect of a society, macro-societal activity or geographical area, or to point out to changes in these factors” (Volo, 2015, p. 277). Therefore, indicators can describe and measure the reality of a destination, facilitating understanding of a particular territory and the elements and processes that take place there (Torres-Delgado & Palomeque, 2014). Sustainable tourism indicators can be used to: i) to monitor sectoral development so as to facilitate the assessment of policies and practices; ii) to measure sectoral progress and develop suitable strategies for a preferred future and iii) to communicate knowledge via the generation of quantitative and objective data that provides a fuller understanding of tourist phenomena in their spatial context (Castellani & Sala, 2010; Valentin & Spangenberg, 2000). The World Tourism Organization (2004) has promoted indicators as an essential planning and management tool for tourism planners because they i) provide information on issues and areas of concern (impacts, product quality, threats, etc.), ii) help evaluate the tourism plan performance and iii) provide evidence to assess the planning and policy framework). However, it is particularly important to look beyond how knowledge is produced and shared, towards how it is absorbed, in order to understand the role of such indicators in tourism policy making (Pee & Kankanhalli, 2016).

A linear use of indicators in policy making follows the principles of evidence-based policy. Accordingly, policies are formulated based on a continuous process of gathering and critically appraising evidence (Davies, 2004). This culture has been taken up in a range of public sectors, after the numerous calls for accountability and transparency faced by democratic countries (Davies & Nutley, 2000; Head, 2016). While ideal, this approach has been considered a myth, finding a limited applicability into the real world (Hammersley, 2013). This is because the rational and linear view of policy making does not reflect the utterly contested and messy nature of policy making (Geyer & Rihani, 2010). Scholars acknowledge such complexity, arguing that tourism policy decisions are a social construct and as such are dependent on the various interests, values, ideologies and relationships of stakeholders from both inside and outside government, and the institutional framework in which they all operate (Bramwell, 2011; Dredge & Jenkins, 2007, 2011; Hall, 2008; Hall & Jenkins, 1995; Stevenson et al., 2008). Hence, the term ‘evidence-based’ is discarded and replaced with ‘evidence-influenced policy making’ (EIPM), whereby evidence is no longer the foundation for the policy process but rather one of the influencing factors that can lead to policy change or development (Duncan, 2005; Head, 2010).

The concept of ACAP has been applied to best explain destinations’ and tourism firms’ capacity to innovate (Thomas & Wood, 2014, 2015; Williams et al., 2020). ACAP is a construct initially developed to describe a firm’s learning process, specifically its ability to identify, assimilate, and exploit new knowledge derived from outside sources (Cohen & Levinthal, 1990). Organisations gain competitive advantage by acquiring and making use of information found in their external environments (Freeman, 2001). ACAP helps us to understand how organisations transform that knowledge/data about their operating environment into evidence that can be used to improve their organisational performance. This learning can, in turn, spur innovation in the form of new products or services. We posit that ACAP can be used to explain the influence that sustainable tourism indicators play in the DMOs decision making process in order to learn about the health of their destination and adapt accordingly.

However, the effective impact of sustainable tourism indicators in the context of DMOs could be hampered by governance with complex interrelations among stakeholders (with diverse and divergent perspectives) and a dynamic and fragmented environment (fuzzy boundaries) (Amore & Hall, 2016). DMOs have potential on innovative capacity and social learning (Luthe & Wyss, 2016), due to a predominant role as a manager of firms and industries networks that allow an exchange of information, use of synergies and coordination of action (Bornhorst et al., 2010; Volgger & Pechlaner, 2014). However, as the mandate of DMOs is typically to increase visitor volume and expenditure, innovations that incorporate sustainability rhetoric will essentially be driven by (short-term) economic factors (Amore & Hall, 2016; Tervo-Kankare, 2011; Wyss et al., 2014).

ACAP has rarely been applied specifically to public sector research (Harvey et al., 2010; Richards & Duxbury, 2015), even though knowledge management has a positive impact on public sector organisational effectiveness (Pee & Kankanhalli, 2016). However, whilst the fundamentals of learning may not vary between public and private institutions (Boyne, 2002), there are contextual distinctions that bear on the key actors associated with motivations and capacities for acquiring new knowledge (Hartley, 2006; Richards & Duxbury, 2015) as well as on the goals for acquiring that knowledge (Rashman et al., 2009). Whilst external knowledge provisioning is an established and accepted activity for developing the entrepreneurship competitive advantage of private firms (Argote & Ingram, 2000; Zahra et al., 1999), knowledge creation by public organisations is driven by policy prescriptions rather than being a part of the internal objectives of the public organisation (Hartley & Skelcher, 2008; Rashman et al., 2009).

Those DMOs that have more inclusive mandates, provide more flexibility for staff to define their roles and be innovative, and have a wider range of stakeholders in their board, will be more dynamic in responding to stakeholder needs (Boksberger et al., 2011). This dynamism and responsiveness, in turn, leads to hybrid forms of knowledge production and decision-making. Accordingly, these new-governance modes lead to rebalance the neoliberal with the participation or, in other words, consider not only economic returns but also social benefits (Hjalager, 2020). Thus, governance innovation is slowly moving towards embracing and creating value with multiple stakeholders in collaborative and co-creative formats (Fotino et al., 2018).

Zahra and George (2002, p. 186) revised and expanded the concept of ACAP initially developed by Cohen and Levinthal (1990), framing it as a four-dimensional “set of organisational routines and processes by which firms acquire, assimilate, transform, and exploit knowledge to produce a dynamic organizational capability.” Zahra and George hold that ACAP is a dynamic process and emphasise the processes that facilitate absorption of new knowledge. Their reformulation is underpinned by the introduction of two complementary components of ACAP: potential and realised capacity, and it is related to information/knowledge and indicator use in every governance episode within a DMO (see Figure 1). Further, Zahra and George’s model considers both of internal and external factors that moderate the development and deployment of ACAP, including the ease with which information flows within an organisation.

Figure 1. Potential and realised capacity in the use of sustainability indicators for policy making.

Potential absorptive capacity

Potential ACAP incorporates the capacities to identify and collect relevant external information (acquisition) as well as to understand and interpret this information (assimilation). The information for EIPM can come from a variety of sources. Perhaps the most obvious mechanisms for gathering evidence are monitoring and evaluation systems, made up of process and performance indicators. Monitoring frameworks can be established to report on progress, as measured by a set of indicators, and gather data in a consistent and rigorous manner (such as ETIS).

Much of the literature about sustainable tourism indicators refers to the creation of indicators to allow for information acquisition (Miller & Twining-Ward, 2005; Torres-Delgado & Saarinen, 2014). This literature has helped us identify key requirements to design of indicators. First, selecting a concrete and meaningful group of indicators is essential. A balance between a scientific approach and political or public consensus is needed; top-down indicators neglect stakeholder views and can be too complex and difficult to implement (Tanguay et al., 2013), while bottom-up indicators can be subjective and politicised (McCool et al., 2001; Tanguay et al., 2013). Second, indicators need to strike a balance between their contextual specificity and their global relevance, to include both local and global impacts. Most sustainable tourism research focuses on local-scale and short-term issues within administratively defined units (Blancas et al., 2011; Torres-Delgado & Palomeque, 2018). In addition, data are often available in relation to administrative units (e.g., municipal or regional) that are not appropriate to interpret the findings, or to make policy decisions (Schianetz et al., 2007). Third, calculating indicators is a further challenge since we value what we measure, rather than measuring what we value; while data collection is expensive and sustainability data are limited (Torres-Delgado & Palomeque, 2014), governments can only make policy decisions based on available data (Lozano-Oyola et al., 2012). Finally, understanding the meaning of indicators requires the politically contested process of defining sustainability thresholds (Blancas et al., 2011).

The second step of ACAP, the assimilation of the indicators acquired, confirms that evidence is neither neutral nor incontestable. Assimilation can be impeded by numerous demand-side factors since as mentioned earlier, policy making is not simply context-free technical problem solving (Sanderson, 2006), but a complex process that is context sensitive and highly political. Factors such as the hegemonic dimension of power are considered critical in understanding the legitimisation of evidence in tourism policy making (Hall, 2010). Power may not entirely lie in government actors, but may indirectly be held by strong interest groups within the private sector that try to steer the tourism agenda (Bramwell, 2011; Dredge & Jamal, 2015). Values and ideologies of influencing stakeholders are also important factors since they shape how evidence is perceived (Hall & Jenkins, 1995; Hall, 2008); certain evidence may be considered offensive if it does not align with a government’s position (Farmaki et al., 2015; Guenther et al., 2010; Tanguay et al., 2013), for example an economic measure may be deemed too market-oriented or too socialistic for the governing party. In addition, political and socio-cultural environments that do not allow the forming of strong stakeholders’ networks can hinder the sharing and assimilation of knowledge (Presenza & Cipollina, 2010). Other influences on policy decisions include habits and tradition, expertise and experience, judgement, pressure groups, resources and timeframes (Sutcliffe & Court, 2005). The combination of all these factors leads policymakers to select evidence to advance specific agendas, so there is a potential for EIPM to give the policy-elite the means to increase their strategic control by manipulating evidence and using it to “devalue the voices of ordinary citizens” (Dredge & Jenkins, 2011; Marston & Watts, 2003, p. 3). Hence, the collection of quality data is a prerequisite, not a guarantee, to EIPM (Rutter, 2012).

Realised absorptive capacity

Realised ACAP speaks to an organisation’s abilities to merge its existing and newly acquired knowledge and synthesise it (transformation) and to benefit from this knowledge to create a new product or service (exploitation). Thus, it is a turning point where information is translated into knowledge. The differing practices and incentives, and a lack of common ground, between the producers of evidence (in the form of indicators) and the policymakers who are to use it, can create barriers to productive research use (Nutley & Davies, 2000; Xiao & Smith, 2007).

A tourism destination’s competitiveness is determined by “its ability to increase tourism expenditure, to increasingly attract visitors while providing them with satisfying, memorable experiences, and to do so in a profitable way, while enhancing the well-being of destination residents and preserving the natural capital of the destination for future generations” (Ritchie & Crouch, 2003, p. 2). Although this oft-quoted definition makes sustainability an explicit requirement of destination competitiveness, DMOs tasked with serving the needs of destination stakeholders are likely to define their organisation’s success with more inward looking, operational and short-term metrics than a destination would (Bornhorst et al., 2010). It is not clear whether public sector tourism agencies represent stakeholder interests in quite the same way (Dredge, 2006a).

The ability of a DMO to transform knowledge depends on how well such knowledge fits the current organisational structure and values (Nutley et al., 2003). DMOs are therefore likely to develop dynamic capabilities to achieve their own goals, and not necessarily those of the destination. To be taken on board, (sustainability) policy reforms need to be sufficiently ambitious to arouse interest, but not so ambitious that they require fundamental overhauling of organisational systems (Sabatier, 1986). Factors such as restructuring government departments, shifting priorities, evaluating findings, and high staff turnover, can greatly reduce the opportunities for evidence-based policy to be developed (Guenther et al., 2010). Timetables on which the policy and research processes tend to operate can be so different as to prevent collaboration from being productive for either party (Rutter, 2012; Sanderson, 2002).

Knowledge exploitation is the organisational capability that allows DMOs to build on existing competencies with the newly acquired knowledge (Zahra & George, 2002). We reiterate that the purpose of this article is to analyse the process of design and use of sustainable tourism indicators to make policy decisions that inform destination competitiveness, hence reaching the stage of knowledge exploitation as set in ACAP.

Methodology

We focus on the impact that ETIS has had on destination governance, as the use of sustainable tourism indicators as part of destination decision-making management systems is central to European tourism policy (Torres-Delgado & Palomeque, 2014). Action 16 of the Communication (European Commission, 2010) mandated the European Commission to develop a system of indicators for the sustainable management of tourist destinations. Accordingly, in February 2013, the European Commission issued a contract to develop ETIS as a comprehensive and flexible system, suitable for a wide range of tourist destinations: i) to create awareness of the impacts of tourism; ii) to form stakeholder groups and identify responsibilities for data collection and actions; iii) to collect, record and analyse data and iv) to enable ongoing development and continuous improvement (European Commission, 2019). We therefore evaluate impact based on the decision that drove the European Commission to sponsor ETIS, namely that there is an evidence gap in tourism policy making and that policy makers have neither sufficient sources of evidence to appreciate the negative unintended consequences of tourism policy nor the mechanisms to use this evidence to take informed decisions (Estol & Font, 2016).

Based the literature review, we acknowledge the flaws in this rational explanation in what is a more complex reality, and yet we deliberately take a parsimonious approach to study the ability of sustainable tourism indicators to fill this evidence gap and, in so doing, to facilitate innovation capacity. Understanding ACAP in the public sector ought to combine objective and subjective measures (Harvey et al., 2010). Hence, we took a dominant/simultaneous QUAL + quan design for the purpose of data expansion, the most common mixed methods approach in sustainable tourism studies (Molina-Azorín & Font, 2016). Mixed methods are particularly appropriate to engage with stakeholders in multiple ways in order to reflect collectively on data previously collected, and to promote societal change (Molina-Azorín & Font, 2016).

First, a scraping process of Google allowed us to harvest and systematically analyse the results of the search engine, based on the [“European Tourism Indicator System” + ETIS] query. See Palomo and Montalvo (2011) and Montalvo et al. (2018) for more details on this method. Initially, the Google search returned a total of 197 URLs and 86 PDF documents. After a manually-supervised screening process, the crucial information to ascertain the impact of ETIS was obtained through a systematic review and analysis of 105 URLs (duplicated and non-relevant URLs were removed) and 99 PDF documents (reading the identified documents provided leads for new relevant documents that were subsequently added to the list) published between 2013 and 2019. The word ETIS appeared on 91.7% of the results and was the main topic in 58.4% of them. There were 39 research papers (including journal articles, proceedings, doctoral theses, and reports). Also, 43 documents contained case studies on tourism indicators; 38 specifically on ETIS.

Next, we co-organised a workshop on sustainable tourism indicator design and use, making use of the critical mass of experts already gathered to attend the Interreg project MITOMED+ (Models of Integrated Tourism in the MEDiterranean Plus), in Malaga (Spain) April 2018. The event attracted 114 people representing European destinations from 11 countries, international organisations such as UNWTO, the European Statistical System, the European Commission and academics. The participants included members of the MITOMED + funded project, as well as other destinations and experts primarily from Mediterranean countries gathered through snowball sampling, based on their track record in sustainability management. The first day was dedicated to introducing an online platform for indicator benchmarking in Mediterranean destinations. The second day was organised as two roundtables: one to identify challenges relating to adopting and calculating STIs (potential ACAP), and one to better understand opportunities to use STIs to inform decision-making (realised ACAP). The third day was used to identify practical solutions, in small stakeholder groups, and to prioritise the results through an online voting app. Four authors of this study participated in organising and delivering the workshop.

Finally, the same four authors interviewed over 50 DMO representatives and experts from across Europe in relation to their knowledge and use of sustainable tourism indicators. The initial sampling frame was the list of destinations that participated to ETIS, listed in the European Commission website. Contacts were manually retrieved from a Google search. The list was then expanded by snowball sampling. Approximately half of the sample had responsibilities for sustainable tourism activities within their organisation, while the other half had responsibilities for statistics, product development, marketing or held management level positions. Over 20 DMOs are mentioned by name in the results. Interview questions were informed by the four steps in the ACAP literature, which allowed the interviewees to chronologically narrate the storyline of their sustainability actions over time until the present, contextualising their use of indicators as part of broader policy decisions, which is why some of these cases trace back to the early 2000s. The interviews were used for qualitative respondent validation of the data gathered through the content analysis of the documents scraped from Google, and the workshops, in order to test whether they recognised the theory used to explain the quantitative data as the reasoning behind their behaviour (Manzano, 2016; Mays & Pope, 2000), as well as to gather further evidence of the impact achieved by the use of indicators in their tourist destinations.

A summative content analysis, involving counting and comparisons, was used to analyse the data from the online documents and websites, while a directed content analysis was used to inform the design of the workshops and the structure of the interviews, and the subsequent analysis of the resultant data. In all instances, the line of enquiry and coding of the results aimed to trace the logical progression across the four stages of ACAP, while remaining sensitive to the underlying context of the cases analysed.

Results

In order to evaluate the impact of ETIS, it is necessary to first trace the process of its development and piloting through a desk review of the publications scraped from a Google search. Over 200 destinations expressed interest in testing the ETIS toolkit in two pilot phases (2013–2014 and 2014–2015), although fewer than 40% of them actually participated. After ETIS was launched, the European Commission arranged four meetings (face to face in July 2014 and January 2016, and video conferences in June and October 2015) to exchange experiences among destinations, structured to discuss i) Organisational and management issues (Steps 1 to 4 of the toolkit); and ii) Implementation of indicators and data collection (Steps 5 to 7 of the toolkit). The results showed that 65% of the destinations that participated in either phase were satisfied with the ETIS toolkit, deeming it to provide sufficient guidance for its implementation. The pilots allowed DMOs to suggest improvements in the core and optional indicators, destination profile and dataset, and guidance and terminology in the seven ETIS steps. The European Commission provided examples of good practice and expert support to overcome difficulties, and collected information to adapt and improve the proposed set of indicators (for example, several core and optional indicators were proposed to be removed, and optional indicators were redefined as core and vice versa (Romagosa & Sirse, 2016). Table 1 presents the main conclusions of the ETIS implementation from the two pilot phases.

Table 1. ETIS. Conclusions from each implementation step. Source: Adapted from Romagosa and Sirse (2016).

ETIS step	Main conclusion
Step 1 Awareness raising	Lack of media support and co-operation from partner organisations
Step 2 Creation of a destination profile	Destination profile form applicable and minimal difficulties in filling it
Step 3 Forming Stakeholder Working Group (SWG)	Inactive members of the SWG, and more interest from the public sector than from the private
Step 4 Role/Responsibilities of SWG	Difficulties on agreeing about timeline for data collection, and minor challenges on responsibilities
Step 5 Collecting & registering data	Data difficult to obtain, and/or resources were not available
Step 6 Analysis of results	SWG agreement on priorities but difficulties to agree on action plans
Step 7 Continuity & Improvement	Indicators and data are not reviewed regularly and additional resources are difficult obtained

Pilot case studies show that about 30–40% of the requirements of ETIS could be met with official statistical data. However, the European Commission’s Virtual Tourism Observatory platform, requested by pilot destinations to enable national and international on-line benchmarking and networking, has not been launched to date, nor has any kind of certification, labelling or ranking been delivered to provide recognition of, or encourage, the DMOs’ involvement (cf. Feyers et al., 2019). Destinations were faced with: i) the costs of conducting surveys on visitors, residents and tourism businesses (Modica et al., 2018; Tudorache et al., 2017); and ii) the task of tailoring the indicators to their destination (López Palomeque et al., 2016; Tudorache et al., 2017; Zabetta et al., 2014).

The two pilot phases led to a 2016 restructure of ETIS into 43 core indicators, with supplementary indicators tailored to different geographical areas, sizes of tourist destination and levels of data availability (see European Commission, 2016). With an aim to continue to raise awareness of the destinations’ commitments to sustainable tourism, in April 2016 the European Commission organised the “ETIS and Accessible Tourism Awards,” where eight destinations were recognised for efforts made to use ETIS to measure and enhance their sustainable management performance. After the final award event in 2016, the European Commission ceased to support all ETIS related activities, transferring the ownership of ETIS to single destinations.

Figure 2 summarises the ETIS-related publications (identified through the Google scraping process, dating from 2007 to June 2019) that document the process outlined above. In the period running up to 2013, a series of documents making the case for ETIS were published. There was a peak of productivity in 2013, when ETIS was commissioned and the first toolkit released, and again in 2016, when the revised toolkit and summary of pilot case studies were published. After this year, we mostly find location-specific evaluations. Of the documents reviewed, 94% were produced in the EU, of which 49% were published by the European Commission itself, with the rest coming from DMOs and academics, mainly in Spain (9%), United Kingdom (6%), Germany (5%), Portugal (4%) and Italy (4%). These documents referred to the international (57%), national (22%) and sub-national (21%) application of ETIS. The European Commission confirmed that its own ETIS documents had been downloaded over 60,000 times by May 2019. We do not have data about documents referring to ETIS published elsewhere.

Figure 2. ETIS-related publications over time.

In 31% of the documents, a sentiment about ETIS was expressed. These sentiments were positive in 97% of cases, reflecting the voluntary nature, and user-friendly approach, of ETIS to implementing simple methods of collecting locally-relevant indicators with potential for European benchmarking. At the same time, 17% of the documents identified challenges in implementing ETIS, mostly stemming from the difficulty of involving actors to provide data, the lack of availability or reliability of the data, the cost and lack of expertise to conduct new field research, and the lack of applicability of some indicators. Documents also highlighted conceptual shortcomings in the definition of indicators and functional shortcomings in the benchmarking tool. The documents that reported actual impacts of implementing ETIS (8% of total) emphasised that the collection of baseline data generated stakeholder awareness on impacts, created momentum and enhanced collaboration. Ten percent of the documents provided recommendations to improve ETIS, to facilitate data collection, to collect more accurate data on fewer indicators, to allocate stakeholder duties and manage their expectations, to adapt the system to the territorial reality and to provide comparison with similar destinations.

Table 2 presents the evidence collected from the reviewed case studies organised by the ACAP steps.

Table 2. ACAP. Evidence collected from the case studies.

ETIS requirement	Yes, explicitly	Yes, implicitly	No	No data
Acquisition	48.6%	10.8%	8.1%	32.4%
	59.5%		8.1%	32.4%
1. Create Awareness	40.4%		6.1%	53.5%
1.a. Existence of local destination coordinator	24.2%	21.2%	6.1%	48.5%
1.b. Communication decision to stakeholders	45.4%	3.0%	6.1%	45.4%
1.c. Gaining political support, publicly announced	15.1%	12.1%	6.1%	66.7%
2. Create destination profile	33.3%		6.1%	60.6%
2.a. Destinations that have completed a destination profile form or equivalent	12.1%	21.2%	6.1%	60.6%
Assimilation	47.2%	5.6%	8.3%	38.9%
	52.8%		8.3%	38.9%
3. Form stakeholder working group (SWG)	60.6%		6.1%	33.3%
3.a. Presence of SWG	54.5%	18.2%	6.1%	21.2%
3.b. Evidence of SWG activity	36.4%	12.1%	6.1%	45.4%
4. Establish roles and responsibilities	33.3%		6.1%	60.6%
4.a. Agreement on setting targets, taking action and planning how to achieve these aims	33.3%	–	6.1%	60.6%
5. Collect and record data	36.4%		7.1%	56.6%
5.a. List of data already existing, how regularly	9.1%	21.2%	6.1%	63.6%
5.b. List of data not existing, and which of this data has been collected, how regularly	12.1%	27.3%	9.1%	51.5%
5.c. Number of indicators for which there is data	30.3%	9.1%	6.1%	54.5%
Transformation	16.7%	5.6%	5.6%	72.2%
	22.2%		5.6%	72.2%
6. Analyse results	16.7%		6.1%	77.3%
6.a. Short term prioritisation	9.1%	9.1%	6.1%	75.8%
6.b. Setting a short-term action plan	12.1%	3.0%	6.1%	78.8%
Exploitation	5.6%	2.8%	5.6%	86.1%
	8.3%		5.6%	86.1%
7. Enable ongoing development and continuous improvement	18.2%		9.1%	72.7%
7.a. Medium-long term sustainability plan	18.2%	–	9.1%	72.7%

Knowledge acquisition

The documents containing case studies on tourism indicators show that there have been substantial efforts made to generate awareness of ETIS and its contribution to tourist destination governance. Destinations that participated in the pilot phases showed greater awareness of ETIS than those that did not. Only 45% of the case studies reported that they had identified a local destination coordinator (ETIS step 1.a). In several instances, a university took the role of project coordinator whilst in other cases it was the responsibility of the destination tourism board or tourism observatory. We found that 48% of the destinations reported making multiple efforts (some through public events and others through closed meetings in which only the key tourism players were invited) to communicate decisions to stakeholders (ETIS requirement 1.b), and some reported on the shortcomings of not sufficiently engaging stakeholders from the outset. The process of gaining political support (ETIS requirement 1.c) was announced publicly in destinations like Sardinia (Italy) and the Pallars Jussà County Council or the Barcelona Province (Spain), but these were the exception (only 27%) rather than the rule. Although in 2016 (see Table 1) destinations anticipated minimal difficulties in creating a destination profile (Step 2.a), the latest documents analysed indicated that only 33% were able to fully complete this step. Our workshop showed how the main knowledge acquisition challenges related to the ability of DMOs to gain information to measure basic sustainability principles, and to make methodological developments to capture more complex data in cost effective ways (see Table 3).

Table 3. The design of sustainable tourism indicators.

Challenge	Identified action	Importance (5= very important, 1= not important)
Knowledge acquisition: Capture basic information	Use of available Open Data	4.5
	Design and implement estimations of already existing data	3.9
	Make use of already existing (public and private) records for the local level on sustainability data	3.7
	Use of georeferenced data	3.7
	Design and implement new surveys	3.1
	Investment on Big Data	2.8
Knowledge acquisition: Methodological development	Design an indicator system that is sustainable over time	4.7
	Co-design methodologies with both producers and users of indicators	4.7
	Address geographical scale of the destination	4.5
	Use spatial data	3.8
Knowledge assimilation: Reconciliation of producers and users of data	Capacity building to use data for decision making	4.3
	Capacity building of data producers to communicate data in a user-friendly manner	4.2
	Consensus among producers and users of the aim of measuring sustainability – coherence with the tourism sustainable strategy	3.9
	Capacity building to media on how to interpret data	3.3

Source: authors, from MITOMED + Interreg project, Malaga (Spain) April 4-6 2018.

Our workshop and interviews showed how key staff personally committed to sustainability and employed by DMOs are increasingly developing routines to scan information in cost-effective ways. There were few examples of destinations developing comprehensive indicator sets, such as Diputació de Barcelona (Spain), as ETIS may have expected. However, DMOs were still learning which useful information sources exist and how to use them, and had mostly developed baseline data for specific indicators as a result of project-specific funding, but not core organisational funding. There was an acute awareness that collecting data can be expensive and the more-proactive DMOs were finding out which information is already produced in-house and how it can be combined with specific additional information needs. They were doing this, for example, by: i) standardising data collected by businesses to improve destination-level monitoring (with an example found in the Burren (Ireland) with the Burren Ecotourism Network, and the Burren and Cliffs of Moher Geopark); or, more specifically, by ii) relating questions in the visitor survey to the strategic objectives of the destination (for example in Istria (Croatia) by the Institute of Agriculture and Tourism Poreč (Croatia)). The funding to implement sustainable tourism indicators was usually external and specifically from EU funds. Less frequently, funds came from the local government (e.g., South Sardinia, Italy), and in such cases, it tended to be for one specific indicator of agreed importance.

The greatest hurdle at this point was learning to collaborate with other stakeholders to agree on what are valid data, how to interpret the data, and what implications these data have for subsequent actions. Data collection seemed to work best when it had a specific purpose in mind, for example: i) to create a culture of dialogue, trust and collaboration (e.g. Skyros, Greece; Samaria National Park, Greece; Green Scheme, Slovenia); ii) to capitalise on the collaboration momentum (Visit South Sardinia, Italy); iii) to support fundraising and public support proposals (e.g. Majjistral National Park, Malta) and vi) to create the pre-conditions to develop a destination management organisation (e.g. National Agency for Protected Areas, Albania). The Alqueva region (Portugal) combined all these purposes in the ETIS implementation project as part of a strategy to promote a new destination based on Dark Sky activities. By 2009, Alqueva had already launched a strategy including a working group on tourism indicators that brought together 26 public and private organizations as well as tourism, economic and agricultural experts to evaluate several systems of indicators on sustainable tourism. Thus, in September 2012, this earlier experience enabled Alqueva to begin implementing the ETIS successfully. Its efforts and results were rewarded as “ETIS Social and Cultural Impact Achievers” at the ETIS and Accessible Tourism Awards 2016. The Alqueva example demonstrates how the process of developing an organisational culture of gathering data takes time.

Assimilation

We found multiple examples of destination consortia that adapted ETIS for their own purposes. The Interreg MITOMED + consortium adopted 33 out of the 43 core ETIS indicators, according to the needs of regions and municipalities in the Mediterranean. Green Destinations of South East Europe adopted the complete ETIS methodology and set of indicators. The European Environment Agency’s Tourism and Environment Reporting Mechanism relied on EUROSTAT databases and were deliberately mapped against ETIS indicators. In addition, ETIS was often mentioned as the starting point for raising awareness within DMOs about the use of indicators, who went on to adopt either methodologies like Green Destinations (L’Estartit, Spain), joined likeminded destination associations like NECSTouR (Alqueva, Portugal), or the UNWTO International Network for Sustainable Tourism Observatories (Croatian Sustainable Tourism Observatory, Croatia).

While destinations often expressed concern about a wholesale application of ETIS, most case studies were not able to implement ETIS completely, thus they chose to adapt ETIS to their needs and resources (e.g. Soomaa, Estonia acknowledging the ETIS flexibility by adding wilderness indicators). We found that 61% of the destinations formed a Stakeholder Working Group (ETIS requirement 3.a.). It is unclear how operational these working groups have been over time, but at the time of writing the different case studies, there was evidence of operational activity (ETIS requirement 3.b.), for example, 48% of the case studies reported meetings and action lists, or provided results in the form of presenting the ETIS Toolkit, reviewing progress of the dataset implementation, presenting rough results from the dataset, and analysing the results in detail. The quality of the evidence of activity of these groups varies substantially. In 33% of the cases, groups’ roles and responsibilities were established, such as agreements on setting targets, preparing action lists and planning how to achieve these aims (ETIS requirement 4.a). Some groups allocated every indicator to a given stakeholder, while at other destinations, some stakeholders did not want to assume roles without funding being allocated to them.

ETIS requires destination stakeholders to collectively agree on which data to collect and how to collect and record it. This agreement starts with preparing a list of data that already exists (ETIS requirement 5.a). Destinations are then encouraged to list deficient or missing data, to plan how missing data can be collected (ETIS requirement 5.b), and to finally list the number of indicators for which there is data (ETIS requirement 5.c). Many destinations struggled with this phase (36%), both because existing tourism data were scarce and because data to be collected (e.g. through surveys) were costly and time consuming. These challenges were highlighted by the workshop participants who went on to identify specific actions to capture basic information, as well as how to design specific methodologies to collect more complex data, in cost and time effective ways (see Table 3).

There is limited evidence of DMOs having developed the skills to collect and analyse information about tourism impacts for themselves, beyond economic data on visitor arrivals and expenditure (City of Amsterdam, Holland; City of Barcelona, Spain; Junta de Andalucía, Spain). Few examples of assimilation came from the reuse of existing data for the purposes of understanding sustainability of the current business model, such as the Institute for Tourism in Zagreb (Croatia) and Almuñecar (Spain). However, there were also no examples of citizen-generated data (data that individuals or their civil society organisations produced to directly monitor, demand or drive change on issues that affected them), or data generated from the private sector, that contributed towards a destination-wide sustainability indicator system. It was typical for DMOs to outsource the data collection and analysis to a local university (e.g. East Macedonia and Thrace, Greece; Visit South Sardinia, Italy; Montecatini Terme, Italy; Institute for Tourism Zagreb, Croatia; Diputació de Barcelona, Spain).

Transformation

There was limited evidence of tourist destinations reaching the stage of transformation. Only 22% reported the introduction of organisational changes in working practices that resulted from collecting sustainable tourism indicators (cf. Nawaz & Koç, 2019). In particular, we did not gather evidence that the work on ETIS led to the short-term prioritisation of actions (18% reporting on ETIS step 6a), or the setting of a short-term action plan (15% reporting on ETIS step 6b). There was consensus amongst interviewees and workshop participants (see Table 4) that a tourist destination’s long-term success depends on the ability of its stakeholders to use evidence to reach consensus about the need to change and take informed decisions. Destinations suffering from overtourism were found to be more likely to develop tools to promote consensus (for example the City of Barcelona, Spain; the City of Amsterdam, Holland). Destination stakeholders, led by their DMO, aimed to combine their existing knowledge and organisational routines with the newly acquired knowledge from evidence, such as indicators.

Table 4. The use of sustainable tourism indicators.

Challenges	Identified actions	Importance (5= very important, 1= not important)
Knowledge transformation: Make organisational changes to optimise the management of sustainability	Create audit reports and improvement plans as result from the indicators.	4.5
	Create a participatory entity with different stakeholders, e.g. local economic cluster. Get the destination staff to redesign and adapt the system.	4.4
	Re-design the role of the DMO from marketing towards management	4.3
	Create a sustainability dashboard with easy traffic light system. Make data simple, pre-analyse it (provide numbers that are digested).	4.3
	Value the role of data development. Create ownership of data. Get government buy in to approve action plans and budgets.	4.1
	Upstream delegation. Use the locally collected data to push issues at national agendas and national institutions.	3.9
	Delegate the sustainability management process to specific business unit or separate NGO.	3.2
Knowledge transformation: Introduce sustainability measures based on evidence from indicators	Benchmark: Compare with other destinations	4.2
	Develop trust amongst stakeholders for the implementing body.	4.1
	Cost: Find way of making the measure cost-neutral or for cost to be absorbed within general budgets.	4.0
	Create new products that act as demonstration examples.	3.9
	Buy in: Identify an issue that most stakeholders agree is important at the same time.	3.9
	Relevance: Learn to make indicators audience-relevant.	3.8
	Timely: organise events that highlight experiences from other destinations and that get your organisation to have to showcase their own work.	3.8
	Time: Allocate expert input to manage the action.	3.7
Knowledge exploitation: Upscale from an individual sustainability measure to a complete sustainability strategy	Skills: reskill staff to understand what sustainability is and how it affects their job.	4.5
	Merge sustainability and quality criteria. Integrate sustainability indicators in broader policy debates.	4.5
	Co-production: improve communication amongst departments.	4.2
	Ensure that sustainability is written within the legal framework and or the organisational objectives.	4.1
	Values: change what is considered important.	4.1
	Link one’s own indicators to Sustainable Development Goals, Green Destinations, ETIS, global foot printing/supra-regional programme for justification, funding and framework of understanding.	3.9
	Create index for overview of sustainability in the destination.	3.7

Source: authors, from MITOMED + Interreg project, Malaga (Spain) 4–6 April 2018.

Our interviews showed that using indicators to identify the need for change was one of the hardest things that a DMO needs to do. It was tempting both for politicians and for civil servants to ignore information that contradicts preconceived ideas and current organisational routines, which for DMOs are framed by visitor numbers and expenditure. Tourism policies rely on collaborations and so the individuals within a DMO who were spearheading the use of sustainable tourism indicators regularly had to explain to their team and their stakeholders why change is needed, and what the risks of both changing, and not changing, are. As identified by workshop participants (see Table 4), this could be achieved, for example, by developing trust amongst stakeholders towards the implementing body; creating government ownership of data; identifying issues that most stakeholders agree are important; developing a sustainability dashboard with easy traffic light system that is able to compare data with other destinations (e.g. the European Commission’s Virtual Tourism Observatory platform requested by pilot destinations at the “ETIS and Accessible Tourism Awards”). Indicators were used as a diagnostic and management tool, for example, to: i) target markets with lower seasonality, higher expenditure and more sustainable behaviours (e.g. Tourism and Economic Development Directorate, Ministry for Gozo, Malta); and ii) plan and manage the land use deriving from tourism (e.g. Sea Expert Azores, The Azores, Portugal). Indicators were useful to provide non-partisan evidence to build trust and momentum (e.g. Samaria National Park, Crete, Greece) and to allocate funding to collectively agreed priorities (e.g. Durbuy, Wallonia, Belgium).

It was rare to find examples of how DMOs had changed their organisation to optimise the management of sustainability. This required DMOs to have the capability to reorganise their responsibilities and reallocate resources according to changing requirements that become evident through their organisational learning. There are cases of current departments having increased their remit to now include further data sets that can be interpreted in relation to sustainability. For example, the Public Enterprise for Tourism and Sport Management of Andalusia (Spain) prepared destination scorecards that were adjusted to the objectives of the different tourism plans developed in the region; this process educated destination managers to continuously monitor tourism planning through indicators. We saw the creation of additional roles and structures, such as the Observatory of Tourism in Barcelona (Spain) that was created to coordinate the statistical entities that generate tourist data of Barcelona at municipal and provincial scales. Providing homogenised and quality tourism data, the Observatory fostered an interrelated tourism vision that facilitated coordinated decision making between different management entities, thus improving the articulation of tourism products and the management of information and impacts. It was rare to have cases where there was substantive change to the current structures, or where budget allocation for marketing had been reallocated to destination management. The city of Amsterdam was a clear example where, after the unexpected increase in tourism resulting from the marketing campaign “I Amsterdam”, the city’s government decided to reallocate marketing funding to planning and management of tourism impacts with a particular focus on crowd management.

Most interviewees agreed that goodwill is the result of these institutional representatives demonstrating that: i) stakeholders’ opinions genuinely matter; ii) change happens and iii) the community collectively benefits from change. Stakeholders trust the person representing an institution more than the institution itself; to be effective, therefore, stakeholder management requires a personal approach (Samaria National Park, Crete, Greece; Sea Expert Azores, The Azores, Portugal) and substantial amounts of time (Durbuy, Wallonia, Belgium; Tourism and Economic Development Directorate, Ministry for Gozo, Malta). Responsiveness by a solitary employee does not equate to organisational transformation, though, as the cases identified are spearheaded by committed individuals who often struggle to get broader organisational commitment, as seen in the challenges outlined in Table 4.

Exploitation

The majority of the published case studies reflected the early stages of tourist destinations implementing ETIS or using ETIS to develop their own organisational structures, and did not make any reference to exploitation of the data gathered nor actions taken at destination level in order to be more sustainable and competitive. In relation to processes leading to exploitation, few destinations (18%) made reference to developing medium-long term sustainability plans that explicitly used sustainable tourism indicators (ETIS Step 7a), and these references were mostly declarations of the importance of doing so, rather than concrete examples. Destinations such as Soomaa (Estonia), St Ives (UK), St Tropez (France), Alqueva (Portugal), Poreč and Zagreb (Croatia), Alexandroupoli and Keramoti (Greece) were at an early stage of their ETIS piloting/implementation, and no concrete benefits had been achieved at that time. South Sardinia (Italy) stood out from the others due to its declaration that some actions were taken as a result of the meetings held, namely: i) the adoption of an urban development plan in all five municipalities; ii) the establishment of a marine protected area; iii) the increase of cycle paths in Cagliari and iv) the establishment of a sustainability and environmental education centre to increase sustainability awareness in tourists and locals alike.

Few destinations made reference to actual performance benefits that resulted from new decisions taken by DMOs in response to sustainable tourism indicators generally or ETIS in particular. The case study from Terrae Anio Iubensanae in Lazio (Italy) reported that the new brand allowed the creation of innovative products that matched their identity to the right tourist segment. The development of the new products improved competitiveness and drove a 5% increase in visitors. Similarly, in the case of Slovenia’s Green certificates, 15 destination coordinators participated in their June 2015 training, which generated peer pressure to prove which the greenest destination was.

Our interviews contained few instances of case studies that demonstrated that sustainability increased the competitiveness of a destination or that sustainable tourism indicators contributed towards that change, a key premise behind the roll out of ETIS by the European Commission. Cases tended to be issue specific, data were often not published, and actions were hard to attribute to early participation in ETIS. For example, the municipality of Torroella de Montgrí-L’Estartit (Spain) used evidence that tourism activities in a Marine Nature Reserve generated 12 million euros per year, and more than 200 direct jobs, to lobby the regional government for resources to create a sense of local pride, and to discourage less sustainable forms of tourism development, for example, by protecting land from development. Similarly, the municipality of Sant Llorenç des Cardassar (Spain) used data to promote incentives to manage the cost of waste management. Tripling recycling in hotels saved 800,000 euros for the hotels and 2 million euros for the municipal government. While interviewees noted the benefits of introducing evidence on one particular issue for which they reached stakeholder consensus to act, they also acknowledged the difficulties in using such opportunities to generate systematic change at institutional level that might lead to a holistic sustainability strategy.

Few destinations developed their own institutional programmes to calculate and benchmark destinations using sustainable tourism indicators. Green Scheme (Slovenia), for example, used indicators to get multiple destination stakeholders, including DMOs, to use indicators in order to change policies on tourism marketing and visitor flow management. Diputació de Barcelona (Spain) developed a system for tourism businesses to develop their sustainability pledges and to use local action groups to support each other in meeting these pledges, with the backing of an independent certification programme. Terrae Anio Iubensanae (Italy) made a census of their destination’s resources to design innovative products, because small municipalities alone did not have the know-how by themselves; this allowed them to create routes to increase demand to lesser known attractions and to promote enterprising local businesses. What these programmes have in common is how collective systems provide the knowledge, the peer support/pressure and the momentum to become catalysts for positive change.

Interviewees reported that politicians were willing to use data to support a policy decision that was already approved. However, there was less willingness to acknowledge data that contradicted policy, dismissing the data as not definitive, not relevant to that particular location, or as an unintended, but necessary, social or environmental cost of an otherwise seemingly beneficial policy. Hence, the main exploitation challenge identified by participants in our workshop was to upscale from an individual sustainability measure to a complete sustainability strategy (see Table 4). Interviewees reported that this would require: i) a change of organisational values; ii) better understanding of how sustainability permeates into staff roles and better cross-department communication and iii) the inclusion of sustainability as part of the core organisational objectives, legal frameworks, external standards, overall measurement indexes and the definition of quality.

Discussion

ETIS has provided a practical tool, a political reference point and a peer-group momentum that has contributed to a shift in DMOs’ understandings of the scope of their responsibilities towards more actively planning for and managing tourism (McLoughlin et al., 2018; Zabetta et al., 2014). Having said that, the results of our mixed-methods research confirm that the process of policy change is slow and that, to understand and influence policy making, we need to study how knowledge, in this case embedded in sustainable tourism indicators, is co-produced, shared and then absorbed (Pee & Kankanhalli, 2016). Our study supports the premise that ACAP is a useful tool to examine the black box of how DMOs gather and use knowledge in order to make policy decisions. Breaking down the organisational routines and processes of knowledge acquisition, assimilation, transformation and exploitation helps identify opportunities and barriers for DMOs to develop their dynamic capabilities (Zahra & George, 2002).

We found a wide range of tourist destinations that have acquired knowledge about sustainable tourism indicators, and ETIS in particular. Stakeholder communication is necessary to have a better understanding of the importance of sustainable tourism indicators (Cannas, 2019), as the first stumbling block often found in evidence-based policy making is stakeholder agreement about what counts as evidence (Nutley et al., 2003). Our results confirm that the completeness and comparability of tourism data is still poor, at European and world levels (Mazanec et al., 2007), for monitoring both destination competitiveness (Crouch, 2011) and sustainable tourism indicators (Buckley, 2012). There is huge variation in the amount of adopted sustainable tourism indicators that can be obtained within each destination’s statistical system, though, ranging from 75% in Portugal (Farinha et al., 2019) to 40% in Italy (Zabetta et al., 2014) and 16% in Romania (Tudorache et al., 2017). Economic data are usually readily available, while environmental and social impact indicators are the hardest to collect (Modica et al., 2018). In this sense, ETIS has been a useful self-assessment tool (Zabetta et al., 2014) that has given several of the study’s destinations a valuable reality check concerning data resources.

The importance of social and cultural sustainability is less understood by some stakeholders than the importance of preserving natural assets that are tourist attractions, or the imperatives of income and job creation (Cannas, 2019). Absence of dedicated resources for data collection leads to prioritisation of economic indicators, collected annually, while tourist, resident and business surveys are conducted typically every two to four years (see also Farinha et al., 2019; Modica et al., 2018; Tudorache et al., 2017). The study by McLoughlin et al. (2018) shows the complexity involved in setting up a system for data collection from multiple sources that all feed into a single dashboard. For example, some ETIS indicators could be calculated from data available from water and waste collection services, or by estimating carbon footprint based on distances travelled and formulas provided by ETIS. Other information required purposefully collected data from resident, visitor and enterprise surveys (McLoughlin et al., 2018).

The ability to assimilate knowledge goes hand in hand with the ability to adapt this knowledge (in our case the ETIS indicators) to the realities of the destination, in terms of data availability and information usefulness in relation to stakeholders’ needs (Farinha et al., 2019; Modica et al., 2018; Tudorache et al., 2017). The ability of DMOs to adopt indicators depends on their capacity to learn how to choose what information is important in their destination and how that data can be organised in a way that it can become a useful set of indicators. The success of a DMO does not depend on the quantity of the information collected, but on their capacity to understand what this information means in relation to the functions that the DMO currently has, or should have. Indicators have been used to raise awareness and gain consensus about the importance of certain impacts, but not to create a destination governance mechanism nor to introduce sustainability as a priority for the mechanisms in place (McLoughlin et al., 2018; Modica et al., 2018). DMOs that have assimilated the essence of what ETIS stands for have identified a clear purpose for the data collected. This requires having the skills to process data so that it can be readily understood, interpreted and distributed to the relevant stakeholders; only then have those DMOs transformed data into valuable indicators. DMOs have limited capacity to engage with important questions about sustainability that cannot be answered with the currently available data. Currently available data only responds to pre-identified problems framed in pre-conceived priorities, but does not engender the dynamic capabilities required to rethink what is important. Following available data, economic priorities will prevail over social and environmental impacts (Lozano-Oyola et al., 2012; Torres-Delgado & Palomeque, 2014).

EIPM, be it through ETIS or other knowledge sources, requires institutional support (Cannas, 2019; Modica et al., 2018; Nutley et al., 2003). While ETIS provides momentum to get multiple stakeholders within a destination to consider the need for better governance and to get local stakeholders to work together to talk about the sustainable use of local resources (Zabetta et al., 2014), it does not always ensure that these stakeholders have a sense of ownership of the process. Community-based monitoring, applied to ETIS, empowers local communities to inform evidence-based policy (Zabetta et al., 2014), but this community monitoring is rare. Data collection outsourced to universities kickstarts a process but does not lead to knowledge assimilation, and does not increase organisational capability in the DMO. For example, ETIS has been used to inform the design of an IT-based, destination participatory Group Decision Support System with an algorithm-based GIS system to benchmark and visualise indicator data (Tudorache et al., 2017) that may become too complex for stakeholders to understand. It is important that ETIS be used to create a decentralised service landscape, with shorter feedback loops, to increase the number of potential users of evidence and providers of feedback (Rutter, 2012).

While DMOs’ networking capabilities have been proven to increase their legitimacy amongst stakeholders (Volgger & Pechlaner, 2014), they have not been used to show how DMOs learn through these networks. Zabetta et al. (2014), report that it is not clear how a destination can use an ETIS diagnosis to introduce initiatives to improve sustainability and competitiveness. Hence, DMOs typically do not transform their organisational structures in response to the newly acquired knowledge, when they are found to lack the willingness to accept the mandate for destination management (Dredge, 2006a). For example, although the municipality of Barcelona has increased its efforts to promote sustainable tourism, it has not affected how Turisme de Barcelona, the public-private DMO, governs itself (Serra et al., 2017). For examples such as this, ETIS would provide the chance to create a flexible management framework that would help progress sustainable tourism indicator data into affirmative action (Twining-Ward & Butler, 2002).

An organisation has developed dynamic organisational capabilities in relation to knowledge exploitation only when it routinely and systematically exploits evidence (Zahra & George, 2002). Academics have examined processes of sustainability data collection (McLoughlin et al., 2018; Modica et al., 2018) but have not analysed policy making informed by the indicators. While gathering data is a technical task, the evaluation of such data for decision-making is political (Zabetta et al., 2014). ETIS stakeholders want to see how the public sector takes policy decisions based on data collected through the ETIS process (Zabetta et al., 2014), as the influence of evidence on policy is balanced against other factors, such as, inertia, ideology, and financial considerations (Dredge, 2006a; Walker, 2000), which slow down and dilute the impacts of decisions. Change is slow, and the explanation behind how DMOs acted on ETIS usually lies in their long track record of sustainability actions.

ETIS indicators have been used to legitimise decisions already taken but not to create the level of change expected. While ETIS evidence has the potential to inform tourism planning (McLoughlin et al., 2018), this has often not materialised. For example, legally required, five-year environmental plans in Ireland included tourism in 93% of cases, however, they lacked “robust and detailed tourism policies, strategies, budgets and guidelines” to facilitate their implementation (McLoughlin & Hanrahan, 2016, p. 33). Ireland also has developed a comprehensive, sustainable tourism indicator system, endorsed by the national tourist board, that remains unused (McLoughlin et al., 2018). Not a single local authority included sustainable tourism indicators (including those of ETIS) to underpin such plans.

Although ETIS is a useful tool to analyse destinations with objective parameters and to monitor their progress towards a more sustainable development, the European Commission has not continued to support its implementation since it launched the toolkit (Tudorache et al., 2017). After two pilot phases, only a handful of destinations, with strong political commitments and/or financial funds, have been able to keep on producing indicators informed by ETIS. Most destinations have adapted the original ETIS indicators to their own objectives and statistical sources. Because the European Commission did not fund a secretariat or the development of an online global database to compile and benchmark data, the destinations interviewed have searched for alternatives to benefit from the lessons learned. They have developed their own open platforms to share and compare tourism indicators (MITOMED+) and adopted alternative methodologies (primarily Green Destinations and the Global Sustainable Tourism Criteria for destinations). Hence, the real value of ETIS has been the raised awareness of the need to collect sustainability data amongst politicians and DMO civil servants, and not the 43 European sustainable tourism indicators in the toolkit. ETIS has contributed to understanding the need to have indicators that go beyond the volume and expenditure of tourists to design evidence-based policies.

Conclusions

This study makes a theoretical contribution by showing how ACAP can be used to explain the challenges in using indicators to inform policy. In doing so, it contributes to the nascent application of ACAP and knowledge management to public sector organisational effectiveness (Harvey et al., 2010; Pee & Kankanhalli, 2016; Richards & Duxbury, 2015). We provide ample evidence of DMOs acquiring knowledge about the importance of sustainable tourism indicators through ETIS, and how a number of these DMOs assimilate this knowledge by adapting ETIS and developing their in-house systems. Documentary analysis, interview data and focus groups provide few examples of DMOs, or their policies, being transformed as a result of the use of sustainable tourism evidence in the form of indicators or other sources of data. We find even fewer instances with tangible evidence of how indicators are exploited to improve tourism sustainability and competitiveness. The use of ACAP to study the innovation process of public sector agents is a novel approach in policy science that provides a nuanced understanding of how sustainability evidence is used (or not) in the policy process.

This study also has practical implications. We aimed to evaluate the impact of sustainable tourism indicators on destination competitiveness, based on the ETIS scheme funded by the European Commission to address the evidence gap in tourism policy making. The Commission’s “Agenda for a sustainable and competitive European tourism” (Estol & Font, 2016) announced a series of policy interventions that were supposed to transform sustainable tourism policy making. Evidence shows how the process of creating impactful change is considerably slower than rational approaches to policy making might expect, and that institutions require a considerable amount of time to adjust to new information. The main benefit of implementing ETIS in a destination is not the data itself, but the creation of social capital amongst destination stakeholders (Tudorache et al., 2017), which should lead to changes in policies that, in due course, make destinations more competitive. This study shows that a single intervention like ETIS needs to be understood in the complex context of how DMOs operate, and that for any intervention to permeate, it is necessary to provide a more sustained and coordinated programme of activities. In the meantime, this review of the impact of ETIS has led to the European Commission contracting the research team to write an Impulse paper (REFERENCE TO BE ADDED AFTER REVIEW) to inform future actions from the European Commission to promote measurement methods of sustainability in tourist destinations.

There are limitations in an exploratory study like this one, that lead to further research opportunities to study of under which conditions some of these insights hold true. A more nuanced understanding of the complex mechanisms that allow and impede DMOs to progress in the four stages of ACAP would allow policy makers to fine tune programmes to promote behaviour change (Volberda et al., 2010). Our study has also focused on how DMOs are implementing and using indicators. However, this is only a partial understanding of how tourist destinations operate, and it is important to apply ACAP to study network learning (Knight, 2002), i.e. the ability of whole networks of organisations to learn as a group. The unit of analysis in subsequent research ought not to be the DMO but the ability of destination stakeholders to work collaboratively towards a common goal. Although stakeholders perceive that successful DMOs lead to successful destinations (Bornhorst et al., 2010; Volgger & Pechlaner, 2014), more emphasis needs to be placed on the study of participatory processes for sustainable tourism governance, including the use of sustainable tourism indicators (Castellani & Sala, 2010; Dredge, 2006b). The success of public sector knowledge management depends on their capacity to share and apply knowledge gained through their networks (Dawes et al., 2012) and not just to gain authority by participating in such networks (Volgger & Pechlaner, 2014). Hence, we need to study ACAP in relation to collaborative management and multi-organisational working (Harvey et al., 2010). Further research needs to consider the public sector’s coordination and socialisation capabilities of sustainable tourism evidence (Jansen et al., 2005; Volberda et al., 2010).

Disclosure statement

No potential conflict of interest was reported by the authors.