The politics of strategic environmental assessment indicators: weak recognition found in Chinese guidelines

Abstract

The choice and use of indicators is not only technical and science-led, but also a value-laden social process, and thus concerns public participation and political judgement. This article approaches the Chinese strategic environmental assessment (SEA) indicator system from a science–policy interface perspective and aims to: (1) contribute to the general recognition of indicators functioning in SEA; and (2) analyse, through a Chinese case study, to what extent national guidelines address this science–policy interaction. The overall finding is a strong emphasis on technical/science aspects in the Chinese SEA guidance, and a weak explicit recognition that policy plays a role in choosing and using indicators. Recent development, however, indicates a growing recognition of the politics involved and thus also leads to greater involvement of stakeholders.

Keywords:

• SEA
• indicators
• guidelines
• science–policy interface
• China

Introduction

Strategic environmental assessment (SEA) aims at ensuring that potential environmental impacts are identified and considered in decision-making and this integration of the environmental consequences occurs at the earliest possible stage of the decision-making. One way of assisting this process is using indicators for representing environmental conditions, predicting and measuring impacts and communicating with relevant stakeholders. By identifying critical issues, indicators simplify the complex reality of a situation into easily communicable signs. Indicators provide information in a ‘simpler, more readily understood form than complex statistics or other kinds of economic or scientific data’ (Hammond et al. 1995, p. 1). They support decision-makers (Cloquell-Ballester et al. 2006), aid communication (EEA 2005) and can increase transparency to stakeholders (Helbron et al. 2011). Further, identification of appropriate indicators can secure a holistic understanding of the impact of planning on a large space and time scale (Haughton et al. 2009). Indicators have to be carefully selected as they influence ‘what baseline data are collected, what predictions are made and what monitoring systems are set up. Poorly chosen ones will lead to a biased or limited SEA process’ (Thérivel 2004, p. 76). Indicators are also studied from the perspective of SEA guidelines and frameworks (Fischer 2006).

The point of departure in this article is that indicator development and use are always found at the interface between science and policy, which is defined as ‘social processes which encompass relations between scientists and other actors in the policy process, and which allow for exchanges, co-evolution, and joint construction of knowledge with the aim of enriching decision-making’ (Van den Hove 2007, p. 807). The scientific process relates to technical components such as theoretical considerations like cause–effect relationships and measurability, data structure aggregation and availability (Fischer et al. 2010). The political process, for legal issues, political considerations and by public request (Fischer et al. 2010), relates more to the communicative aspects, formally or informally, and the questions of whether to use indicators, which indicators to use, consumption of time and financial consumption, information aggregation level and who is involved at certain stages of the SEA process. It thus involves both personal and political values.

The article looks into the handling of the science–policy interface in Chinese SEA guidance. The case of China is a choice motivated by new guidance development taking place, which aims to respond to the country's rapid economic growth, especially within the energy and urban development sectors, as a consequence of which the Ministry of Environmental Protection of China (MEP) has drafted a new version of guidelines with more sector-oriented guidance.

The Chinese SEA indicator system

Indicators are intensively used in Chinese SEA practice. The Technical Guidelines for Planning Environmental Impact Assessment 2003, hereafter referred to as the Technical Guidelines (2003) was launched on 1 September 2003 (The State Environmental Protection Administration of China – SEPA (now renamed MEP 2003). Planning-EIA is the term used currently in China to refer to SEA. The Technical Guidelines (2003) provide a recommended procedure to guide practitioners in identifying environmental objectives and indicators, and recommend six sets of indicators for SEAs in energy, regional planning, urban development, agriculture, land use and industry planning. After four years of practice, the Technical Guidelines (2003) were subjected to reflection and improvement. In 2007, the former SEPA launched a committee to revise it. In 2009, a draft-revised version (hereafter referred to as the Technical Guidelines (2009)) (MEP, 2009a) was prepared by MEP. The Technical Guidelines (2009) consist of one general and five sectorial guidelines. They provide five sets of recommended indicators for coal mining, urban master planning, forestry, onshore oil and natural gas exploitation and land use planning. The revised version emphasizes the core role of environmental objectives and indicators in SEA. However, guidance on how to involve different values when choosing indicators or who should be involved in this process is still missing. The guidelines are currently still being revised (except for one already published).

Looking at the Chinese context, it appears that the political side of devising and using indicators has received limited scholarly attention. Zhao et al. (2003) point out that current SEA research focuses on ‘how to assess’, while indicators concern ‘what to assess’. Bao et al. (2001) discuss the principles for classifying SEA indicators and propose a method for weighing indicators. Xu (2009) discusses how to establish a comprehensive index system for SEA by proposing an integrated index model. Guo et al. (2003) argue that the DPSIR (Driving forces, Pressures, States, Impacts and Responses) model is useful for simplifying the complex relationship between society and environment and provides a basic framework for indicator use. By contrast, Fan and Zhou (2008) claim that the DPSIR framework oversimplifies cause–effect chains, and therefore indicators should be adjusted according to the context to better reflect the complex reality and to use indicators effectively. Guo et al. (2003) point out that most SEA indicator studies have been limited to a general level without much guidance for practice. Tao et al. (2007) criticize the Technical Guidelines (2003) as ‘an extremely general process and lack[ing] a detailed procedure’ (p. 255), and therefore sectoral guidelines are necessary. The Centre of SEA at the Chinese University of Hong Kong surveyed the ‘Effectiveness criteria for PEIA in China’ in 2009–2010 (CSEAC 2010, draft) and suggest that one of the best practice criteria for improving effectiveness is to select indicators for assessment objectives during scoping.

The use of SEA indicators is receiving more attention in China, which is clearly reflected in the current revision of the Chinese SEA indicator system. The primary aim of the article is to contribute to the understanding of how Chinese SEA guidance handles both the scientific and the political sides in selecting and using SEA indicators, and whether any mediation of science–policy interaction is involved.

SEA indicators at the science–policy interface

Owing to the complex nature of the environment and society, SEA practitioners face a number of difficulties when designing appropriate indicators (Scholes and Biggs 2005). SEA indicators contribute in many ways to the linkage between science and policy (Turnhout et al. 2007). Although this is generally perceived as a positive linkage, it is not without problems. It must be recognized that developing and deciding upon indicators involves the interests, needs and values of the involved stakeholders. If the political and value aspects are neglected and the focus is instead only on technical aspects, the opportunity to benefit from a close relationship of using indicators and decision-making might be missed. Cloquell-Ballester et al. (2006) suggest that all decision-makers and stakeholders should agree on indicators in the earliest stages of SEA. This helps ensure that objectivity and transparency and indicators design can positively affect participation levels of the general public, experts and decision-makers. Kurtz et al. (2001) argue that the complexity in choosing and using indicators invites different actors to be involved, thus opening up new interpretations of indicators' development in SEA process. Donnelly et al. emphasize the inclusiveness in the selection process (Donnelly et al. 2007, in Fischer & Onyango 2012). Through a workshop-based approach, they try setting up a multi-disciplinary team to develop criteria for SEA indicator selection (Donnelly et al. 2006) and test how indicators could be selected (Donnelly et al. 2008).

While the authors above all indirectly touch upon the politics of indicators, others directly stress that a political process is involved in creating indicator systems. Bossel (1996), for example, underlined that indicators express values. Turnhout et al. (2007, p. 225) characterized indicators' development as ‘demand-driven, interdisciplinary, uncertain and value-laden’. Levett (1998) emphasized that indicators ‘are inputs to policy as well as consequences of it’ (p. 294) and that the chosen indicators reflect different worldviews. When choosing indicators for environmental sustainability, some are based on scientifically described goals and measurements. Other, more non-instrumental functions of indicators related to decision-making were identified by Gudmundsson et al. (2010). These include ‘providing common reference frames’ and ‘suppressing attention to certain aspects that are not measured’ (p. 29). This more interpretive view of knowledge, which is complementary to natural science models, invites some reflections about the linkage between social learning and indicators: ‘indicators of sustainability will only be effective if they support social learning by providing users with information they need in a form they can understand and relate to’ (Shields et al. 2002, p. 150).

Role of guidance in the interface

The interface between SEA indicators and policy-making is certainly influenced by different contexts and institutional structure. SEA guidance is one important part of the institutional framework. The Chinese guidance is flexible and subject to some interpretation and at the same time creates some stability with regard to both science and policy within SEA processes. We argue that the guidance plays this intermediary role between science and policy, establishing procedural recipes to follow, and thereby defining the interface between SEA and policy-making.

How the guidance on development and use of indicators handles and sustains the interface, acknowledging the political and value-laden dimension, is analysed by looking into the following issues:

	Recognition of the validity of different kinds of knowledge;
	Recognition of the politics and value-laden activities involved in the development and/or use of indicators in SEA processes, and guidance on how to handle these issues in practice;
	Recognition of a science–policy interface, and guidance on how and who to engage in the process.

Methodology and data

The analysis is based upon a documentary study of the national guidelines, interviews with SEA actors, and an online survey.

Document analysis

The two versions of technical guidelines are analysed. The majority of the discussion is based on the Technical Guidelines (2009) consisting of a general guideline and a series of sectoral guidelines:

	Technical Guidelines for Planning-EIA (General principles; 2009, under revision);
	Technical Guidelines for Planning-EIA (Coal Industry Mining Area Plan; 2009, published);
	Technical Guidelines for Planning-EIA (Urban Master Plan; 2009, under revision);
	Technical Guidelines for Planning-EIA (Forestry Planning; 2009, under revision);
	Technical Guidelines for Planning-EIA (Onshore Oil and Natural Gas Field General Exploitation and Development Plan; 2008, under revision);
	Technical Guidelines for Planning-EIA (Land Use Plan; 2009, under revision).

This documentary study aims to establish formal expectations for developing and using indicators in SEA and to assess how science and policy domains are reflected in the guidance. The aggregation level of information in indicators (explained in another study) is identified as ‘Non-aggregation’, ‘Aggregated indicators’ and ‘Complex aggregation indicators’. A Non-aggregation indicator is based upon a single item of information (for example, mg Pb/l). An Aggregated indicator is composed of two or more sub-indicators that relate to different sets of information (for example, mg Pb/kg bodyweight of salmon). A Complex aggregation indicator is composed of two or more sub-indicators with a complex or even ambiguous structure (for example, sustainability of rivers). Complex indicators require interpretation by practitioners. This in turn has implications for the data required for using them.

Interviews

To deepen the analysis, interviews with SEA practitioners, researchers, experts and administrators were undertaken (Table 1). The interviews were inspired by the science–policy interface presented above, and were carried out with loosely structured questions. Questions focused on investigating the political aspect of choosing and using indicators.

Table 1 Overview of interviews.

Interviewee		Time
G01 practitioner and researcher	Professor in SEA	January 2011
G02 researcher and expert	Vice General Engineer, Appraisal Centre for Environment & Engineering (ACEE), MEP
G03 expert, administrator	Director, Department of SEA, ACEE, MEP	February 2011
G04 expert, administrator	Director, Department of EIA, MEP

Survey

An online survey was undertaken between June and August 2012. The survey was designed with ‘SurveyXact’ developed by Ramboll, Denmark. Seventy-five potential respondents, including practitioners, researchers, administrators and stakeholders, were invited of which 46 responded. Of these, two are from government/administration, 24 from consultancy, 18 from academia and two from other institutions. The questionnaire contained questions on (a) how they interpret guidance handling indicators, (b) experience with designing indicators and (c) experience with impacts of using indicators. The first two parts of the questionnaire were designed for this study and the last part was used for other studies of the role of indicators in communication in SEA.

Most of the documents analysed in this article are in Chinese, and so are the interviews. The quotations from documents and interviews were translated by the authors.

Science–policy domains

The science domain: from technical minimalistic to complex indicator systems

According to the Technical Guidelines (2009), ‘the final report shall describe environmental indictors used’ (p. 14). Therefore, indicators are seen as an essential part of the SEA process: ‘environmental objectives are the base of Plan-EIA, and indicators are designed to assess the feasibility and achievability of those objectives’ (p. 8). The official explanation of the revised guidelines explains the important role of indicators: ‘It loudly emphasizes the core role of environmental objectives and indicators in SEA as the most important basis for the whole assessment’ (MEP, 2009b; The explanation for Technical Guidelines 2009, p. 6).

The guidelines emphasize indicators' quantitative nature: ‘indicators should be selected to represent the environmental objectives quantitatively or semi-quantitatively’ (Technical Guidelines 2009, pp. 8–9). Another study looked into the nature of indicators, examining indicators' aggregation levels in the two versions of guidelines (Gao et al. 2013). The comparison shows that the indicators are shifting from relying on lower aggregation indicators (Non-aggregation and Aggregated indicator) in 2003 to higher aggregation and complexity indicators (Complex aggregation indicator) in 2009. The share of ‘Complex aggregation indicators’ (such as Sustainability and Clean Energy Development) increased from 28 to 40% from 2003 to 2009, while the ‘Non-aggregation indicator’ (for example COD emissions) decreased from 25 to 17%. As a consequence of more ambiguous aggregated indicators, informally more discretion is given to the practitioners, and more frequently politics and value-added aspects find their way into the formulation and use of indicators; thus the political side of devising and using indicators is stressed.

Reflecting upon increased aggregation, the online survey shows that 80% of the respondents find that more aggregation of indicators to some extent is positive, as it helps to quantify environmental and social concerns. Among the respondents, 89% would like to have more guidance on the development and use of indicators. Among those, 81% emphasized specific recommended lists for sectoral indicators and 67% wanted better procedures or methods for selecting indicators. Only very few (14%) of the respondents were concerned with who should be engaged in selecting indicators.

The importance of indicators has also been highlighted among the survey respondents. Some 87% of the respondents find indicators useful or very useful in data collection: 96% in assessment and 83% in evaluation and approval. Indicators may simplify and condense the handling of vast amounts of information, therefore better informing decision-making. Thus, indicators are related to the political domain and the communication needs of the SEA process. How the guidance relates to this point is discussed in the following section.

The policy domain: weak reflexivity and guidance

Regarding the official recognition of the political side of developing and using indicators, The Technical Guidelines (2009) are considered to be weak. They neither incorporate statements or discussions about the value-laden elements in the process of choosing indicators nor explicitly reflect upon how indicators influence thinking and the role of values and policies. The survey respondents, conversely, clearly recognize the political aspect: 87% perceive the selection of indicators as both a technical and a political process. However, only 13% think that the guidelines sufficiently address this political/value side of indicators. In addition, quality control of SEA is also partly based upon indicators, embedding a value-laden activity. It is expected that the appraisal committees will evaluate SEA against such guidelines (G01, G03 2011). However, the experience of ACEE is that ‘the committees do not review an SEA against this guideline but mostly rely on personal experiences, which leads to a situation that experts have different understandings of SEA without a common standard’ (G02 2011).

Regarding the specific guidance on how to handle the political aspect and the science–policy interface embedded in indicator systems, the Technical Guidelines (2009) suggest an inclusive selection process and thus indirectly recognize that knowledge production is also a political process: ‘Indicators should be selected relevant to plans in different sectors based on the experts' consultation and public comments collection’ (MEP, 2009b; The explanation for Technical Guidelines 2009, p. 10). In the sectoral guidelines, a similar suggestion is also explicated: ‘The indicators could be selected through plan analysis, experts' consultation and public participation’ (Technical Guidelines for Planning-EIA (Urban Master Plan) 2009, p. 8).

Arguments for public involvement include securing proper scope in the assessment and democracy: ‘A broader public participation can facilitate a more precise evaluation of the impact, reduce the possibility of excluding any themes or issues, and make the decision-making more democratic’ (Technical Guidelines for Planning-EIA (Forestry Planning) 2009, p. 8). However, there is no indication as to what extent participation will influence the final list and what the consequences would be, despite an encouragement for broader participation in selecting indicators. Going beyond the written guidance, the question of how to decide upon indicators in the single SEA case, the response from the MEP was that the basis should be ‘experience from the previous projects, experts' experience and communication with planning sectors’ (G04 2011). Here, inclusiveness is touched upon, although not including the public or politicians. In practice, although the importance of public/non-governmental organization (NGO) involvement was recognized as important or very important by 76% of the survey respondents, very few of them had actually experienced involving the public (70% never/rarely experienced) or NGOs (78% never/rarely experienced) in designing indicators for SEA.

It is further argued that indicator selection is an on-going process: ‘The recommended indicators list should be adjusted or extended during the SEA’ (Technical Guidelines for Planning-EIA (Land Use Plan) 2009, p. 6). The survey results also show that 76% of the respondents select some indicators from the guidance and supplement them with others, while 20% rely only on the guidance. Regarding flexibility, 26% had experienced indicators being selected at an early SEA stage and never changed, and 30% had experienced it as an on-going process. The main triggers for adjusting the chosen indicators during the process are input from politicians (61%) and planning teams (59%), and not so often that from the public and/or NGO's (22%).

However, despite the existence of guidance on stakeholder involvement, indicators are presented in such a way that they seem to be certain and objective. In particular, the lack of explicit recognition and reflexivity upon the subjective and value-laden elements in indicator systems is found to be critical.

Conclusion

This paper presents an analysis of the national-level guidance for developing and using indicators in SEA from a technical–political interface perspective. Regarding the technical aspect, there is a strong demand from decision-makers for using indicators in SEA for condensed information facilitating the setting of goals and objectives, assessing impacts quantitatively and designing monitoring properly. To a certain extent, practitioners have some degree of discretion when selecting indicators, which positively supports the context dependency and indicators development for different purposes and cases. This discretion can be expected to increase owing to the ambiguity embedded in the higher aggregation level of indicators. However, as seen in the light of the overall conclusion that the technical/scientific domain is almost solely addressed in the guidance, how can the bias of experts' professional backgrounds, values and interests be avoided? In this context, Rametsteiner et al. (2011) found that the ‘political norm creation dimension is not fully and explicitly recognized in science-led processes’ (p. 61). The risk is that knowledge, which is more subjective and uncertain in nature, will not be involved in selecting and using indicators unless they are explicitly presented and discussed. By contrast, the formulation of indicators could be biased because professionals could compose indicators in a way that is more in line with their own ideas, or even manipulate processes.

The overall finding is a lack of both recognition and specific guidance on the political and value-laden part of Chinese indicator systems. There is a need for reflexivity and guidance on how to explicitly and transparently deal with both scientific and political processes. By making these processes more comprehensive, both knowledge production and norm creation can be involved in the selection and use of indicators in SEA. Finally, as indicators become widely used in Chinese SEA, and as for many practitioners indicators are useful in public participation and communicating with decision-makers, it is increasingly important to critically examine how they are produced and how the focus of knowledge they create affects decision-making. It seems obvious that many of the problems encountered in traditional planning and SEA theory regarding rationality and decision-making (Kørnøv and Thissen 2000) are surfacing again, albeit now also adding to the picture that power does not only present itself in decision-making and SEA processes but also emanates from the construction of indicators. Some of these aspects should be further elucidated in future work looking more closely into the practices of indicator use in a few Chinese cases.