The state of the art of impact assessment in 2012

Keywords:

• State of the art
• impact assessment
• effectiveness
• theory
• opportunities and threats
• strengths and weaknesses

This special issue of Impact Assessment and Project Appraisal aims to present the state of the art of a number of impact assessment tools. It is timely given that environmental impact assessment is now 42 years old (beginning on 1 January 1970 when President Richard Nixon signed the National Environmental Policy Act in the USA). It updates the last International Association for Impact Assessment (IAIA) overview of the field that was published in 1995 (Vanclay and Bronstein), which preceded the 1996 International Effectiveness Study (Sadler 1996). The Effectiveness Study itself has been updated, but, at the time of press, any conclusions are still pending.

The initial problem for the editors was to decide the appropriate focus for papers, as only six could be selected, which suggests we have made judgements about the relative importance of various types of impact assessment. We decided not to include papers that dealt with generic types of impact assessment, or components of impact assessment, that could be applicable to any process, so cumulative effects assessment was not included, nor was public participation, despite our acknowledgement of their importance. Several of the papers in this special edition do address these concerns in relation to their specific topics, however.

Determining which forms of impact assessment should then be the focus was no easy choice; we considered ecological impact assessment, climate impact assessment and technology assessment among others. We acknowledge that our choices could have been very different, and some readers may not agree with them, but we chose to consider some forms of assessment originally covered by Vanclay and Bronstein for which we were aware there was extensive practice globally, as well as two newcomers that we believe have now achieved this status. So this issue covers the state of the art of environmental impact assessment (EIA), strategic environmental assessment (SEA), policy assessment, social impact assessment (SIA), health impact assessment (HIA) and sustainability assessment, where SEA and sustainability assessment have emerged as significant bodies of theory and practice since the publication of Vanclay and Bronstein (1995). We have left open the possibility of producing a further issue dealing with some of the impact assessment processes we could not consider on this occasion.

Having identified the topics, our next challenge was to identify appropriate authors to be invited to write each paper. We extended invitations to the Chair(s) of IAIA Sections where possible, and to others who have demonstrated leadership in their fields, for example by the publication of recent articles or books. In many cases the invited authors also drew upon the expertise of others; the HIA paper, with its 10 authors from that IAIA Section, represents a particularly collaborative effort. We are extremely grateful to all the authors for their tremendous hard work and responsiveness throughout the 12-month process that is behind this special edition.

In the interests of academic rigour and continuing the spirit of collaboration among a broad range of contributors, we relied on three anonymous peer reviewers for each paper rather than the standard two. All of the reviewers engaged deeply with the material and made significant contributions to the final forms of each paper. We set a word limit of 7,000 words per article, which every paper failed to meet (they are all closer to 8,000 words in length). This is partly because the remit for each paper is very large as we are asking authors to sum up everything there is to know about the state of the art, a topic for which there has been 17 years of practice since the Vanclay and Bronstein (1995) book.

We also considered what the focus of each paper should be. To allow some overview to be taken of the state of the art of impact assessment in general, we tried to standardize the content, asking for:

•	Brief history/evolution of practice/explanation of basis for practice (e.g. why is there practice in this area?).
•	International perspectives (where practised, legislative arrangements etc.).
•	Current strengths and weaknesses of the assessment processes (drawn from practice globally).
•	Current opportunities and threats in relation to external influences on the practice of the assessment process.
•	Where to next?

Although the authors have not necessarily addressed our request by rigidly following a prescribed format, the broad standardization of approach has allowed us to identify common threads through the papers. Reaching this point was not the end of the debate about content, however, because the author of the EIA paper (Morgan 2012) raised an excellent question about how EIA should be defined – with a purely biophysical focus or as an all-encompassing practice embracing a broad range of considerations including social, health and other sustainability concerns, reflecting typical international EIA practice? In light of our common view that it should be the latter, this raised questions about the boundaries between the EIA paper and the others in this special edition. We determined that the EIA paper should have something of a special status, befitting EIA's position as the original form of impact assessment, and that this paper should consider the state of the art of EIA to a large extent as the state of the art of impact assessment itself.

Readers should also be clear that some level of knowledge of the impact assessment processes is assumed – it is not the case that these articles explain the impact assessment processes in detail. Instead, the assumption is made that the reader understands the basic steps involved in the process they are reading about, thereby allowing the authors to build on that basic understanding to update the reader on the extent of practice and on its evaluation. If this basic understanding is missing, the extensive references associated with each of the papers provide a rich source of additional learning, which we would encourage readers to delve into.

We have divided the following consideration of the articles in this editorial into three broad categories: the first deals with topical issues which recur from one paper to another, and are therefore issues which cross over impact assessment areas; the second relates to external issues, including threats and opportunities to further practice of forms of impact assessment; the third relates to ongoing practice areas of concern.

Before presenting the results against each of these categories, it is worth highlighting the influence of the paper authors on their own findings. We noted in particular that our requirement for authors to consider strengths, weaknesses, opportunities and threats caused some difficulty for some authors. Deciding what is a strength or a weakness, for example, is normative in that it depends what the authors think is a good or a poor aspect of practice. Here we would refer readers to the policy assessment paper (Adelle and Weiland 2012) where the authors have referred to the research which has been conducted on the process, and divided this into four types, the first two of which follow a positivist research philosophy, and the latter two post-positivist. The authors are critical in terms of the framing of normative concepts like ‘strength’ and ‘weakness’, as the findings would be different depending on the framing. In the policy assessment paper, the authors have taken care not to provide strengths and weaknesses based on an implicit framing. The relevant discussion in each of the other papers tends to focus on the process of assessment, and framing is not explicit – but it will be clear that the authors are also well aware of the theoretical debates about effectiveness.

Topical issues

Many of the papers touch on the debates raging in the academic literature over the effectiveness of impact assessment. What comes through clearly, however effectiveness might be framed, is that the country context is critical, and that pluralism, where different stakeholders clearly have different views about what works and how assessment processes should be conducted, is at the core of debates about effectiveness. The papers on EIA, SEA and SIA all deal with relatively mature impact assessment processes for which there has now been a long history of research into effectiveness.

What is clear is that evolving considerations of effectiveness matter for the practice of impact assessment, as legislation and guidance evolve based on research which is framed based on considerations of effectiveness. Many of the developments that have taken place up to the present day have been grounded in process performance considerations that focus on the information provision model for impact assessment (which argues that better information leads to better decisions). However, there is recognition that this is a poor representation of how decision-making actually works and therefore there is a need to better understand decision processes and their interaction with the impact assessment process. The paper on sustainability assessment (Bond et al. 2012) develops a framework for effectiveness, based on practice in other areas of impact assessment, and the paper on policy assessment examines how research is focusing on different elements of effectiveness.

Thus, a second topical issue, closely linked to effectiveness issues, is the theoretical basis on which the influence of impact assessment processes on decision-making is understood. There is consensus among the special issue papers that consider these theoretical issues (which include those on EIA, SEA, sustainability assessment and policy assessment) that there has been a move over time from positivism to post-positivism, such that the original basis upon which impact assessment was legislated through NEPA 1969 (the positivist information provision model) was overly simplistic and there is a need to understand much better how impact assessment and decision-making inter-relate. The theoretical debates are important in the evolution of impact assessment, because they set the research agenda which has the potential to influence future practice. Assuming that post-positivism allows a more realistic view of the influence and role of impact assessment processes, this is a step in the right direction, but theory evolves too, and it is unlikely that any degree of consensus will be achieved on the appropriate theoretical lens with which to view impact assessment.

The final topical issue that has been consistently raised is the importance of knowledge and learning to the successful practice of impact assessment. In the policy assessment paper, we see that the authors categorize one type of research based on the need for informed debate and critical reflection, which revolves around issues of knowledge and learning. The SEA paper refers to learning outcomes being a specific determinant of the effectiveness of the process, while the sustainability assessment paper highlights strategies for embedding learning in the process. These papers conceptualize types of learning as a basis for understanding how best to encourage critical reflection. What is clear here is that impact assessment is beginning to be seen not just as a tool for informing and influencing decision-makers, but as a process which changes the views and attitudes of stakeholders who engage with the process such that their own attitudes and practices change outside of the immediate decision-making context. That is, the influence of impact assessment processes may extend well beyond the narrow decision window in which they operate. There is also recognition that such learning operates on an institutional and social level as well as on an individual level.

External issues (threats and opportunities)

What comes through very clearly from the state-of-the-art papers on EIA, SEA, policy assessment and sustainability assessment is that the current global recession is a significant threat to practice. There is a clear concern that those perceiving a pressing need to grow stagnating economies view impact assessment as a potential barrier to economic growth, and there are examples cited of governments seeking to simplify impact assessment processes to remove this barrier. The authors of the sustainability assessment paper offer some evidence that this negative view of impact assessment providing an unwelcome barrier to development is cyclical – re-occurring at times of recession. Only the SIA and HIA papers have not identified the recession as a threat to practice specifically. For SIA, we might postulate that the amount of practice being driven by international development banks may be largely unaffected, and for HIA, the primarily voluntary basis for practice might indicate that a need is understood and practice follows the identification of this need. For impact assessment conducted voluntarily (which applies to some examples of each of HIA, SIA and sustainability assessment) the need for assessment is recognized. Where impact assessment is conducted because of legal or administrative mandate, this need is only likely to be questioned by observers who doubt the effectiveness and economic underpinning of the process. Concern over the economic benefit of undertaking impact assessment is an ongoing stigma which has never been shaken off.

Another clear issue which impact assessment practice needs to accommodate is climate change and, in many jurisdictions, the consideration of climate impacts is required within impact assessment legislation. While this has already led to the development of a separate stream of climate impact assessment, which interestingly was the subject of a chapter in Vanclay and Bronstein (1995), the EIA, sustainability assessment and SEA papers explicitly acknowledge the need to better accommodate climate impacts.

A recurring opportunity to further the practice of impact assessment is its promotion by international organizations. For example, the role of the United Nations Economic Commission for Europe (UNECE) has been key in promoting the development of SEA with a significant focus on human health, and the United Nations Environment Programme (UNEP), the World Bank, the United Nations Development Programme (UNDP) and the Organisation for Economic Co-operation and Development (OECD) are credited with facilitating the spread of impact assessment. The EIA, SIA and sustainability assessment papers all refer specifically to the International Finance Corporation's Performance Standards, which have also been adopted by some private lenders as the Equator Principles, thereby expanding impact assessment to developing country contexts where practice has traditionally been weak. The paper on EIA recognizes the key role that international conventions have played in the spread of EIA, to the extent that only two countries in the world have not embraced the practice of EIA (one of those has been recognized as an independent country for less than 1 year at the time of press). For policy assessment, the role of the OECD has been recognized in pursuing the practice of regulatory impact assessment, and for HIA, the key role of the World Health Organization is highlighted as it has not only promoted HIA as a practice, but has also been instrumental in lobbying for the inclusion of health in SEA. This makes it very clear that furthering practice is aided by the endorsement of powerful organizations. To date, sustainability assessment has not had this endorsement, but the pattern for the development of practice seems to be one in which practice initially focuses in certain geographical or sectoral areas, and diffuses out slowly, with acceleration being achieved once the benefits are recognized by international organizations.

Continuing practice issues

Flexibility is frequently highlighted as being key to practice in varying contexts (e.g. SEA, sustainability assessment, HIA). The point being made is that the context within which impact assessment takes place is important, and the strict adherence to a standard approach in the wrong context is generally seen as being counter-productive. This flexibility is particularly emphasized in relation to sustainability assessment where it is argued that the goal of the process (sustainable development) is normative and therefore the process needs to respond to differing stakeholder interpretations.

Inadequacies of public participation are frequently raised, with the SIA paper suggesting the process is failing to meet expectations for deliberation. The HIA paper refers to the acknowledged difficulties with community engagement, while the EIA paper makes it clear that poor, or absent, public participation remains an issue (after 42 years of practice), which is a view also reflected in the SEA paper. The sustainability assessment paper authors indicate that public engagement needs to be improved in order to facilitate learning.

Various complaints that recur in the literature associated with different impact assessment processes are highlighted in one or more papers. For example, cumulative effects are still poorly considered (EIA and SIA); equity and resources are poorly covered (HIA); capacity (of human resources) is often inadequate for effective practice (SEA, SIA and HIA); transparency could be better (particularly over uncertainty – HIA); there is poor consideration of alternatives (EIA, SEA, SIA, HIA, sustainability assessment); and dialogue or understanding between sectors is inadequate (HIA and SIA). These many issues could equally apply to all the different areas of impact assessment considered, even though they have been highlighted as continuing issues in just a few.

Despite all these weaknesses and threats, the various impact assessment processes seem to be flourishing. Despite the concerns over the effectiveness of impact assessment, more and more practitioners apply the approaches to the decisions that are made. There is generally agreement on broad process principles, but debates still exist over methodological approaches; this is inevitable given the different theoretical perspectives that exist and, given the diversity of contexts and resources available for funding impact assessment, is not necessarily a bad thing.

All forms of impact assessment are still subject to a lack of confidence among all stakeholders in the value of the exercise, and sceptics have ample evidence in terms of the weaknesses identified throughout this special issue that there are significant problems with practice. However, at present, it seems that the processes maintain sufficient credibility such that practice continues to expand and evolve. However, where threats appear, like the current economic recession, the future of impact assessment may be determined by the ability of researchers and practitioners to demonstrate tangible benefits for a diverse set of stakeholders. In a field where costs can be calculated, but benefits usually cannot, the evidence for practice is currently on shaky foundations and, given that impact assessment was founded on the basis of providing evidence-based decision-making, some reflection is timely; this state-of-the-art special issue contributes to this reflection.

Acknowledgements

We would like to thank all the special edition authors, as well as the reviewers who contributed so much to this special edition.