Scholar and practitioner views on science in environmental assessment

ABSTRACT

This study investigated the views of environmental assessment (EA) scholars and practitioners on science in EA. Fifty-six online survey responses were received, including 35 scholars and 21 practitioners. To supplement the survey data, we interviewed 13 Canadian EA practitioners. The study indicates that EA scholars were more dissatisfied with the quality of science in EA than the practitioners and their perceptions were found to be related to their understanding of science and underlying expectations of scientific practices in EA. This study confirms a gap between science inside and outside EA. Factors contributing to the barriers to filling this gap include EA stakeholders’ different understandings of and expectations for the quality of science, the role of scholars in EA, and the purpose and objectives of EA. These disagreements imply insufficient and/or ineffective communications among EA stakeholders, which should be addressed if a more collaborative arrangement is to be developed for improvements in the quality of science in EA.

KEYWORDS:

• Environmental assessment
• science
• scholars
• practitioners
• satisfaction

Introduction

Environmental assessment (EA) is a widely acknowledged formal political decision-making and environmental planning tool in many countries (Sadler 1996; Noble 2015a). It can be traced back to the 1970s and was developed in the United States of America (USA) as a formal political response by the government to increasing public concerns about environmental impacts caused by human developments (Ortolano and Shepherd 1995; Fischer and Noble 2015). Due to its political roots, EA was criticized as a political decision-making tool used by proponents and politicians to justify project decisions that had been made already when the EA process was initiated (Noble 2015a). In addition, this history has led to the failure of EA decisions to meet requirements of rationalist models (Lee et al. 1995; Ortolano and Shepherd 1995). Political decisions include complicated trade-offs among various factors (e.g. economic benefits at the expense of environmental costs, and vested interests among different relevant parties and stakeholders). Therefore, a rational EA decision would not be practically made in the political arena (Cashmore et al. 2004).

There was a disagreement on the purposes of EA among major EA stakeholders which arose from the diversity of interests and objectives expressed by different participating groups (Enríquez-de-Salamanca 2018). Beanlands and Duinker (1983) used four major EA stakeholders (i.e. government administrators, proponents, consultants, and research scientists) in the Canadian context as an illustration of the vested interests and objectives of different participating groups in the EA process. Fuller (1999) echoed Beanlands and Duinker (1983) in describing the interests among various EA stakeholders, and admitted that the expectations of different EA stakeholders are not complementary and sometimes even in conflict. There was an ultimate conflict embedded in the underlying purposes of EA from different stakeholders. The different objectives of stakeholders led to different perceived purposes of EA and also to different expectations on the quality of EA and scientific practices implemented in the EA process (Enríquez-de-Salamanca 2018). Finally, these different expectations were reflected in the range of quality of environmental impact statement (EIS) documents, a major product of the EA process (Fuller 1999).

EA was developed over 40 years ago, and the earliest EA-related best practice guidelines and EIS handbooks were published in the 1970s (e.g. Cheremisinoff and Morresi 1977). Strong scientific principles for EA practices have been established also since the 1970s (e.g. Holling 1978). As a result of more attention, these principles have been further strengthened since the 1980s (e.g. Caldwell et al. 1982; Beanlands and Duinker 1983). However, Mackinnon et al. (2018) conducted a thorough literature review on scientific developments associated with EA since the 1970s and identified a large gap between science inside EA and science outside EA, which means that scientific quality in EA still requires improvement.

The concepts of science inside EA and outside EA were introduced by Greig and Duinker (2011) when they were trying to demonstrate a mutually supportive cycle between research scientists and proponents for improving the implementation of science in the EA process. Greig and Duinker (2011) categorized science into two groups: science outside EA refers to reliable ecological effects knowledge that is created by research scientists outside EA (e.g. Lester et al. 2010). Science inside EA refers to the ecological knowledge that is used and applied in EA impact predictive models by EA practitioners operating inside EA (e.g. Dipper 1998; Jones and Fischer 2016).

In the early 1980s, Beanlands and Duinker (1983) pointed out a necessary step to transform the adversarial relationship between scientists and EA practitioners into a more collaborative one for improving the overall quality of science in EA. In later studies, the importance of a collaborative relationship between scientists and other stakeholders (e.g. policy-makers, citizens, and project proponents) has been identified in terms of generating a well-informed resource management decision and achieving a goal of sustainable natural resource management (Rogers 2006; Roux et al. 2006; Gibbons et al. 2008; Ryder et al. 2010). Barriers to achieving such a relationship have also been discussed in the literature, such as cultural differences, education background differences, lack of incentives for collaboration, and lack of opportunities for communication (Briggs 2006; Roux et al. 2006; Ryder et al. 2010). These barriers lead to fundamentally conflicting values, beliefs, and understandings of science, and consequently generate difficulties for these stakeholders to mutually understand each other (Briggs 2006; Ryder et al. 2010).

Greig and Duinker (2011) emphasized that an agreement among EA stakeholders as to the need for improved quality of science in EA is a necessary condition. However, EA stakeholders seem to have different perceptions on the quality of EA performance. On the one hand, the quality of EA practices is always dissatisfying to EA reviewers. The EIS is a key product of EA systems and it has been commonly used to reflect the quality of EA practices performed by EA practitioners (e.g. proponents and consultants hired by proponents). Reviews of EISs indicate that there were discrepancies between what should be done as described or guided in the literature and what had been done as stated in EISs in Europe (Glasson et al. 1997; Thompson et al. 1997; Barker and Wood 1999; Gray and Edwards-Jones 2003; Canelas et al. 2005), in Canada (Duinker 2013), in the USA (Tzoumis 2007), and in Bangladesh (Kabir and Momtaz 2012). In contrast, in a Dutch survey, EA practitioners were generally satisfied with the quality of EA reports (Arts et al. 2012). In addition, EA practitioners, mainly consultants, were generally satisfied with the quality of scientific work done in EA in an Australian survey (Morrison-Saunders and Bailey 2003) and in a Finnish survey as well (Jalava et al. 2010). Even though these EA practitioners indicated that they were dissatisfied with the importance of science placed at different stages in an EA system, when they were asked to self-evaluate their performance, their satisfaction levels were generally very high (Morrison-Saunders and Bailey 2003).

It is essential to understand the perception of various EA stakeholders on the quality of science embedded in EA practices. Effective EA decision-making needs scientific support (Morrison-Saunders and Sadler 2010); effective scientific support needs a collaboratively harmonious relationship between EA scholars and practitioners and also a consensus towards the perceived satisfaction levels with scientific work in EA practices (Greig and Duinker 2011). There is a lot of literature about theoretical studies and empirical studies regarding the role of science and the quality of scientific practices in EA. However, there have been only a few studies investigating the perceptions of EA practitioners on the scientific practices in EA (e.g. Caldwell et al. 1982; Sadler 1996; Morrison-Saunders and Bailey 2003; Jalava et al. 2010), and no literature has been found that investigate the perceptions of both EA practitioners and scholars and compare the differences (if any) between them. This study provides data about EA scholars’ and practitioners’ perspectives on the degree to which science in EA needs improvement, and the factors that have the potential for improving the state of scientific work within EA. Our research questions focus on:

1. What are the levels of satisfaction of EA scholars (i.e. research scientists) and practitioners (i.e. government scientists, consultants, EA administrators, and non-governmental organizations (NGOs)) with the quality of science embedded in an EA process?

2. What is the perceived status of science in the EA process by scholars and practitioners?

3. What factors have influenced the quality of scientific practices in EA?

Methods

Online surveys

Surveys are a frequently used instrument in other similar EA inquiries (e.g. Sadler 1996; Morrison-Saunders and Sadler 2010). Opinio (ObjectPlanet 2018), online survey software, was selected for this study based on the considerations of low cost and data security. Survey questions included eight demographic questions investigating respondents’ EA-related working experience, teaching experience, roles played in EA, ages, education, genders, countries that they have most worked in, and contributions to EA-related literature, and eight other questions regarding the perceptions of survey respondents on (1) the quality of science at the stages of the EA process; (2) factors influencing the quality of science in EA; (3) the power of various EA stakeholders; (4) the contributions of scientific support from the scholars’ community to the EA process; and (5) the status of science in EA practices. Original survey questions can be found in the on-line supplementary material of the article. In our survey introduction, we instructed potential survey respondents with the following text:

‘We acknowledge a wide range of interpretations of what science is in general, and what scientific approaches people think are appropriate in EIA. For our purposes, it is sufficient for you to answer the survey with your own interpretation of science in the context of EIA. We also acknowledge the essential role of traditional knowledge, especially among Aboriginal peoples, in providing vital insights in the process of EIA, but this survey focusses on formal scientific approaches and methods’.

We avoided offering our own conception of what is science in the context of EA, and what scientific approaches we think are the most suitable in EA, on the expectation that it might lead survey respondents to challenge us based on their own views and divert their response energies away from the questions we posed. The five-point Likert scale was used to investigate the respondents’ satisfaction with the quality of science at the stages of the EA process (i.e. ‘very dissatisfied’, ‘dissatisfied’, ‘neither satisfied nor dissatisfied’, ‘satisfied’, and ‘very satisfied’). A three-point scale was used to investigate the respondents’ perception of the importance of factors influencing the quality of science in EA, and the power of various EA stakeholders in the EA process (i.e. low, medium, and high importance/power/influence). ‘I do not know’ was also provided as an additional option in the survey questions. These survey questions helped to contribute to the formulation of generalizations about the trends and themes regarding the perceived quality of science applied in the EA process among the key stakeholders in the EA community.

Invitations to engage with the online surveys were distributed in two ways: (1) the International Association for Impact Assessment (IAIA) network via its monthly e-news and the normal distribution list (i.e. IAIA Connect) and (2) email following a literature search for EA-related journal paper authors (i.e. Environmental Impact Assessment Review, and Impact Assessment and Project Appraisal). IAIA is a well-known leading global network in the field of impact assessment, and the IAIA network was used by previous research studies to investigate EA professional perspectives on EA inquiries (e.g. Sadler 1996; Morrison-Saunders and Sadler 2010). Because the majority of IAIA network members are EA practitioners, in order to balance the ratio of EA scholars and practitioners in survey results, around 200 journal paper authors were selected from the journal papers that were published from 2014 to 2016 in Environmental Impact Assessment Review as potential survey respondents. All surveys were anonymous, and the instrument was kept open from May 2016 to December 2016. By the end of 2016, 56 survey respondents completed the survey, including 35 EA scholars and 21 EA practitioners. Raw survey responses were downloaded from Opinio in the format of Microsoft Excel (MS Excel) (Microsoft 2018) and then were categorized based on their self-identified roles into two groups – ‘EA scholars’ and ‘EA practitioners’.

Minitab (Minitab Inc 2018) was used for doing the Mann–Whitney U test, a non-parametric statistical analysis method, to deal with the ordinal data generated from Likert-scale questions (De Winter and Dodou 2010). Statistical significance values were calculated and the difference between the satisfaction levels of EA scholars and practitioners with the quality of science in EA was identified. The statistical p-value significance thresholds selected for this study were 0.1 and 0.05. As for the survey data regarding the attitudes of respondents to the quality of science in EA, ‘satisfied’ and ‘very satisfied’ were merged, and ‘dissatisfied’ and ‘very dissatisfied’ were also merged. The number of respondents who expressed ‘I do not know’ or did not respond to a specific survey question were subtracted from the calculation of the percentage of respondents’ satisfaction levels. NVivo (QSR International Pty Ltd 2018) was used for dealing with qualitative data from open-ended survey questions and also additional comments provided by respondents to explain their answers to Likert-scale questions. These qualitative data were coded, categorized, analyzed, and distilled using an a posteriori coding scheme.

Interviews

Due to the small sample size of survey responses received, interviews were considered as a complementary instrument to enrich the survey results. Interview questions included six demographic questions which were highly similar to the demographic questions included in the online surveys, and eight other questions regarding the interviewee perspectives on the scientific practices in EA. Thirty-six Canadian EA professionals were purposively selected for interviews based on the personal knowledge of the authors. Thirteen out of 36 EA professionals agreed to be interviewed. One interviewee acts as both a scholar and a practitioner, and the others act purely as practitioners. None of the potential interviewees acting purely as EA scholars agreed to be interviewed. Therefore, interview data were not categorized into two groups as had been done for survey results.

Due to the diverse locations of purposively selected EA professionals in Canada, the interviews were conducted individually either over the phone or in person at a location of the interviewees’ convenience. All interviews took between 40 and 120 min, depending on the interviewees’ responses. The interviews were audio-recorded and then transcribed verbatim by the authors. Similarly, MS Excel (Microsoft 2018) was used to create figures to represent interviewees’ demographic information. NVivo (QSR International Pty Ltd 2018) was used for dealing with qualitative data from interview questions. An inductive coding method was used for interview data first, and then the resulting coding nodes were compared with the coding nodes generated from survey results to identify whether interview data and survey results are complementary and consistent or conflicting.

Results

Survey demographic data

Thirty-five survey respondents were identified as EA scholars. Most of them had a medium to high level of teaching experience (83%) and a medium to high level of contribution to the EA-related literature (80%). Twenty-one survey respondents were identified as EA practitioners, and they generally had a comparatively lower level of teaching experience and literature-writing experience than the scholars. The roles of the EA practitioners included consultants (71%), EA regulators (19%), NGOs (5%), and other (5%). Both pools of survey respondents had a generally high level of EA experience: 67% of scholars and 84% of practitioners had more than 5 years of EA experience. Most scholars had doctorate degrees or had even worked as post-doctoral fellows (73%). However, most practitioners had Master’s degrees (57%) and some had doctorate degrees (29%).The countries and/or continents that the survey respondents mostly work in on EA projects are varied (Figure 1). Interestingly, 35% of EA scholars answered survey questions from a biophysical science point of view, and 65% from a social science point of view; however, 81% of EA practitioners answered from a biophysical science point of view, and only 19% from a social science point of view.

Figure 1. Countries/continents that survey respondents work in on EA projects.

Survey results

Both scholars and practitioners were highly dissatisfied with cumulative effects assessment (82% and 80%, respectively). Perceived satisfaction levels for other EA components were highly variable for both scholars and practitioners (Table 1). For example, regarding species at risk, EA scholars generally expressed higher satisfaction than EA practitioners. However, this observation was not statistically significant (Table 2). Scholars were statistically significantly more unsatisfied with the quality of science used in the identification of valued ecosystem components (VECs) compared to practitioners (p = 0.0299, alpha = 0.05). Scholars were slightly more unsatisfied with approaches used for impact prediction compared to practitioners (p = 0.0974, alpha = 0.1). No statistically significant differences were observed in other individual EA components (Table 2), indicating there was no difference in perceived satisfaction levels with these individual EA components.

Table 1. Satisfaction levels (%) expressed by EA scholars and practitioners (i.e. satisfied, neither satisfied nor dissatisfied, and dissatisfied) with EA components. Bold numbers in the table refer to the relatively higher percentages of survey respondents. The topics were grouped based on a general understanding of their frequencies included in an EA system.

	Scholars (%)			Practitioners (%)
Topic	Satisfied	Neither	Dissatisfied	Satisfied	Neither	Dissatisfied
Group 1
VECs	39	35	26	62	28	10
Time and Space Bounds	39	22	39	57	10	33
Baseline Data	38	24	38	60	10	30
Impact Prediction	36	21	43	60	10	30
Impact Significance	29	24	47	55	5	40
Monitoring and Follow-up	15	20	65	33	19	48
Group 2
Species at Risk	55	16	29	35	30	35
Cumulative Effects Assessment	6	12	82	10	10	80
Sustainability Assessment	27	27	46	10	32	58
Public Participatory Processes	29	34	37	38	19	43
Social Impact Assessment	23	23	54	35	10	55
Human Health Impact Assessment	20	30	50	35	20	45

Table 2. Statistical significance in differences between satisfaction levels expressed by EA scholars and practitioners with EA components. Selected confidence level is 90% and 95% (alpha = 0.1 and 0.05).

EA components	Statistical significance (p-value)	EA components	Statistical significance (p-value)
Group 1	p = 0.0008	Group 2	p = 0.3654
VECs	p = 0.0299	Species at Risk	p = 0.8600
Time and Space Bounds	p = 0.1423	Cumulative Effects Assessment	p = 0.2330
Baseline Data	p = 0.1188	Sustainability Assessment	p = 0.7253
Impact Prediction	p = 0.0974	Public Participatory Processes	p = 0.3187
Impact Significance	p = 0.1336	Social Impact Assessment	p = 0.4254
Monitoring and Follow-up	p = 0.1215	Human Health Impact Assessment	p = 0.1945
P-value = 0.0092

At an aggregated level, EA scholars expressed greater dissatisfaction with the quality of science used in most EA components compared to EA practitioners, which was confirmed using a Mann–Whitney U test (p = 0.0092, alpha = 0.05). The six components in group 1 are the most common ones included in an EA process (see Beanlands and Duinker 1983), because they can be found in almost all EA systems, and the six components in group 2 are less common than those in group 1. Specifically, EA scholars were significantly more unsatisfied with those most common six EA components than practitioners (p = 0.0008, alpha = 0.05) (Table 2).

Reasons for the satisfaction with these EA components can be categorized into two groups. First, survey respondents were satisfied with what had been done, such as simple cause–effect approaches which were used to identify VECs. Second, their satisfaction was based on the current level of proponents’ capacity and availability of analytical tools.

Reasons for the dissatisfaction with EA components can be summarized into 10 groups:

1. non-existent EA components;

2. inadequate implementation;

3. very weak scientific foundation;

4. an overly narrow, unclearly or incorrectly defined scope and focus;

5. ignored consideration of uncertainties;

6. big gaps between what should be done based on scientific literature and what has been done in real EA practices;

7. insufficient baseline data collection;

8. ambiguous decision-making processes;

9. political barriers; and

10. limited resource investments (e.g. time, human, and financial resources).

Fourteen factors were included in the survey to investigate the perceived importance levels ascribed by the EA scholars and practitioners; these were categorized into three groups based on survey responses (Table 3). The first group includes seven factors that were considered highly important to both scholars and practitioners. The importance of factors in the second group was considered to be of medium importance by most EA scholars and practitioners. The third group consists of five factors that were considered more important by EA scholars than practitioners. Using a Mann–Whitney U test on single factors, EA practitioners rated the importance of time availability slightly higher than scholars, and EA scholars rated the importance of the first four factors in group 3 higher than practitioners (Table 3). Responses for the first three factors in group 3 reflect different attitudes of scholars and practitioners towards the importance of scientific community participation in the EA process, and statistical analysis on the grouped responses on these three factors indicates that scholars felt their participation in EA was more important than how EA practitioners thought about it (p = 0.0071, alpha = 0.05). Scholars explained that their participation could bring more scientific considerations into project decisions; however, practitioners considered that EA is not scientific research and indicated that cooperation between scholars and practitioners can be helpful only when scholars can allow/accept different value systems to be freely discussed. Interestingly, compared with scholars, a higher proportion of practitioners expressed a dissatisfaction with the contribution of the scientific community to EA (Figure 2).

Table 3. Importance levels (%) and statistical significance in the differences between the importance levels expressed by EA scholars and practitioners (i.e. low, medium, and high importance) with influential factors in influencing quality of science in EA. Factors were grouped based on the level of importance perceived by scholars and practitioners. Selected confidence level is 90% and 95% (alpha = 0.1 and 0.05).

	Scholars (%)			Practitioners (%)
Factors	Low	Medium	High	Low	Medium	High	p-values
Group 1
Time availability	3	31	66	0	14	86	p = 0.0507
Funds availability	6	28	66	5	24	71	p = 0.6420
Training levels of practitioners	6	28	66	0	24	76	p = 0.7718
Proponents’ attitudes towards social responsibility beyond regulatory requirements	6	26	68	10	33	57	p = 0.2125
The expectations of regulators regarding the quality of science in EA	2	49	49	5	38	57	p = 0.6792
Structure of the regulatory process for EA	11	29	60	15	20	65	p = 0.5763
EA practitioners’ opinions about the importance of science in EA	11	40	49	21	32	47	p = 0.3495
Group 2
Public discourse in EA	9	57	34	15	50	35	p = 0.4069
Support level for EA practitioners from the general scientific community	26	48	26	30	45	25	p = 0.4067
Group 3
The participation of the scientific community in EA	14	37	49	35	30	35	p = 0.0725
Opportunities for the scientific community to participate in EA	14	43	43	15	65	20	p = 0.0902
Cooperation between EA scholars and practitioners	11	40	49	20	50	30	p = 0.0833
Stakeholders’ concern for the quality of science in EA	6	43	51	19	43	38	p = 0.0970
The availability of scientific literature	11	46	43	28	24	48	p = 0.3570

Figure 2. Satisfaction levels (%) by scholars and practitioners with the contribution of the scholarly scientific community to the EA process. Statistical differences indicated that EA scholars and practitioners have essentially the same satisfaction levels (p = 0.278, alpha = 0.1).

Additional factors were reported by scholars and practitioners. For example, scholars emphasized that a clear definition of science and a clear guideline directed by regulators in terms of the methods for conducting scientific analysis are also highly important in influencing the quality of science. Practitioners also listed the following factors:

• (1) integration of science and traditional, local cultural, and indigenous knowledge;

• (2) independence of EA practitioners;

• (3) effectiveness of communication;

• (4) community engagement;

• (5) political interference;

• (6) methodological guidelines;

• (7) opportunities for practitioners to learn the theoretical foundation of EA;

• (8) scientific understanding of cause–effect; and

• (9) accordance with dominant value systems.

Both scholars and practitioners indicated that the contribution of science is compromised by other factors such as traditional knowledge and public interest in the project decision-making process. Weak synergy between scholars and government regulators/decision-makers also results in limited scientific insights embedded in EA regulations and government guidance documents. Referring to the ways that science should be strengthened so that it can improve the effectiveness and efficiency of the contribution to project decision-making, the scholars proposed the following priorities:

• (1) more consideration of uncertainty;

• (2) better training of EA practitioners;

• (3) better understanding of the gap between scientific problems in academics and real EA problems in practice;

• (4) increased funding;

• (5) improved participatory processes;

• (6) higher scientific standards required by permitting authorities to practitioners;

• (7) greater levels of independence of scientists and science;

• (8) better collaboration between scholars and practitioners;

• (9) improved involvement of scholars in the EA process; and

• (10) better adaptive learning from the EA process.

The practitioners suggested the following ways:

• (1) a better application of precautionary principles in decision-making processes;

• (2) more time and funding availability;

• (3) more consideration of uncertainty;

• (4) stronger partnerships between stakeholders;

• (5) strengthened scientific standards by regulators;

• (6) more-transparent decision-making processes;

• (7) earlier participation of scholars in the EA process;

• (8) scientists with more field experience and more collaborative and humble attitudes; and

• (9) continuous refining of the best available science.

For most stakeholders, EA scholars and practitioners expressed the same level of power and influence on their role in influencing the EA process (Table 4), and there was no statistical difference between merged responses to perceived power and influence of all stakeholders by both pools (p = 0.8957, alpha = 0.1). For observations on single stakeholders, those in group 1 were considered highly powerful by both pools of EA scholars and practitioners, and the power and influence of stakeholders in group 2 were considered in the medium level by both pools of respondents (Table 4). Scholars ascribed a higher level of power to proponents than did practitioners (p = 0.0113, alpha = 0.05), and there was no statistical significance found in the difference of the two pools of respondents’ attitudes about other stakeholders. For group 3, practitioners considered scholars and aboriginal groups to have a more powerful role in EA than what scholars believed they had, and the difference between the attitudes of the scholars and practitioners towards the importance of scholars was confirmed by statistical analysis (p = 0.0875, alpha = 0.1).

Table 4. Power and influence levels (%) perceived by EA scholars and practitioners (i.e. low, medium, and high power and influence) in the generic EA process. Stakeholders were grouped based on the level of influence perceived by scholars and practitioners. The highest percentage of each stakeholder was italic in the table.

	Scholars (%)			Practitioners (%)
Stakeholders	Low	Medium	High	Low	Medium	High
Group 1
Politicians	11	20	69	19	10	71
Regulators	3	28	69	0	29	71
Proponents	3	14	83	15	30	55
Group 2
NGOs	17	72	11	24	52	24
Consultants	6	51	43	14	57	29
The public	26	54	20	14	57	29
Group 3
Scholars/Scientists	57	37	6	38	52	10
Aboriginal groups	27	43	30	32	26	42

Interview demographic data

Twelve interviewees were identified as EA practitioners, including consultants, proponents, EA regulators, government reviewers, and NGOs, and one interviewee was identified as both EA practitioner and scholar. Most interviewees were not experienced EA-related teachers or professors or experienced literature contributors, but were experienced EA professionals due to their very high level of EA-related work experience (Figure 3). All had a Canadian EA background. Interestingly, no interviewee answered questions from a social science perspective. Most indicated that they answered from a mixed perspective (Figure 3).

Figure 3. Demographic information about interviewees.

Interview results

The factors behind interviewee satisfaction with the quality of science in EA echoed the factors offered by survey respondents, and the reasons why interviewees were dissatisfied can also be categorized into the same 10 groups as those presented in survey responses.

For the status of EA, interviewees indicated that current EA is about mitigating undesirable outcomes and has become standardized about moving a project through a regulatory process rather than a scientific process. They said this standardized process leads to a lack of creativity in finding solutions for EA issues, and inefficient use of proponents’ time and money. In addition, some important stakeholders are marginalized in this kind of standardized regulatory process due to their low political power. EA is considered as a consulting and negotiating process among various stakeholders rather than an evaluation process based on scientific evidence. There has been disagreement on EA purposes between various stakeholders; therefore, they would use EA as a tool for their own purposes. For example, proponents want to get their projects approved, so they want to understand the concerns of other stakeholders about their projects to guarantee their projects can be approved. Indigenous groups and land owners want to protect their properties; therefore, they would treat EA as a way of opposing proposed projects. Those who are concerned about climate change would treat EA as a way of reminding the government of the importance of climate change and how a proposed project would contribute to an increase in emissions of greenhouse gases. Bringing these different stories to any public hearings by stakeholders makes EA complicated.

For the status of science in EA, some interviewees indicated that there is a generally weak scientific foundation for EA, which results in a low-quality discourse. The main factors contributing to a weak scientific foundation for preparing EISs include: unqualified EA practitioners, time and cost constraints on consultants by proponents, a low expectation of regulators about the quality of science, and low participation of scholars and government scientists in the EIS preparation phase. There are two other scientific issues flagged by interviewees. First, science is misused when scientific data are purposively selected to support stakeholders’ arguments. For example, on one hand, scientific uncertainty is used by intervenors to block the project, and this is not an appropriate way of helping a project to be improved. Academic scientists often reveal support for the positions of specific intervenors. If so doing, they often engage in a narrow selection of evidence for those positions. On the other hand, proponents have been brought incomplete information and have been selective in the analysis in the EA report to get their projects approved. Second, there is a lack of independent science. Governments do not have sufficient budgets to hire scientists to do EA. Scientific work is generally done by professional people hired by proponents, and these professionals consequently are not independent to speak freely about negative project impacts. In addition, the current EA structure does not demand and encourage independent science.

Some adverse tensions were identified in the relationships between various EA stakeholders. Consultants expressed satisfaction with the general quality of science because they believe that the level of science expected by government reviewers is reached. However, government reviewers indicated that they are not allowed to speak freely on scientific issues and negative environmental issues under political pressure. Regulators expressed satisfaction with the amount and quality of scientific information they receive and post to the public record. However, a low acceptance level of science by regulators was criticized by other stakeholders as a reason why scientific quality in EA was unsatisfactory. Negative impacts that are flagged by government reviewers and NGOs cannot sway the decision of politicians and the Ministers in charge because political will makes the weight of economic benefits and concerns embedded in EA decisions higher than that of environmental concerns.

Discussion

Survey results indicate that EA practitioners were generally satisfied with the quality of science in EA. This finding is consistent with results of two other surveys with respect to practitioner perspectives on the quality of science in EA – one at an international scale (Sadler 1996), and another in Western Australia (Morrison-Saunders and Bailey 2003).

EA practitioners expressed a higher satisfaction level in the current survey responses regarding the quality of science in EA than EA scholars. Satisfaction reasons given by survey respondents can be generally summarized as that survey respondents expressed satisfaction based on what they have experienced as achievements in the scientific practices in EA. Interestingly, when EA practitioners expressed satisfaction levels with the quality of scientific practices in EA, they were self-evaluating themselves. EA practitioners conduct EAs; therefore, in this survey, they were self-evaluators. In contrast, EA scholars acted as external evaluators for scientific practices in EA. Therefore, EA scholars may be more critical about evaluation of the scientific practices in EA than the EA practitioners. Based on this logic, it would not be surprising that EA practitioners showed a higher satisfaction level than EA scholars. The same logic was also used by Greig and Duinker (2011) to explain their understanding of survey results presented in Morrison-Saunders and Bailey (2003) regarding a high satisfaction level with the quality of science by EA practitioners in Western Australia.

Another explanation for differences between satisfaction levels of EA scholars and practitioners is that EA scholars and practitioners have different understandings of science and this distinction results in their different expectations of the quality of science implemented in EA. Non-scientists consider science as a process of data collection; therefore, when they discussed the quality of science, they generally thought about how much data they have collected, and how good the quality of data collection is (Bisbal 2002; Briggs 2006; Sullivan et al. 2006; Ryder et al. 2010; Green and Garmestani 2012). However, scientists consider science as a more rigorous analysis process, starting with an identified research problem, and then based on the defined scope of the research question to determine the scope of data collection and to select an appropriate research method to collect and analyze data. A thorough data analysis would be applied after data collection and finally a conclusion would be drawn based on a logic flow embedded in the analysis (Bisbal 2002; Briggs 2006; Sullivan et al. 2006; Ryder et al. 2010; Green and Garmestani 2012). These different understandings of science embrace different values and beliefs, and sometimes these values are in conflict in the environmental decision-making policy arena (Briggs 2006; Ryder et al. 2010).

Science is not easily practiced in EA and the following factors have contributed to difficulties of its implementation: (1) a low expectation or an unclear direction from regulatory administrators as to the quality of science; (2) limitations in human, time, and financial resources; and (3) adversarial political tensions between stakeholders. A low expectation or unclear direction from regulators may be understood from two perspectives. First, regulators have a solid understanding of science, but they put a low-quality threshold on the requirements on the scientific practices in EIS guidelines due to some reality constraints such as staff shortages, loss of experienced staff, and financial insufficiency (Morrison-Saunders and Bailey 2009). Second, regulators may not have a deep understanding of science, so they have no idea about what an appropriate quality threshold on the scientific practices in EIS guidelines should be like. As a result, they use vague and ambiguous language to write EIS guidelines. A low expectation or unclear direction from regulators can also reflect a root of most of the dissatisfaction reasons explained by survey respondents, such as non-existent EA components, inadequate implementation, an overly narrow scope and focus, unclearly or incorrectly defined scope, big gaps between what should be done based on scientific literature and what has been done in real EA practices, and insufficient baseline data collection. Because regulators do not specify explicit requirements required in the scientific dimensions of EA, practitioners cannot get a clear understanding about the expectations from regulators regarding the quality of science. At the same time, there are also time and financial constraints from the proponents’ side, so it would not be surprising that some scientific quality issues as listed above would be identified and would generate dissatisfaction among EA stakeholders.

The power of regulators has been demonstrated in influencing the environmental performance of proponents in the literature, and proponents also have an incentive to be proactive and/or interactive towards environmental approvals regulation. Specifically, Morrison-Saunders et al. (2001) identified that the expectations of EA regulators were one of the main influencing drivers to the improving quality of science in EA, and regulator pressure was the main incentive and stress on proponents to improve the quality of EISs. In addition, an administrative control mechanism operated by regulators was identified as one of the main drivers to the positive changes in the Brazilian EISs in the period between 1987 and 2010 (Landim and Sánchez 2012). The results of these two studies echoed the study of Annandale (2000), which indicated the power of regulators in influencing the response of the companies dealing with mines to environmental approvals regulations. In a later study, Annandale et al. (2004) identified regulator pressure as a determinant of the environmental performance of development companies. Additionally, Annandale and Taplin (2003) found that development companies treat environmental approvals regulation as an important opportunity to improve their project designs so as to avoid possible negative environmental outcomes caused by poor project designs. Therefore, the expectations of regulators may be considered by proponents as the ceiling of the scientific quality in EA. Sometimes there is a lack of consensus about the clarity of EA regulations, and different stakeholders may have different understanding of these regulations (Arts et al. 2012). If the quality of science in EA is expected to be improved, then improving regulators’ requirements or expectations would be one of the direct ways. In some jurisdictions, this should be possible through regulators’ interpretations of scientific requirements on proponents, and even their informal negotiations about how to interpret statutory and regulatory requirements.

The importance of resources (i.e. human, temporal, and financial) was frequently linked with the improvement in the scientific dimension of EA by the survey and interview participants, and these links can be interpreted from the following four perspectives. First, in the EIS preparation phase, well-qualified practitioners with a high level of expertise and sufficient temporal and financial resources are needed for collecting sufficient baseline data and applying appropriate scientific methods in predicting possible impacts and determining impact significance. Second, at the public participatory stage, sufficient staff, temporal, and financial resources are also important to support the participation of the public and other stakeholders (e.g. NGOs and aboriginal groups) in the EA discourse. Third, in the post-EA phase, sufficient resources are important for governments to conduct monitoring and follow-up and to post the monitoring and follow-up reports to the public record for adaptive learning purposes. Fourth, independent and sufficient research funds are an important factor for creating independent science.

These results overlap with the results of Morrison-Saunders et al. (2001) who pointed out that sufficient time and financial resources were an important determinant to the level of science in EA, and the insufficiency of research funds provided by proponents was considered as a major factor influencing the level of science applied in EA. Additionally, Morrison-Saunders and Bailey (2009) demonstrated that sufficient human resources play an important role in influencing the quality and effectiveness of EA practices. Staff shortages in regulatory agencies caused by a recent resource boom in Australia would reduce the capacity of regulators, so as to hardly meet the requirements and expectations of other stakeholders. The relationships between regulators and other stakeholders may also be strained (Morrison-Saunders and Bailey 2009). In addition, Jones and Fischer (2016) concluded that lack of funds is one of the most pressing barriers to EA follow-up uptake. Therefore, sufficient human, temporal, and financial resources are needed to improve scientific practices and effectiveness of EA.

Based on the survey respondents and interviewees, adversarial political tensions were identified among EA stakeholders, which reflect weaknesses in effective communication between stakeholders in terms of their different expectations and challenges encountered in their different roles. And these communication gaps among stakeholders generate different perceived satisfaction levels with the quality of science in EA. Noble (2015b) indicated that different stakeholders have different values, demands, and expectations in the context of cumulative effects assessment, which may lead to very adversarial social-political EA environment. Better recognition and understanding of different stakeholders’ values may help to improve the effectiveness of EA (Cape et al. 2018). That being said, an improvement in effective communications among stakeholders can promote their mutual understandings, thereby relieving adversarial political tensions and creating a foundation for a collaborative relationship between EA stakeholders. Whether a collaborative relationship can indeed emerge will depend largely on the cultural and governance contexts within which a specific national or sub-national EA system operates (see Monteiro et al. 2018).

The literature emphasizing the importance of changing from adversarial relationships among EA stakeholders to a collaborative way for improving the quality of science can be traced back to the 1980s (i.e. Caldwell et al. 1982; Beanlands and Duinker 1983). The importance of creating interdisciplinary teams in an adaptive learning management system was elaborated by Ryder et al. (2010). Mackinnon et al. (2018) demonstrated the gaps between science inside EA and science outside EA and emphasized that improvement in science inside EA would depend on more collaborative relationships and arrangements among EA stakeholders. However, how to develop and/or improve this kind of collaborative arrangement among EA stakeholders has not drawn enough attention.

Morrison-Saunders and Bailey (2009) shared a positive initiative, the Partnering Agreement between regulators and consultants in Western Australia, and demonstrated benefits generated from the resulting cooperative relationship between these two EA stakeholders. However, it may be easier to promote collaborative relationships between two EA stakeholders than more than three stakeholders, especially when these stakeholders have conflicting values and expectations about the quality of science and effectiveness of EA. Even though regulators and consultants play a totally different role in the EA process, based on our Canadian experience and what is reported by Morrison-Saunders and Bailey (2009), they generally have similar EA-related educational backgrounds and working environments. Therefore, it may be more challenging to motivate two stakeholders with different EA-related training background to cooperate and collaborate.

The level of willingness by EA stakeholders to cooperate and collaborate to improve their working relationships and EA performance is key for initiating such a collaborative arrangement (Morrison-Saunders and Bailey 2009). However, based on survey responses of this study, EA practitioners gave participation of scientific communities in the EA process less importance than EA scholars, which reflects, to some degree, that EA practitioners are unwilling to work collaboratively with scholars. Some survey comments even explained that in reality, EA scholars and practitioners mutually avoid each other due to their different value systems. Additionally, from a scholar’s perspective, participation in EA practice-related activities is voluntary and a scholar’s contribution to these practices is rarely highly valued in the academic context. Scholars thus lack incentives (beyond possible consultancy fees) to participate in EA activities. These challenges obviously should be overcome for developing more collaborative and participatory working relationships among stakeholders to improve the quality of science in EA.

Greig and Duinker (2011) showed a potential collaborative relationship between EA practitioners and scholars for improving the quality of science in EA and pointed out that one condition must be reached to make this kind of collaborative arrangement work – all stakeholders should reach a consensus about the status of science in EA and agree that the quality of science needs to be improved for delivering better EA outcomes. However, based on survey responses, EA scholars and practitioners have different perceptions on the quality of science in EA, and these different satisfaction levels also indicate that at least EA scholars and practitioners do not exhibit a consensus about the status of science in EA; consequently, their willingness to change the current EA situation remains unchanged.

Conclusions

This study indicates that EA scholars were more dissatisfied with the quality of science in EA than EA practitioners. When survey respondents and interviewees explained their satisfaction levels, their perceptions were generally related to their underlying expectations on the scientific practices in EA and their understanding of science in EA. Were we to do the study again, we would seriously consider devising a means of interrogating people’s understandings of science and the roles it plays in EA. This study implies that EA practitioners play an influential role in determining the quality of science in EA and EA scholars play a limited role in influencing the quality of science in EA. However, the EA practitioners did not show a willingness to change this situation. Also, this study implies that EA scholars and practitioners do not exhibit a consensus about the status of science in EA. This disagreement may be a barrier to creating mutual understandings and effective communications among EA stakeholders, thereby hindering the building of a collaborative relationship between EA scholars and practitioners for improving the quality of science in EA (Greig and Duinker 2011).

This study aligns with the conclusion made by Mackinnon et al. (2018) that there are large gaps between science inside EA and science outside EA. Factors contributing to the barriers to filling the gap include EA stakeholders’ different understandings of and expectations on the quality of science, the role of scholars in EA, and the purpose and objectives of EA. These disagreements imply insufficient and/or ineffective communications among EA stakeholders, which should be addressed if a more collaborative arrangement is to be built for a better quality of science in EA.

As for the limitations, due to time constraints, this study collected 56 online survey responses (35 EA scholars and 21 EA practitioners) and interviewed 13 Canadian EA professionals. Therefore, the small sample size of online surveys is not enough for a good statistical representation of all EA scholars and practitioners throughout the world. In addition, the countries and/or continents that the survey respondents mostly work in on EA projects are different (Figure 1), which may also contribute to their different perceptions of the quality of science in EA due to different EA conditions in these countries and/or continents. All interviewees were selected from Canada, and the EA-related working background of all interviewees is mostly in Canada. These factors may generate bias in our findings.

Future research on diverse views about science in EA is suggested to focus on (1) how to promote mutual understandings and effective communications among EA stakeholders so as to create a solid foundation for their potential collaborative working relationships; (2) how to promote the participation of EA scholars in the EA process in order to improve the scientific foundation for EA and generate a high-quality EA discourse; (3) how to improve the expectations of regulators on the quality of science in EA; and (4) how to mediate adverse tensions in the relationships of various EA stakeholders, especially when they have conflictive values, beliefs, and purposes.

Acknowledgments

We would like to thank the International Association for Impact Assessment (IAIA) for distributing online survey links through its normal distribution list and monthly e-news.

Disclosure statement

No potential conflict of interest was reported by the authors.