https://orcid.org/0000-0003-0005-6875
ABSTRACT
Environmental Impact Assessment (EIA) in Indonesia has been established for over 36 years. During this time, the EIA system has evolved to reflect both political and bureaucracy changes. However, the current implementation and effectiveness of the EIA process have not yet to be fully understood. This study aims to review the current implementation of EIA, to assess the effectiveness of EIA process procedurally, and to provide recommendations. In order to examine the effectiveness of EIA, we evaluate and compare the Environmental Impact Report (EIR) of nickel smelters against criteria. The results indicate that the screening and scoping process encountered problems in terms of scientific database availability, cost, and time of the study. Public participation was not effective due to the constraints of information channeling and weaknesses of the discussion process. We found insufficient synchronization between the evaluation and monitoring processes. EIA reporting indicated redundancy, inconsistency, and ambiguity. We recommend the political reinforcement of the will of the government to establish sustainable policy in terms of budgeting, human resources, and procedure of decision-making process in EIA.
Introduction
In Indonesia, a project proponent with specific qualifications is required to undertake an Analisis Mengenai Dampak Lingkungan ([AMDAL], in Indonesian, or EIA) as the principal environmental license before project implementation. EIA, as a decision-making support instrument, is also a management tool used to identify the impact of development projects and design mitigation measures to address potential impact (Morrison-Saunders and Fischer Citation2006; Kabir and Momtaz Citation2013; Jones and Fischer Citation2013, Citation2016).
Previous studies have been published on the topic of Indonesia’s EIA system. Purnama (Citation2003) reviewed the Indonesian EIA system in the period before 2003, and Boyle (Citation1998) highlighted problems with government bureaucracies in Indonesia’s EIA system. Quality documentation of social impact assessment was one of the least addressed aspects of impact assessment in Indonesia (Walker et al. Citation2000). Other studies have highlighted the potential improvement of citizens’ participation by using advanced information technology (Persada et al. Citation2014, Citation2015; Nadlifatin et al. Citation2015). Meanwhile, Gore and Fischer (Citation2014) reported explicitly on a case study of the tsunami Aceh to identify the importance of the implementation of environmental assessment and actors that can be helpful or disruptive in the following disaster. Syafiq (Citation2015) demonstrates significant issues, particularly failure in protecting the marine environment in a case study of Buyat Bay North Sulawesi Province and identified influential factors for improving the effectiveness of marine EIA. However, few papers focused on the details of the current execution of Indonesia’s EIA system and employed a logic model that leads to more profound understanding effectiveness within an impact assessment process and systems. Additionally, the previous studies did not consider diverse key-stakeholders’ perceptions.
To foster the quality of the EIA system, studies on environmental impact should focus on the decision-making process and explore practical methods for increasing public acceptance of EIA proposals (Nishikizawa Citation2015). Besides, detailed research based on case studies is also needed to provide an indicator of the performance of the EIA system. Globally, some assessments of the performance of EIA systems has focused on the quality of EIA reports and their technical contents. In this study, we focused on examining the current practices of EIA and evaluating the effectiveness of EIA process quantlitatively. We also provide structural and technical recommendations for improving Indonesia’s EIA implementation.
In general, researchers agree that EIA effectiveness is multidimensional and it has been delineated into four dimensions: procedural, substantive, transactive, and normative (Sadler Citation1996; Baker and McLelland Citation2003; Bond et al. Citation2013; Chanchitpricha and Bond Citation2013; Rozema and Bond Citation2015; Loomis and Dziedzic Citation2018). For this research, we focus on the procedural effectiveness of impact assessment; this is directly tied to how well different phases of the EIA process are designed, implemented, and enforced against criteria (Sadler Citation1996; Baker and McLelland Citation2003; Loomis and Dziedzic Citation2018).
Methodology
This research employed a popular method for evaluating the effectiveness of EIA systems procedurally by taking a ‘“multi-dimensional approach”’, including stakeholders’ perceptions (Chanchitpricha and Bond Citation2013) and applying the framework or criteria to suitable case studies by reviewing documents of impact assessment qualitatively through a specific case study (shown in ). These approaches are often supplemented with stakeholder interviews and field surveys (Loomis and Dziedzic Citation2018). This research has chosen a specific mining activity project for nickel smelter development to enable a deeper understanding of the main elements that influence the overall quality of EIR (Pinho et al. Citation2007).
Figure 1. Research methodology.
On a process level, the EIR represents a reflection of EIA in practice, and the quality of the EIR is an indicator of the effectiveness of an EIA system (Nadeem and Hameed Citation2006; Badr Citation2009; Ruffeis et al. Citation2010; Morgan Citation2012; Al-Azri et al. Citation2014). The general structure of EIR in Indonesia appears in . In order to examine the present status of Indonesia’s EIA policy and its implementation, it is necessary to analyze key elements of EIA based on four criteria derived through qualitative content analysis and fieldwork. The four criteria refer to the stages in conducting an EIA: screening and scoping practice, the involvement of stakeholders in the public participation process, considering the evaluation, mitigation and monitoring actions of impact, and delivering the report to the public.
Table 1. General structure of the EIR in Indonesia.
Initially, a set of analytical criteria was selected to evaluate the performance of EIA systems in the four identified stages. Most evaluation criteria (see ) used in this research were derived from previous work on effectiveness research including Wood (Citation2003), Nadeem and Hameed (Citation2008), Toro et al. (Citation2010), Clausen et al. (Citation2011), Hapuarachchi (Citation2014), and NCEA (Citation2014).
Table 2. Evaluation criteria to review performance of EIA.
These evaluation criteria were analyzed and discussed vis-à-vis the particular Indonesian EIA policy and guidelines (as described in ) in the following section.
Table 3. Principal EIA regulation and guidelines in Indonesia.
This research was made possible because of our access to complete EIR documents from five nickel smelter plants that are located in the Sulawesi and Java Islands, which all included comparable EIA reports. For the case studies, the project promoters were drawn from different companies to avoid any overlaps. However, overlapping did occur because several local consultant members had also compiled EIR for the same three projects that were located in the same province. Selecting local consultants would provide benefits, including developing a deep understanding of socio-economic and cultural environmental conditions. However, there are times when using local consultants may not be beneficial, for example, when projects use advanced technology of which local consultants have inadequate knowledge.
For data collection, we also conducted a few interviews that involved semi-structured and generally open-ended questions, to obtain the opinions of the participants (Creswell Citation2009). From October to November 2017, we conducted interviews with respective respondents (see ) in several cities such as Jakarta, Depok, Serang, Cilegon, Yogjakarta, Gresik, Surabaya, and Kendari. The selected respondents represent the government, experts, and the smelter industry. In Indonesian society, there are two types of leaders: formal leaders (i.e. local government officials) and informal leaders (i.e. religious leaders or chieftains). Some of these respondents represented the formal local sector.
Table 4. Characteristics of respondents.
Outline of case studies
We chose the case studies of nickel processing (see ) primarily because of the variety of intermediate and final products made and alternative technologies used to reach the final stage. Moreover, the development of nickel smelter project in Indonesia has rapidly progressed after 2014. According to MoE Decree No 5/2012, nickel smelter project is the type of activity that has to prepare EIA report. However, few studies on nickel smelter have been published as a study topic both from an environmental and socio-economic perspective. Four nickel smelters are located on the Sulawesi island, where there are many nickel mines in operation. The nickel smelters in the case studies all use pyro-metallurgy technology with different annual production capacities and final products. Technological options for processing generally cite the specific parameters of the project, so there are a lot of different processing options used in operations (Norgate and Jahanshahi Citation2011)
Table 5. Project descriptions.
Results and discussion
Screening and scoping
In general, screening instruments are ultimately an evaluation of environmental ‘significance’ regarding development activities (Bidstrup Citation2017). The screening process also defines the size or magnitude of the project, the type of pollutants, and scientific rationale. In case of non-compliance, the EIA application will be rejected. Screening instruments seek to determine whether an EIA is needed for the proposed project (Koornneef et al. Citation2008) and to ensure the design of activities will take environmental sustainability into account (Nielsen et al. Citation2004).
The mining industry, like nickel smelting, is an activity of processing and refining metal minerals where ore processing and refining activity has adverse effects on the natural, social, and cultural environments (Appiah and Osman Citation2014). Another concern is the uncertainty of available technological capabilities to overcome the significant negative impacts expected to arise.
Besides, EIA’s scoping process regulations must be undertaken for all obligated activities and subject to environmental permits. The scoping phase starts with the public notification of ToR (Koornneef et al. Citation2008), in which the project initiator describes several aspects such as the business plan and environmental setting, project site boundaries, and possible significant environmental impacts (Hansen and Wood Citation2016). During public notification, residents and all parties involved are entitled to be notified of guidelines that will be specified in the report.
According to the evaluation of scoping from , the survey of significant impacts on the environment, and socio-economic and cultural milieu was shown in all case studies. The result of the survey on environmental components from three project areas (S-3, S-4, and S-5) showed that method, location, and time were not clearly described in the scoping documents. However, for the vast majority of locations, social survey plans were fully complete and had already followed scientific standards, except the S-4 nickel smelter plant.
Table 6. Matrix scoping of nickel smelter case studies.
When preparing documents in the scoping stage, the project proponents must follow guidelines stipulated by the government. However, the majority of EIA project documents have not followed these guidelines, such as the smelter did not clearly state whether their project was following spatial planning regulation (S-1). Moreover, some documents did not provide a complete and adequate description of environmental aspects, for instance, the geophysical description (S-2), the condition of the vegetation and fauna (S-3), water quality and noise and soil quality at the project location (S-5). Baseline data is fundamental as reference for the next stages, such as mitigation and monitoring activities. From , S-5 smelter plant has the least data in comparison to the other smelters. Although S-5 is located in an industrial estate, it should still provide sufficient data to meet the regulatory criteria.
We can see from that the case study of S-4 had the highest percentage of partial coverage data. The S-4 smelter project also had the largest production capacity per year, among others (). The S-4 EIA report indicated that this smelter project is incomplete and lacks detail in terms of sampling size, scale, and the number of locations, compared to the other projects.
The scoping stage should be carried out comprehensively and following relevant rules (e.g. Decree of MoE No 16/2012), and scientific principles. The lack of data also constrains the application of predictive quantitative models as core valuation techniques. The transparency of alternatives is maintained, and systematic environmental analyses are conducted concerning a limited amount of choices (Lyhne Citation2011). Generally, Indonesia also has a problem with scientific databases retaining and storing data in an accessible form (per the of interviews with respondents NG(1), LG(1), LG(3), LG(4), LG(5), and LG(6)). In practice, as mentioned by respondents NG(1) and LG(6), copy and pasting of previous reports was standard. The project proponent’s reasons were the cost to conduct a detailed study, limited time, and lack of expertise. In the end, the received view of the proponent on scoping reinforced the notion that the EIA process was an administrative and financial burden on development (Hansen and Wood Citation2016)
Public participation
In Indonesia’s EIA regulation, public participation is mandatory for all project proponents to describe their activities and get feedback from the community. Public consultation in Indonesia includes two components: a public hearing and written comments to the EIR. Public participation has been prevalent across these public hearings, predominantly by local people near the project site.
Public participation activities for the five nickel smelter plants can be seen in . Essentially, the purpose of public participation is to achieve power-sharing among the people, industries, and government agencies through a fair decision-making process (O’Faircheallaigh Citation2010). Ideally, public participation and consultation are an arena of ideas and comments by experts and involve the public openly. All stakeholders have a right to be involved in the preparation of the EIA process and the freedom to provide feedback (Marshall et al. Citation2005). The public should also be able to contribute their abilities to the making of better-quality decisions (Lockie Citation2001). In practice, the elections and involvement of community representatives are characterized by pseudo-participation rather than broad participation (Marzuki Citation2009). The involvement of the public is used merely as an additional data source or means to educate the affected people. This provision is not actively enforced, and consultation is mostly limited to experts. In Indonesia’s context, public involvement is only a formal process that implements public participation guideline.
Table 7. Public participation of EIA process on case studies.
Regarding the number of participants, the activities were only attended by between 33 and 53 local participants from several stakeholders. The public hearings for all projects were attended by all categories of stakeholders, including local leaders, environmental organizations (NGO), officials of concerned government agencies, and academics. In practice, those who participated were directly invited by the technical committee of the environmental agency as officials, which means they represented each party (per interviews with NG(1), LG(1), LG(2), LG(2), LG(6), LG(7), and NI(1)). The number of participants was limited by two inhibiting factors, namely the financial burden of the project proponent for preparing the public participation activity, and ineffective publicity.
By law, Indonesia’s EIR must be disclosed, and public opinions accepted at any time during any of the stages of the EIA process. The public has a right to access all the EIA documents and give comments to decision-makers any time (Suwanteep et al. Citation2017). Furthermore, open public access to information can improve local capacity development with benefits for project owners, regulators, and local communities (Arts et al. Citation2001). In reality, public access to these reports is relatively limited because they can only be viewed during office hours. There is also a need for governmental agencies and project proponent to assist in translating EIR in local and non-technical language (Nadeem and Fischer Citation2011; Kengne et al. Citation2013). Project owners should announce relevant information through compatible and reachable media. The media can describe the negative or positive image of a project and become an early detection of potential social conflicts that may occur (Dare et al. Citation2014). In places where illiterate members of communities are dominant, newspaper adverts may be complemented by traditional village announcements (Jha-Thakur Citation2011). In some cases, many residents missed published notices because they do not read newspapers regularly.
During the public hearing sessions, an economic issue about a local labor provision and compensation package became a major concern for residents in all nickel smelter projects (). Communities, in general, only pay attention to the environmental changes that will occur due to the waste generated by the activity. Particularly in mining communities, public awareness, and involvement in the EIA process is limited to a few elites (Appiah and Osman Citation2014).
Table 8. Public concerns and their consideration in the EIA report.
Guidelines for public participation is defined in the decree of MoE No. 17/2012. This decree explains every proposed project that requires EIA must be announced and facilitates the expression of local people’s views and interests. Briefly, the guidelines for public involvement has four obligations. They are (1) to ensure the information disclosure for the community; (2) to protect the interests of the public; (3) to empower the community in the decision-making process, (4) to conduct public involvement in the environmental permit process. From a democratic point of view, an inclusive approach towards EIA participation seems reasonable (Rega and Baldizzone Citation2015). Adequate mechanisms to carry out public participation can develop an increase in participation and avoid potential conflicts between stakeholders (Bond et al. Citation2004), in particular, by strengthening the involvement of local stakeholders in playing more active roles and responsibilities in decision making related to shared welfare (Broeder et al. Citation2016). In many developed countries, the public is supposed to have a considerable influence on the EIA process, while in the majority of developing countries including Indonesia, public participation is often treated as a procedural exercise instead of a living process (Nadeem and Fischer Citation2011).
Public participation in Indonesia’s mineral industry is dominated by a ‘political-economy game.’ In Indonesia, the involvement of the public in the decision-making process is constrained by an inflexible procedural mechanism that marginalizes the public. Project owners want a smooth and straightforward decision-making process. In public hearings, project owners highlighted the economic and social benefits for people only once project development had been implemented. Unfortunately, they did not adequately explore the environmental and health impacts. Therefore, other stakeholders have accused the project owners of neglecting some essential elements in the public’s interest during the decision-making process; for example, the handling of emissions from smelter processing activities.
On the other hand, a low education level within the affected community is usually the cause of a lack of knowledge about environmental regulations (Purnama Citation2003) and mineral processing impacts. Unstable economic conditions of people who live around the smelter location are perhaps the primary constraint for effective public involvement. A balanced position of political power to all stakeholders should be enhanced in public participation, including the affected community.
Implementation of impact evaluation, mitigation measures, and monitoring
The process of evaluating all potential impacts was developed with scientific methods, discussions with experts, and public consultations involving relevant stakeholders. This step produces a list of potential impacts without considering the magnitude and the importance of each. The evaluation of the potential impacts has separated the type of effects that need to be explored in detail with several considerations, including conformity with regulations, level of community concern, and the socio-economic and ecological value of these components. Ultimately, a list of important hypothetical impacts was reviewed in more detail, with several levels ranging from no significant impact to significant impact with low, medium, and high categories (as shown in ).
Table 9. Matrix of impacts evaluation and monitoring plan.
The evaluation of essential impacts on the construction stage focuses more on socio-economic impacts, while the environmental impact, especially for the emergence of hazardous waste, is not considered. In the operation stage has the same pattern as the construction stage in terms of the types of impacts that were considered potentially crucial with emphasis an assessment of water quality and noise disturbance. In the post-operation stage, stakeholders have concluded that the socio-economic aspects are potentially more critical. Meanwhile, they did not consider the environmental impact to be a significant problem.
In the stages of construction and operation, job provision and business opportunities were determined as a potentially significant impact. In detail, the job provision of the S-5 project was determined to be higher than other four projects due to this smelter location being close to the center of the provincial capital, which is densely populated and with a higher number of job seekers than others. While smelter S-1, S-2, S-3, and S-4 are located in the remote area of Sulawesi island which has much more lower number of population, compared with Java island. The fewer population reduce the job seekers competition to be accepted as a worker of the nickel smelter project.
From all stages of the project, stakeholders of nickel processing appear to have a sceptical view of the environmental impact problem, but they are more attracted to socio-economic impacts, specifically job provision and business activities. Indeed, considering the social factors of the project is essential because it can increase the effectiveness of impact mitigation and prevent the emergence of other adverse impacts (Hanna et al. Citation2016). This situation is germane to developing countries, which still focus more on economic development than environmental aspects.
Monitoring as a mitigation activity is based on an evaluation report of significant impacts that have been agreed upon by stakeholders for ensuring the future quality of EIAs (Jha-Thakur Citation2011). Monitoring is understood to be a decisive factor. During periodic monitoring of EIA execution between three to six months per year, the project owner should report the frequency of significant impacts of physical, biological, or socio-economic nature to the environmental agency of the government. This agency then will evaluate and inspect on-site to ascertain the validity of the report (Toro et al. Citation2010). Then there should be complete conformity and consistency between impact evaluation and the monitoring process.
We calculated conformity analysis to evaluate how the level of consistency and the relationship between evaluation and monitoring impacts the EIA document. In the conformity analysis, factors like magnitude, scale, and frequency of impacts were not assessed; thus, the effects are not fully reported. The partial conformity of significant impacts describes the percentage of linkages between the total number of corresponding component responses listed in the monitoring process, divided by the amounts of actual significant impacts specified in the EIR.
From these findings, we can see that the S-1 project had low partial conformity, but the percentage of cumulative conformity was higher than other smelters. In contrast, on S-3 project that had high partial conformity, cumulative conformity was the lowest. If the percentage of partial conformity is high, it shows a higher focus on monitoring activities operationally, which increases the effectiveness of EIR. The percentage of cumulative conformity, however, indicates the consistency level of the monitoring report on significant impact within the evaluation process. We found that using separate documents of evaluation and monitoring at different times and places generates non-conformity in the report. Ideally, monitoring activities should be a continuation of the output of the evaluation activities.
According to the results of interviews with NG(1), LG(1), LG(3), LG(6), LG(7), LG(9), EX(3), and NI(1), we found that the implementation of monitoring has become a fundamental and the most critical problem in the Indonesian EIA system. Monitoring activities tend to merely fill the environmental agency’s archives, as opposed to informing better decisions. The main reason has been the lack of law enforcement, even though the authority of the environmental inspector is considerable. MoEF (Citation2017) identified five determinant factors which impede the enforcement and compliance of environmental regulations in Indonesia, namely: human resources not having the necessary capacity and capability, incomplete institutional structures with small data networks, shortcomings of regional governments when undertaking authority, limited budgets, and lack of technology utilization. Moreover, according to the results of interviews, another veiled and influential factor was political intervention. Achieving effective monitoring and enforcement, thus requires a robust regulatory framework, clear environmental standards, adequate staff, financial resources, and administrative and political will. The absence of some or all of these factors has a bearing on compliance monitoring and enforcement practices (Smith and Rosenblum Citation2011).
Reporting
The overall EIR documentation of the study sites appears in , which shows that the S-3 and S-4 reports had the highest and lowest total number of pages, with 841 and 495, respectively. Meanwhile, the EIAn section accounted for most substantial proportion of all projects (between 47% and 63% of the total report). The EIAn section discusses impact mitigation that is supported by data, such as results of sampling for each environmental, socio-economic, and cultural component, community interviews, etc.
Table 10. Number of EIR pages of case study.
The smelter S-4 has the least number of report pages compared to other projects, but it has the most prominent annual production capacity . Referring to the ToR document, smelter S-4 had the most deficient baseline environmental and socio-economic data during the scoping stage. For instance, vegetation and fauna data were not completed at all. Moreover, data on air, water, and soil quality were insufficient in terms of scale, magnitude, and sampling locations. Data obtained at the scoping stage will affect further steps, such as the evaluation and monitoring of impacts. Indeed, incomplete data will result in fewer pages in the total page of the report.
The factors that determine the overall quality of EIR relates to the relevant information and assess the significance impacts clearly and precisely. A short EIR is only cover an insufficient level of information and inadequate level of detail. On the other hand, a longer EIR does not necessarily achieve the goal more effectively than a concise one. EIRs of case studies showed that although the total pages of the report were long, the quality of data was questionable. If EIRs that are excessively long contain large amounts of irrelevant information and are consequently downgraded in their quality assessment, decision-making can become problematic regarding the planning and approval of projects (Cashmore et al. Citation2002; Glasson et al. Citation2005; Kabir and Momtaz Citation2013).
Conclusions and recommendations
To evaluate the performance of Indonesia’s EIA system, we employed a qualitative content analysis on the pivotal phases of EIA, reviewing official documents and investigating complete EIRs from case studies of mineral processing activities. We concluded that the nickel smelter projects experienced difficulties in conducting the screening and scoping stages, mainly due to the shortcomings of the scientific database. The results of scoping activities showed that the nickel smelters had already conducted socio-economic surveys according to scientific standards, while surveys of environmental components have faced cost constraints, limited time, and lack of qualified expertise.
From the case studies, we also identified some problems in the public participation activity associated with inadequate provision of information, incompatible execution of participation methods, and inefficient decision-making processes. Stakeholders have marginalized public participation as only an administrative process, rather than a participatory planning and decision-making process.
In addition, we found a discrepancy between the impact evaluation process and monitoring activities in the case study. Evaluation of the critical impacts of nickel smelter activities emphasized job provision while ruling out the potential presence of hazardous waste in the operation stage. Further, we examined the less synchronous and ineffective implementation of the monitoring process. Regarding the reporting aspect, we identified various areas of duplication, inconsistency, and ambiguity.
According to the evaluation of case studies, we argue that Indonesia should prioritize the revitalization of mechanisms to improve EIA implementation in specific aspects. The following proposed plan for the mineral processing industry is recommended, with a focus on technical actions, to improve the effectiveness of the EIA system in Indonesia:
The low accuracy of the data from the scoping process will affect the next stage in the evaluation and mitigation of the critical impacts of the activity. EIA reviewers should have the ability to identify these weaknesses from the start so that they can be considered during the decision-making process.
MoEF, together with local government agencies and universities, needs to continue developing a complete and easily accessible scientific database on a web-based system. The integrated web-storing system will provide extensive media knowledge sharing for project promoters and consultants when they construct the EIR.
In order to overcome low involvement of local people, government agencies should collaborate with project promoters to ensure that the local community can readily determine how major issues will be addressed by translating the language to a local or familiar dialect as well as simplifying the communication. The government should also evaluate the utilization of local newspapers and make media more accessible for the entire local community. The government could employ social media, such as Facebook and Twitter, to spread information about the urban community; for those in remote areas, there is usually a community institution for communication.
We suggest establishing a continuous reporting method by combining EIAn and EIMP into a single document to avoid redundancy and reduce non-conformity at every stage
Regarding law enforcement, we recommend that the government should provide and allocate sufficient funds for monitoring activities. We also suggest that the local government develops capacity and capability periodically by sending inspection staff to undertake training in specific skills.
Acknowledgments
We would like to thank the Ministry of Energy and Mineral Resources of the Republic of Indonesia for their financial support provided for fieldwork. We also would like to express our gratitude to the anonymous reviewers for their suggestions for improving this paper.
Additional information
Funding
References
- Al-Azri NS, Al-Busaidi RO, Sulaiman H, Al-Azri AR. 2014. Comparative evaluation of EIA systems in the Gulf cooperation council states. Impact Assess Project Appraisal. 32(2):136–149.
- Appiah DO, Osman B. 2014. Environmental impact assessment: insights from mining communities in Ghana. J Environ Assess Policy Manag. 16(4):1450031-1–1450031-20.
- Arts J, Caldwell P, Morrison-Saunders A. 2001. Environmental impact assessment follow-up: good practice and future directions findings from a workshop at the IAIA 2000 conference. Impact Assess Project Appraisal. 19(3):175–185.
- Badr EA. 2009. Evaluation of the environmental impact assessment system in Egypt. Impact Assess Project Appraisal. 27(3):193–203.
- Baker DC, McLelland JN. 2003. Evaluating the effectiveness of British Columbia’s environmental assessment process for first nations’ participation in mining development. Environ Impact Assess Rev. 23(5):581–603.
- Bidstrup M. 2017. The ‘grey’ assessment practice of IA screening: prevalence, influence and applied rationale. Environ Impact Assess Rev. 62:233–239.
- Bond A, Morrison-Saunders A, Stoeglehner G. 2013. Designing an effective sustainability assessment process. In: Bond A, Morrison-Saunders A, Howitt R, editors. Sustainability assessment pluralism, practice and progress. Oxon (UK): Routledge, Taylor & Francis Group; p. 231–244.
- Bond A, Palerm J, Haigh P. 2004. Public participation in EIA of nuclear power plant decommissioning projects: a case study analysis. Environ Impact Assess Rev. 24(6):617–641.
- Boyle J. 1998. Cultural influences on implementing environmental impact assessment: insights from Thailand, Indonesia, and Malaysia. Environ Impact Assess Rev. 18:95–116.
- Broeder LD, Chung KY, Geelen L, Scholtes M, Schuit AJ, Wagemakers A. 2016. We are all experts! does stakeholder engagement in health impact scoping lead to consensus? A Dutch case study. Impact Assess Project Appraisal. 34(4):294–305.
- Cashmore M, Christophilopoulos E, Cobb D. 2002. An evaluation of the quality of environmental impact statements in Thessaloniki, Greece. J Environ Assess Policy Manag. 4(4):371–395.
- Chanchitpricha C, Bond A. 2013. Conceptualising the effectiveness of impact assessment processes. Environ Impact Asses Rev. 43:65–72.
- Clausen A, Hoa VH, Pedrono M. 2011. An evaluation of the environmental impact assessment system in Vietnam: the gap between theory and practice. Environ Impact Asses Rev. 31(2):136–143.
- Creswell JW. 2009. Research design: qualitative, quantitative, and mixed methods approaches. 4th ed. Thousand Oaks (CA, US): Sage Publications, Inc.
- Dare ML, Schirmer J, Vanclay F. 2014. Community engagement and social license to operate. Impact Assess Project Appraisal. 32(3):188–197.
- Glasson J, Therivel R, Chadwick A. 2005. Introduction to environmental impact assessment third edition. New York (NY): Routledge.
- Gore T, Fischer TB. 2014. Uncovering the factors that can support and impede post-disaster EIA practice in developing countries: the case of Aceh Province Indonesia. Environ Impact Assess Rev. 44:67–75.
- Hanna P, Vanclay F, Langdon EJ, Arts J. 2016. The importance of cultural aspects in impact assessment and project development: reflections from a case study of a hydroelectric dam in Brazil. Impact Assess Project Appraisal. 34(4):306–318.
- Hansen E, Wood G. 2016. Understanding EIA scoping in practice: a pragmatist interpretation of effectiveness. Environ Impact Asses Rev. 58:1–11.
- Hapuarachchi AB. 2014. Effectiveness of environmental impact assessment (EIA) in addressing development-induced disasters: a comparison of the EIA processes of Sri Lanka and New Zealand [dissertation]. Christchurch: Lincoln University.
- Jha-Thakur U. 2011. Environmental impact assessment follow-up in India: exploringregional variation. J Environ Assess Policy Manag. 13(3):435–458.
- Jones JP, Fischer TB. 2013. EIA for wind farms in the United Kingdom and Germany. J Environ Assess Policy Manag. 15(2):1340008-1–1340008-30.
- Jones R, Fischer TB. 2016. EIA follow-up in the UK -A 2015 update. J Environ Assess Policy Manag. 18:1650006-1–1650006-22.
- Kabir SMZ, Momtaz S. 2013. 15th years of environmental impact assessment system in Bangladesh: current practice, challenges and future directions. J Environ Assess Policy Manag. 15(4):1350018-1–1350018-30.
- Kengne CVN, Evouna SEM, Bitondo D. 2013. Public hearings in environmental and social impact assessment for energy sector projects in Cameroon. Impact Assess Project Appraisal. 31(1):64–73.
- Koornneef J, Faaij A, Turkenburg W. 2008. The screening and scoping of environmental impact assessment and strategic environmental assessment of carbon capture and storage in the Netherlands. Environ Impact Assess Rev. 28:392–414.
- Lockie S. 2001. SIA in review: setting the agenda for impact assessment in the 21st century. Impact Assess Project Appraisal. 19(4):277–287.
- Loomis JJ, Dziedzic M. 2018. Evaluating EIA systems’ effectiveness: a state of the art. Environ Impact Asses Rev. 68:29–37.
- Lyhne I. 2011. Between policy-making and planning SEA and strategic decision-making in the danish energy sector. J Environ Assess Policy Manag. 13(03):319–341.
- Marshall R, Arts J, Morrison-Saunders A. 2005. International principles for best practice EIA follow up. Impact Assess Project Appraisal. 23(3):175–181.
- Marzuki A. 2009. A review on public participation in environmental impact assessment in Malaysia. Theor Empir Res Urban Manag. 3(12):126–136.
- Ministry of Environment and Forestry [MoEF]. 2017. Laporan Tahunan 2017 [Annual report 2017]. Jakarta. [publisher unknown].Indonesian.
- Morgan RK. 2012. Environmental impact assessment: the state of the art. Impact Assess Project Appraisal. 30(1):5–14.
- Morrison-Saunders A, Fischer TB. 2006. What’s wrong with EIA and SEA anyway? A sceptic’s perspective on sustain- ability assessment. J Environ Assess Policy Manag. 8(1):19–39.
- Nadeem O, Fischer TB. 2011. An evaluation framework for effective public participation in EIA in Pakistan. Environ Impact Asses Rev. 31(1):36–47.
- Nadeem O, Hameed R. 2006. A critical review of the adequacy of EIA reports-evidence from Pakistan. Int J of Hum Soc Sci. 1(1):54–61.
- Nadeem O, Hameed R. 2008. Evaluation of environmental impact assessment system in Pakistan. Environ Impact Asses Rev. 28(8):562–571.
- Nadlifatin R, Razif M, Lin SC, Persada SF, Belgiawan PJ. 2015. An assessment model of Indonesian citizens’ intention to participate on environmental impact assessment: a behavioral perspective. Proceedings of the 5th sustainable future for human security (SustaiN 2014). Procedia Environ Sciences. 28:3–10.
- Netherlands Commission for Environmental Assessment [NCEA]. 2014. Systems approach to EIA effectiveness. [publisher unknown].
- Nielsen EH, Christensen P, Kørnøv L. 2004. EIA screening in Denmark: a new regulatory instrument ? J Environ Assess Policy Manag. 7(1):35–49.
- Nishikizawa S. 2015. Environmental impact assessment research in Japan: retrospective and prospective. J Environ Assess Policy Manag. 17(1):1550013-1–1550013-9.
- Norgate T, Jahanshahi S. 2011. Assessing the energy and greenhouse gas footprints of nickel laterite processing. Miner Eng Spec Issue. 24(7):698–707.
- O’Faircheallaigh C. 2010. Public participation and environmental impact assessment: purposes, implications, and lessons for public policy making. Environ Impact Asses Rev. 30(1):19–27.
- Persada SF, Lin SC, Nadlifatin R, Razif M 2015. A lean based analysis on the Indonesian citizens’ participation in environmental impact assessment (EIA): a potential improvement through web based system and SMS-gateway. The Third Information Systems International Conference. Open Access Journal of Information Systems (OAJIS); Surabaya. Institut Teknologi Sepuluh Nopember (ITS).
- Persada SF, Razif M, Lin SC, Nadlifatin R. 2014. Toward paperless public announcement on environmental impact assessment (EIA) through SMS gateway in Indonesia. Procedia Environ Sciences. 20:271–279.
- Pinho P, Maia R, Monterrosoa A. 2007. The quality of Portuguese environmental impact studies: the case of small hydropower projects. Environ Impact Asses Rev. 27(3):189–205.
- Purnama D. 2003. Reform of the EIA process in Indonesia: improving the role of public involvement. Environ Impact Asses Rev. 23(4):415–439.
- Rega C, Baldizzone G. 2015. Public participation in strategic environmental assessment: a practitioners’ perspective. Environ Impact Asses Rev. 50:105–115.
- Rozema JG, Bond AJ. 2015. Framing effectiveness in impact assessment: discourse accommodation in controversial infrastructure development. Environ Impact Asses Rev. 50:66–73.
- Ruffeis D, Loiskandl W, Awulachew SB, Boelee E. 2010. Evaluation of the environmental policy and impact assessment process in Ethiopia. Impact Assess Project Appraisal. 28(1):29–40.
- Sadler B. 1996. International study of the effectiveness of environmental assessment: environmental assessment in a changing world. Canada: IAIA and Canadian Environmental Assessment Agency.
- Smith E, Rosenblum P. 2011. Government and citizen oversight of mining. Enforcing the rules. New York (USA): Revenue Watch Institute. [publisher unknown].
- Suwanteep K, Murayama T, Nishikizawa S. 2017. The quality on public participation in environmental impact assessment (EIA) reports in Thailand. J Environ Assess Policy Manag. 19(02):1750008-1–1750008-21.
- Syafiq M. 2015. Improving the effectiveness of environmental impact assessment in Indonesia regarding the protection of the marine environment from industrial pollution. J Environ Assess Policy Manag. 17(3):1550026-1–1550026-17.
- Toro J, Requena I, Zamorano M. 2010. Environmental impact assessment in Colombia: critical analysis and proposals for improvement. Environ Impact Asses Rev. 30(4):247–261.
- Walker JL, Mitchell B, Wismer S. 2000. Impacts during project anticipation in Molas, Indonesia: implications for social impact assessment. Environ Impact Asses Rev. 20(5):513–535.
- Wood C. 2003. Environmental impact assessment in developing countries. Int Dev Plan Rev. 25(3):301–321.