Towards effective scoping in strategic environmental assessment

Abstract

The scoping phase in strategic environmental assessment (SEA) is considered a central issue for the effectiveness of the whole SEA process, enhancing decision-making. Understanding SEA should be context-specific and scoping is paramount for a robust SEA; the main goal of this research was to develop a framework for the evaluation of the procedural effectiveness of the scoping stage in a specific political and planning system context – Portugal. The framework developed comprises a set of 21 criteria divided into six broad themes and was applied to 20 Portuguese SEA scoping reports, through content analysis. Overall, it was found that mandatory issues in the Portuguese SEA context were met and the scoping reports used mainly national guidelines. However, issues of public participation are often discarded and there is no evidence that scoping starts early in the decision-making process, or if it is done concurrently. Nonetheless, the framework developed establishes best practice for the SEA scoping phase, needing to be adapted to the different political and planning system contexts in order to reflect legal and institutional specificities.

Keywords::

• strategic environmental assessment
• effectiveness evaluation
• scooping
• criteria
• content analysis

1. Introduction

In recent years, a concern about strategic environmental assessment (SEA) effectiveness has emerged and this subject has been largely addressed and discussed in the institutional and scientific literature (e.g. Fischer 2002a; Retief 2007; SEPA 2011; EPA 2012). Sadler (1996) identified three effectiveness categories for environmental assessment (EA): procedural, substantive and transactive. Later, Baker and McLelland (2003) added normative effectiveness. In 2013, Chanchitpricha and Bond (2013) summarised these concepts, together with different authors' perspectives on effectiveness definitions and categories for impact assessment processes.

Even though all types of effectiveness are essential for ensuring sustainability is integrated into the decision-making processes, procedural effectiveness is the starting point to an effective SEA and evaluates if ‘the EA [environmental assessment] process conform[s] to established provisions and principles’ (Sadler 1996, p. 39), helping set the standards and higher goals for SEA practice (Sadler 1996).

As noted by Polido et al. (2014), different authors invoke the context-specificity of SEA, relating it to: the political and planning system (Fischer & Gazzola 2006; Bina et al. 2011; Partidário 2011), the type of decision-making addressed (Fischer 2002b; Hilding-Rydevik & Bjarnadóttir 2007), the context of professional practice and capacity (Hilding-Rydevik & Bjarnadóttir 2007; Partidário 2011) or the territorial context (Fischer 2002b).

Simultaneously, procedural effectiveness is also context-specific because it relates to institutional and organisational frameworks of different political and planning systems (Sadler 1996; Fischer & Gazzola 2006), and is dependent on the availability of resources, public participation and the capacity of the professionals conducting the assessment (Chanchitpricha & Bond 2013). Therefore, it could be argued that it is necessary to develop a context-specific framework for its evaluation, as discussed by Bina et al. (2011).

Different authors (e.g. Söderman & Kallio 2009; Fischer 2010) and institutions (e.g. SEPA 2011; EPA 2012) have conducted SEA effectiveness reviews. The reviews used, as main research methods, stakeholder interviews and workshops (e.g. Department for Communities and Local Government 2010; SEPA 2011; EPA 2012) and content analysis of case studies (e.g. Fischer 2010; SEPA 2011). The main issues identified from these reviews, focusing on the scoping stage, relate to: (1) an early start to help SEA integration with decision-making and enhance SEA (Aschemann 2004); however, as stressed by EPA (2012), this should not start too early because there is the possibility of not knowing what information is needed and what is going to be assessed; (2) the need for focus on the environmental issues, objectives and indicators in scoping reports (Retief et al. 2008; SEPA 2011; EPA 2012) and structured scoping to help direct the assessment (Söderman & Kallio 2009); (3) the inclusion of alternatives in the scoping phase to understand the information needed for the assessment, as noted by the UK SEA review (Department for Communities and Local Government 2010); (4) public participation in SEA early stages, beyond the authorities with environmental responsibilities during the scoping phase, using engaging participatory methods, such as workshops or public meetings, which lead to awareness and supportive attitudes from stakeholders (SEPA 2011; EPA 2012); and (5) the preparation of a scoping report to enhance communication between SEA and stakeholders (Söderman & Kallio 2009; SEPA 2011), and even though EPA (2012) found scoping could be resource-intensive, SEPA (2011) notes this is less important compared with the benefits of a well-prepared scoping report.

Furthermore, guidelines for the scoping phase differ from author to author, or in different contexts (see ODPM 2005; OECD 2006), i.e. different authors discuss different ways of developing scoping, e.g. if scoping should be based on a rigid and structured process for certain contexts, or if it should take a flexible approach (Fischer & Gazzola 2006). Institutional guidelines (e.g. DEAT/CSIR 2000; ODPM 2005; GRDP 2006) give emphases to contextualisation issues, such as Plan or Programme (PP) content, objectives and strategic issues. The need to identify SEA objectives and indicators, the key environmental problem areas and baseline information, and integrate this information into the sustainability objectives and targets of other strategic actions is recommended by Therivel (2004) and Fischer (2007), and adopted by the institutional guidelines, such as the ones mentioned above.

The scoping phase is fundamental to an effective impact assessment process (Canter & Ross 2014; Morrison-Saunders et al. 2014), including SEA (von Seht 1999; Mulvihill 2003; Fischer & Phylip-Jones 2008). It determines the extent and level of detail of the assessment and the information to be included in the SEA (Abaza et al. 2004; Snell & Cowell 2006; van Doren et al. 2013), and ensures the whole process focuses on central issues. It is important to improve the SEA scoping phase to help enhance SEA practices and effectiveness (Mulvihill 2003; SEPA 2011; EPA 2012).

The purpose of the present research was to develop a framework for the evaluation of the procedural effectiveness of the scoping stage in a specific political and planning system context, and to apply the framework. The Portuguese case was selected to test the proposed approach. This national context reflects the European Union (EU) SEA Directive, transposed by 28 EU member-states (European Commission 2009; Justice & Environment 2012) and, at the same time, represents a southern European country where there is a lack of scientific literature that analyses, in depth, the effectiveness of SEA processes, including scoping reports, as noted by Fischer and Gazzola (2006). In Portugal, the EU SEA Directive was transposed late, in 2007, and only a few studies were carried out regarding the Portuguese SEA effectiveness (e.g. Partidário et al. 2010). Hence, this research presents an opportunity to help enhance Portuguese SEA practices. The application of the framework was done for a sample of 20 selected Portuguese SEA scoping reports (16 from spatial municipal plans (SMPs) and four from other types of plans – sector, regional spatial planning and special plan) through content analysis, which allowed exploration of how the scoping phase in SEA is developed and conducted in Portugal.

2. The Portuguese SEA system

Even though the EA of plans was a goal of the Environmental Basis Law from 19871 (Lei n. 11/87), the practice of SEA in Portugal is recent and had as its main driver Directive 2001/42/EC on SEA. The transposition of the SEA Directive was done late, only in 2007, through the Decree-Law n. 232/2007 (Decreto-Lei n. 232/2007).2 Furthermore, for the specific cases of spatial planning, SEA must also comply with the Decree-Law n. 380/993 (Decreto-Lei n. 380/99). The Decree-Law n. 232/2007 is flexible about the adaptation of the legislation into the Portuguese autonomous regions of the Azores and Madeira, and in 2010, the Azores adapted the SEA to the regional legal framework through the Regional Decree-Law n. 30/2010/A (Decreto Legislativo Regional n. 30/2010/A).

The national legal framework on SEA defines the PP promoter as the responsible authority for the SEA, i.e. it is the PP promoter that has to verify and decide if there is a need to carry out an SEA. However, if there are doubts in this self-screening procedure, the PP promoter can consult the public authorities with specific environmental responsibilities4 and an interest in the environmental effects of PP. If there is a need to carry out an SEA, then the PP promoter has to determine its scope. It is not mandatory to do a scoping report but it is usually done. Public authorities with an interest in the environmental effects of PP must be consulted on the scope.

Once the scoping stage is finished, the PP promoter has to develop the Environmental Report, which must reflect the inputs, including suggestions and comments, made by the public authorities on the proposed scope. The consultation on the Environmental Report is mandatory, engaging not only the public authorities but also the different interested stakeholders and the general public. Generally, it is the responsibility of the PP promoter to organise public consultations, but the regulations do not state the form they should take, whether through participatory workshops, dedicated web pages or other means.

At the end of the process, and after the PP approval, the PP promoter has to send the Portuguese Environmental Agency (APA – Agência Portuguesa do Ambiente) an Environmental Statement saying how the contributions of the SEA and respective consultations were integrated into the PP, and how it will be pursued in the follow-up stage. The follow-up stage is mandatory, and the PP promoter has to assess and control the effects of the PP on the environment. The follow-up results should be updated annually and available through ‘electronic means’ and, at the same time, sent to APA.

APA is responsible for the storage and treatment of SEA information, ensuring the dialogue and interchange of information with the European Commission and other interested parties (Decreto-Lei n. 232/2007). In order to comply with this role, it is stated that the PP promoters have to send APA the ‘necessary information’ by electronic means (Decreto-Lei n. 232/2007). APA assigns to the PP promoter the responsibility to have the information available, referring only to the final version of the PP and the Environmental Statement (APA 2014). As noted by Partidário et al. (2010), not being explicit about the need to deliver the full documentation of the SEA process makes it almost impossible to keep up to date a database of SEA information. Consequently, there is no centralised information system for consultation of past SEA cases, even though APA may have some available.

In 2010, in order to comply with Article 12(2) of the SEA EU Directive and Article 6(4) of the Decree-Law, APA hired a team of consultants to assess Portuguese SEA practice (see Partidário et al. 2010). That study assessed all the Environmental Reports, Non-technical Summaries and Environmental Statements received at APA from June 2007 to December 2009. The scoping reports were not analysed, and no formal justification was presented for this gap. The study (Partidário et al. 2010) was developed from a sample of 30 SEA processes, 24 SMPs (13 detail plans, seven municipal master plans and four urbanisation plans), four sector plans, one regional spatial planning plan and one programme on the application of the EU cohesion policy 2007–2010. For a detailed explanation of the Portuguese spatial planning system, please see, e.g., DGT (2014), Cardoso and Breda-Vázquez (2007) and Fidélis and Sumares (2008). Partidário et al. (2010) concluded that the SEA processes complied with the legislation, but the integration between SEA and Planning is poor. As for stakeholders' consultation, PP promoters see it as a legislative burden and do not encourage an active debate, or even carry it out at all (Partidário et al. 2010).

By 2010, SEA in Portugal was characterised by two main institutional practice guidelines: the Good Practice Guide published in 2007 by the APA (Partidário 2007), which was recently updated (Partidário 2012); and the Guide for EA of SMPs, from the Portuguese Directorate-General for Spatial Planning and Urban Development (DGOTDU 2008). The first guideline is generic, not PP-specific, while the latter is focused on SMPs (municipal master plans, urbanisation plans and detail plans).

3. Methods

It is necessary to have a flexible research approach when tackling SEA effectiveness evaluation (Retief 2007). Due to the context-specific characteristic of SEA, effectiveness frameworks must be developed for a specific context (Fischer & Gazzola 2006). To achieve the goals set out for this research, (1) a framework for scoping effectiveness through the establishment of a set of criteria and (2) the application of the criteria in 20 scoping reports from Portugal were developed.

3.1 Development of the scoping effectiveness framework

For the development of the scoping effectiveness framework the methodological approach used was based on an integrative literature review, which is a research method that aims to ‘generate new knowledge about the topic reviewed’ (Torraco 2005, p. 365). One of the possible outcomes from this research method is the development of a conceptual framework (Torraco 2005). The integrative literature review comprises: (1) identification of the documents to be reviewed (strategy for selecting literature), (2) analysis of the documents, and (3) synthesis and reporting of the information (development of the conceptual framework).

3.1.1 Strategy for selecting literature

The aim of the review was to find appropriate criteria to develop a framework for the evaluation of SEA scoping effectiveness. Hence, in order to find the appropriate literature for review, the approach presented in Figure 1 was developed. First, it was necessary to understand the boundaries of the type of effectiveness to be studied. As a starting point, the definition of procedural effectiveness chosen for the research was the following: was the SEA process accomplished in accordance with the established guidelines, procedures and criteria?

Figure 1 Conceptual model for developing criteria for scoping effectiveness framework. Source: Adapted from Polido and Ramos (2010).

Second, and to ‘answer’ to the procedural effectiveness definition, it was necessary to identify literature focused on international guidelines for SEA. It was impossible to identify this type of literature through Google Scholar or Scopus because these citation databases did not refer to institutional documents, and a search in a generic Internet search engine, such as Google, would retrieve thousands of results. In that case, the authors selected international guidelines based on the following criteria: (1) international recognised institutions and (2) geographically diverse. A total of six documents were retrieved (DEAT/CSIR 2000; Abaza et al. 2004; Ahmed et al. 2005; GRDP 2006; OECD 2006). A work by Schmidt et al. (2005), which focused on SEA in the EU (SEA EU Directive), was added to this set. It was necessary to include a document covering, in depth, the reality of EU SEA practice, since this subject was only represented through the SEA programme-specific guideline – the GRDP (2006).

At the same time it was also important to identify specific international literature on SEA scoping guidelines. Three guidelines were chosen, due to the historical connection to EA (CEQ 1981), the country's wide experience in sustainability appraisal (ODPM 2005), and for the country's commitment to becoming SEA ‘world leaders’ and, at the same time, regulations which go beyond the SEA EU Directive (Scottish Executive 2006).

Also, different criteria with a focus on SEA and scoping effectiveness, from the scientific literature in English, were searched. This search, supported by b-on5 (Online Knowledge Library), using the key-terms ‘effectiveness’ and ‘strategic environmental assessment’ in the ‘subject’, retrieved 15 documents. The abstracts were read to reject papers not relating to criteria for effectiveness in SEA, and if there were doubts about the scope of the paper, the entire paper would be scanned for a thorough analysis of the content. At the same time, papers addressing effectiveness criteria for the SEA scoping phase were also signalled. In the end, a total of five papers were considered. Furthermore, due to its relevance for the impact assessment community of researchers and practitioners, including SEA, the ‘SEA Performance Criteria’ (IAIA 2002) was introduced into this set, as were the works of Sadler (1996) and Therivel (2004).

Adding the component of context-specificity in the framework, the Portuguese guidelines (Partidário 2007; DGOTDU 2008) and the national SEA regulations were also reviewed.

3.1.2 Analysis of the documents

The analysis of the documents was performed using grounded theory. Grounded theory is a qualitative research method that allows exploratory and inductive research through the development of concepts and theories from data systematically gathered and analysed iteratively (Glaser & Strauss 1967; Strauss & Corbin 1998). The documents were coded extensively and identified aspects concerning effectiveness of the SEA scoping stage. Similar codes were then grouped and regrouped. In the end, six broad themes were retrieved, relating to different criteria (detailed in the next section).

3.1.3 Synthesis and reporting of the information

Through this approach, 21 criteria were established, divided into the six broad themes: (1) PP profile, (2) links with other strategic actions, (3) sustainability issues and SEA objectives, (4) baseline information, (5) communication and public participation, and (6) integration of SEA with the different phases of the PP. At the same time, an ‘Evaluation Guide’ for each criterion was developed, to help implement the content analysis to evaluate scoping effectiveness (see Table 1 and Section 3.2.).

Table 1 Framework and criteria for SEA scoping effectiveness.

Criteria	Evaluation guide	Adapted from
Plan or Programme (PP) profile
1. Describes the strategic mission and vision?	Verify if the scoping report describes the strategic mission and vision	DEAT/CSIR (2000); Partidário (2007); DGOTDU (2008)
	of the PP and the potential implications for the environment,
	societies and economy
2. Describes the PP goals and objectives?	Verify if the scoping report describes the goals and objectives	ODPM (2005); GRDP (2006); Scottish Executive (2006); Partidário
	of the PP: what it aims to achieve; and what justifies those aims	(2007); DGOTDU (2008)
3. Describes the PP content?	Verify if the scoping report describes the main strategic actions	ODPM (2005); GRDP (2006); Scottish Executive (2006); Partidário
	of the PP and the alternatives considered	(2007); DGOTDU (2008)
Links with other strategic actions
4. Refers to the relevant Policies, Plans or Programmes (PPP)?	Verify if the scoping report refers to the relevant PPP for the PP	Fischer (2002a); IAIA (2002); ODPM (2005); GRDP (2006); OECD
	being assessed	(2006); Scottish Executive (2006); Partidário (2007); DGOTDU
		(2008)
5. Refers to the sustainability and strategic objectives of the	Verify if the scoping report refers to the strategic and sustainability	Fischer (2002a); IAIA (2002); ODPM (2005); GRDP (2006); OECD
relevant PPP?	goals of the relevant PPP for the PP being assessed	(2006); Scottish Executive (2006); Partidário (2007); DGOTDU
		(2008)
6. Refers to how these PPP are linked with the PP under	Verify if the scoping report refers to how the strategic and	ODPM (2005); GRDP (2006); OECD (2006); Scottish Executive
evaluation?	sustainability goals of the relevant PPP are linked with the PP being	(2006); Partidário (2007); DGOTDU (2008)
	assessed
Sustainability issues and SEA objectives
7. Identifies the key sustainability and environmental issues	Verify if the scoping report identifies which are the key sustainability	Fischer (2002a); IAIA (2002); Abaza et al. (2004); Therivel (2004);
relevant for the SEA?	and environmental issues related with the PP	ODPM (2005); GRDP (2006); Scottish Executive (2006); Partidário
		(2007); DGOTDU (2008)
8. Defines the SEA objectives?	Verify if the SEA goals are defined	Fischer (2002a); IAIA (2002); Abaza et al. (2004); Therivel (2004);
		ODPM (2005); GRDP (2006); Scottish Executive (2006); Partidário
		(2007); DGOTDU (2008)
9. Identifies the indicators and sources of information for each	Verify if for each sustainability and environmental issue there	Fischer (2002a); IAIA (2002); Therivel (2004); Partidário (2007)
key sustainability and environmental issue?	are identified different indicators and their main source of
	information, to help evaluate those issues
Baseline information
10. Identifies the baseline information?	Verify if environmental aspects likely to be affected were identified,	DEAT/CSIR (2000); Abaza et al. (2004); Therivel (2004); ODPM
	according with the sustainability and environmental issues	(2005); GRDP (2006); Scottish Executive (2006); Partidário (2007);
		DGOTDU (2008)
Communication and public participation
11. Defines the communication strategy for the all SEA	Verify if a communication strategy is identified that defines: public	Baker & McLelland (2003); Schmidt et al. (2005); OECD (2006);
process?	consultation for all SEA phases, stakeholders to consult for all	Scottish Executive (2006); Partidário (2007); DGOTDU (2008)
	SEA phases, the communication techniques for all SEA phases
12. Establishes the need for consultation with competent	Verify if consultation is envisaged with the different authorities with	Sadler (1996); DEAT/CSIR (2000); Fischer (2002a); IAIA (2002);
authorities in the scoping phase?	relevant environmental responsibilities	Baker & McLelland (2003); Abaza et al. (2004); Therivel (2004);
		ODPM (2005); GRDP (2006); Partidário (2007); DGOTDU (2008)
13. Establishes the need for consultation with the general	Verify if consultation with the general public at the scoping phase is	Sadler (1996); DEAT/CSIR (2000); Fischer (2002a); IAIA (2002);
public in the scoping phase?	envisaged	Baker & McLelland (2003); Abaza et al. (2004)
14. Establishes the need for consultation with institutions and	Verify if consultation with institutions or experts for the scoping	Sadler (1996); Baker & McLelland (2003); Abaza et al. (2004)
expert consultants in the scoping phase?	stage is envisaged
15. During the scoping phase were stakeholders consulted?	Verify if the scoping report has evidence that consultation with	Baker & McLelland (2003); Ahmed et al. (2005)
	different stakeholders was conducted before the writing of the report
16. Identifies how the stakeholders' consultation outcomes	Verify if the scoping report explains how the stakeholders'	CEQ (1981); Baker & McLelland (2003); Ahmed et al. (2005)
were introduced in the scoping report?	contributions were integrated into the report.
17. Describes how consultation was conducted?	Verify if the scoping report describes how consultation on the	CEQ (1981); Baker & McLelland (2003)
	scoping phase was conducted
18. Promotes participation in the scoping phase?	Verify if initiatives to promote participation in the scoping phase,	CEQ (1981); Baker & McLelland (2003); OECD (2006); Partidário
	such as workshops, dedicated web page and focus-group meetings	(2007); Retief (2007)
	are envisaged
19. Focus on contributions and concerns of stakeholders?	Verify if the scoping report envisages public participation	CEQ (1981); Baker & McLelland (2003); Retief (2007); Theophilou
	contributions	et al. (2010)
Integration of SEA with the different phases of the PP
20. Defines a methodology/timeline for the all SEA process?	Verify if there is a timetable and a methodology for the SEA that	CEQ (1981); Therivel (2004); OECD (2006); Scottish Executive
	include tasks, goals, timings, stakeholders and responsibilities	(2006); Partidário (2007); DGOTDU (2008)
21. Are the PP and SEA being developed simultaneously?	Verify if there is evidence that the decision-making process and the	Therivel (2004); ODPM (2005); GRDP (2006); Scottish Executive
	SEA are being developed at the same time, linking the conclusions of	(2006); Partidário (2007); Retief (2007); DGOTDU (2008);
	each SEA phase to the PP	Theophilou et al. (2010)

3.2 Application of the framework

For the application of the framework it was necessary to select and evaluate the scoping reports, following the steps described in the next sections.

3.2.1 Selection and characterisation of the scoping reports

Twenty SEA scoping reports were retrieved for Portugal, ensuring a good geographical spread (different NUTS6 II), diversity of technical teams guaranteeing a non-predominance of any team, and the inclusion of different types of plans and the reports' availability at the Portuguese Environmental Agency. The sample was distributed into 16 scoping reports from SEA of SMPs and four scoping reports from SEA of other types of plans (sector, regional spatial planning and special7 plan). The characterisation of the scoping reports analysed is presented in Table 2. The research was applied to scoping reports from 2007 to 2010.

Table 2 Characterisation of the scoping reports analysed.

	PP type addressed	Consultant	NUTS II	Year	Reference
1.	PEOT	erena	PT11 – Norte	2008	ICNB (2008)
2.	PROT	UNL/FCT e UM	PT20 – Região Autónoma dos Açores	2007	DROTH (2007)
3.	PS	UNL/FCT	(National level)	2008	MOPTC (2008)
4.	PS	DHV, Consulmar e BSA	(National level)	2009	IPTM (2009)
5.	PDM	ambep	PT16 – Centro	2008	CM Figueiró dos Vinhos (2008)
6.	PDM	AGRI.PRO Ambiente Consultores	PT 11 – Norte	2008	CM Vinhais (2008)
7.	PDM	DOT e CMF	PT 11 – Norte	2008	CM Felgueiras (2008)
8.	PDM	GIPP	PT16 – Centro	2009	CM Óbidos (2009)
9.	PDM	terra	PT16 – Centro	2009	CM Torres Novas (2009)
10.	PDM	Sinergie	PT16 – Centro	2009	CM Condeixa (2009)
11.	PDM	Lugar do Plano	PT16 – Centro	2009	CM Tondela (2009)
12.	PDM	Hidroprojecto	PT16 – Centro	2009	CM Constância (2009)
13.	PP	Vasco da Cunha	PT16 – Centro	2008	CM Batalha (2008)
14.	PP	nemus	PT17 – Lisboa	2009	CM Lisboa (2009)
15.	PP	Pé Direitos Arquitectos e Lugar do Plano	PT16 – Centro	2009	CM Estarreja (2009)
16.	PP	DCIP	PT17 – Lisboa	2010	CM Lisboa (2010)
17.	PP	PU/CMSCD	PT16 – Centro	2010	CM Santa Comba Dão (2010)
18.	PU	Biodesign	PT17 – Lisboa	2008	CM Loures (2008)
19.	PU	nemus	PT17 – Lisboa	2009	CM Loures (2009)
20.	PU	DP/DOE e CMC	PT16 – Centro	2010	CM Coimbra (2010)

Note: PEOT, special plan; PROT, regional spatial planning plan; PS, sector plan; PDM, municipal master plan; PP, detail plan; PU, urbanisation plan.

3.2.2 Evaluation of the scoping reports

To evaluate the scoping reports through the framework developed (Table 1), content analysis was used as the research method. The content analysis was developed to reduce the subjectivity of the study, since it is a systematic approach used to retrieve relevant information from the analysed documents (Bardin 1977; GAO 1996). Furthermore, it allows for replicable and valid inferences from texts (Krippendorff 2003) and due to its wide scope, different scientific areas may use it (Fischer & Gazzola 2006). Bryman (2012) suggests that the major advantages of this research method are the transparency and flexibility given to the research. However, the method has some limitations, such as the quality dependency of the documents analysed (Bryman 2012), related to their credibility, authenticity, representativeness and availability.

The approach used in this research was adapted from GAO (1996), Roberts (1997) and Neuendorf (2002). The content analysis carried out is qualitative, and the response for each criterion had three possible answers:

• Meets requirements (C): if the response to the question posed by the criterion is fully and unambiguously positive. There is no doubt about the response and the researcher quickly realises that the answer is affirmative.

• Partially meets requirements (Q): if the answer to the question posed by the criterion is not easily understood, or if the researcher is left with doubts about the answer.

• Does not meet requirements (N): if the answer to the question posed by the criterion is totally and unequivocally negative.

The data obtained were then analysed; the absolute frequency for each response in each criterion was calculated. Since the analysis was done to the full scoping report, it was also possible to analyse the consistency of the report. This analysis was performed while the authors read the scoping report and found issues relating to coherence, repetition or confusing information.

4. Findings and discussion

This section is developed in seven parts, of which the first six reflect the broad themes of the scoping effectiveness framework, and the last one discusses the consistency issues found in the reports. A summary of the main results is provided in Table 3. The table shows the evaluation given to each criterion for every scoping report analysed, and the absolute frequency for each criterion in each response.

Table 3 Results of the content analysis carried out on 20 Portuguese SEA scoping reports (2007–2010).

		Scoping report
Criteria		1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	Absolute frequency
Plan or Programme (PP) profile																						C	Q	N
1	Describes the strategic mission and vision?	Q	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	19	1	0
2	Describes the PP goals and objectives?	C	C	C	N	C	C	C	C	C	C	C	C	C	C	C	C	N	C	C	C	18	0	2
3	Describes the PP content?	C	C	C	C	C	C	C	C	C	C	C	C	Q	C	C	Q	Q	C	C	C	17	3	0
Links with other strategic actions
4	Refers the relevant Policies, Plans or Programmes (PPP)?	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	20	0	0
5	Refers the sustainability and strategic objectives of the relevant PPP?	Q	C	C	C	C	C	C	C	Q	C	C	C	C	C	C	C	C	C	C	C	18	2	0
6	Refers how these PPP are linked with the PP under evaluation?	Q	C	C	N	N	C	C	C	C	C	C	C	C	N	C	N	C	N	N	N	12	1	7
Sustainability issues and SEA objectives
7	Identifies the key sustainability and environmental issues relevant for the SEA?	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	20	0	0
8	Defines the SEA objectives?	N	C	C	C	C	C	C	C	N	C	C	C	C	C	C	C	C	C	C	C	18	0	2
9	Identifies the indicators and sources of information for each key sustainability and environmental issue?	N	C	C	C	Q	N	C	N	N	N	N	N	C	C	C	C	C	N	C	Q	10	2	8
Baseline information
10	Identifies the baseline information?	N	Q	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	0	1	19
Communication and public participation
11	Defines the communication strategy for the all SEA process?	Q	C	C	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	2	18	0
12	Establishes the need for consultation with competent authorities in the scoping phase?	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	C	20	0	0
13	Establishes the need for consultation with the general public in the scoping phase?	C	C	N	N	N	C	N	N	N	N	N	C	C	N	N	N	C	N	N	C	7	0	13
14	Establishes the need for consultation with institutions and expert consultants in the scoping phase?	N	C	C	N	C	N	N	N	N	N	N	N	N	N	N	N	N	N	N	C	4	0	16
15	During the scoping phase stakeholders were consulted?	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	0	0	20
16	Identifies how the outcomes were introduced in the scoping report?	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	0	0	20
17	Describes how consultation was conducted?	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	0	0	20
18	Promotes participation in the scoping phase?	N	Q	N	N	N	Q	N	N	N	N	N	N	Q	N	N	N	Q	N	N	Q	0	5	15
19	Focuses on the contributions and concerns of stakeholders?	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	N	0	0	20
Integration of SEA with the different phases of the PP
20	Defines a methodology/timeline for the all SEA process?	Q	Q	Q	Q	N	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	Q	0	19	1
21	Are the PP and SEA being developed simultaneously?	N	Q	Q	N	N	N	Q	N	N	N	N	Q	N	N	N	N	N	N	N	N	0	4	16

Note: C, meet requirements; Q, partially meet requirements; N, does not meet requirements.

4.1 Plan or programme profile

The majority of the reports meet this set of criteria (criterion 1 was met by 95% of the reports, criterion 2 by 90% and criterion 3 by 85%). The issues identified in this set of criteria are mandatory in the Portuguese legislation and in the EU SEA Directive only for the EA/Environmental Report phase, but they are placed in the scoping phase by the Portuguese guidelines (Partidário 2007; DGOTDU 2008), with the exception of alternatives. Identifying alternatives is not explicitly mentioned for this phase in the Portuguese guidelines, nor by the international guidelines considered (GRDP 2006; Scottish Executive 2006).

These issues are commonly a carbon copy of what was previously written in the PP. If the strategic decision-making does not identify alternatives, the scoping report will not present them, and even though several authors (e.g. Therivel 2004) suggest alternatives should also be identified by SEA, in order to achieve ‘more’ sustainable alternatives and to inform the decision-making. In the Portuguese case, specifically, in what is referred to in criterion 3, relating to PP content and alternatives, this does not happen.

4.2 Links with other strategic actions

The first two criteria were largely met, all the scoping reports identified relevant strategic actions (criterion 4) and 90% identified their sustainability and strategic objectives (criterion 5). However, the relationship between the PP under assessment and other strategic actions (criterion 6) was found in only 60% of the scoping reports analysed. The scoping reports meeting criterion 6 systematised the information and it was easily understood how the PP related to other strategic actions. The report partially meeting criterion 6 (scoping report 1) had unsystematic text trying to connect the information making it difficult to read, follow and understand. In the Portuguese guidelines, only DGOTDU (2008) presents a systematised and objective structure to follow, similar to ODPM (2005) and Scottish Executive (2006).

This set of criteria is widely addressed in different literature as a component to be accessed at the scoping phase (ODPM 2005; OECD 2006; Scottish Executive 2006) but, just as the previous section, meeting the criteria is mandatory only for the EA/Environmental Report phase, and is placed in the scoping phase by the Portuguese guidelines.

4.3 Sustainability issues and SEA objectives

All the reports identified the key sustainability and environmental issues (criterion 7), possibly due to the importance given by Partidário (2007) in the Portuguese guidelines. According to Partidário (2007), this set of criteria is what defines the scoping phase and if it is not present constitutes a significant gap. The selection and definition of the relevant sustainability issues and SEA objectives will outline the scope and level of detail of the assessment. It should be noted that the terminology in most of the scoping reports is ‘critical decision factor’, as proposed in those guidelines. This terminology reflects what the author usually calls ‘strategic based model’ (Partidário 2007) or ‘strategic thinking model’ (Partidário 2012). It should be stressed that international guidelines use different terms, such as environmental issues and sustainability aspects (Abaza et al. 2004; Ahmed et al. 2005; ODPM 2005). In order to follow the environmental subjects exemplified in the regulations, the reports analysed present a matrix with the relation between these issues and the selected issues for the SEA being conducted.

As for the SEA objectives (criterion 8), 90% of the reports included them, but only half of the reports presented the indicators and sources of information for each sustainability issue (criterion 9). By not addressing these topics properly, the effectiveness of the reports can be queried. What will a stakeholder analyse if the SEA objectives and indicators which will affect the baseline information collected and analysed, as suggested by Therivel (2004), are not presented in a scoping phase? Also, this author highlights how the lack of a proper set of SEA objectives and respective indicators can lead to biased SEA processes. As stressed by Retief et al. (2008), Söderman and Kallio (2009), SEPA (2011) and EPA (2012), there is a need for focus and systematisation of the relevant issues.

4.4 Baseline information

The criteria concerning the inclusion of baseline information in the scoping phase (criterion 10) are not met. Some authors put these criteria in the scoping phase (e.g. CEQ 1981; ODPM 2005; GRDP 2006; DGOTDU 2008), but others do not (e.g. Partidário 2007). The necessity of including this type of information so early in the SEA process may be debated since it must be further developed in future phases of the process, as discussed by Therivel (2004). Therefore, an intermediate option, where only a very short description and limited set of initial clues on baseline conditions could be provided by the scoping report, may be a more balanced solution.

4.5 Communication and public participation

The communication strategies (criterion 11) are insufficient in 90% of the reports, i.e. lack the identification of stakeholders (beyond public authorities) to be engaged, as well as the consultation techniques to be used during this stage. As for initiatives to promote consultation (criterion 18), only 25% of the reports stated that there was a dedicated website where it was possible to consult and give feedback about scoping. It is highlighted that using only a dedicated website, SEA is excluding the illiterate and those stakeholders without access to the Internet. Moreover, only in 35% of the reports is broad public consultation foreseen (criterion 13), and only 20% of the reports proposed consultation with expert consultants (criterion 14).

Criteria relating to how public participation was carried out (criteria 15 and 17) and how the results were integrated into the process (criterion 16) were not met in any report. There is the possibility that the public participation has occurred but is not reflected in the report; however, these three criteria were based on evidence stated in the report. In addition, there is no evidence on how contributions and concerns from stakeholders were addressed in the report (criterion 19).

The only mandatory criterion, relating to consultation with authorities having environmental responsibilities (criterion 12), was fully met. It was stated in the reports which public authorities should be consulted and when.

Overall, these results may indicate that SEA consultation is carried out only to comply with the regulations, but having limited influence over, or contribution to, strategic action development or downstream actions, as discussed by Noble (2009) for the Canadian practice.

As highlighted in Section 2.1, from the SEPA (2011) and EPA (2012) experience, scoping consultation beyond the public authorities is key for SEA robustness. Efforts should be done to engage public participation, which is essential for the success of SEA and makes the public ‘own’ the decisions, as suggested by IAIA (2002) and Abaza et al. (2004). In addition, detailed information should be given on how the consultation was undertaken and how it is reflected in the report.

4.6 SEA scoping integration with planning

None of the scoping reports identifies the integration moments between the planning process and SEA (criterion 21); however, those reports classified as ‘partially meets requirements’ show an intention to do so in the methodology (criterion 20). This set of criteria is of great importance, as stressed by Aschemann (2004), because it shows evidence of intent for both processes (decision-making and SEA) to be integrated while in development.

In the review assessment conducted by Partidário et al. (2010), for the Portuguese Environmental Reports and statements, it was shown that this integration is not usually explicit, or even if there was any sort of integration with the SEA process at all. Moreover, it was also found that in more than half of the cases, the SEA started after the PP had begun. These findings are mainly in line with the current research outcomes.

4.7 Consistency issues in the Portuguese scoping reports

During the analysis it was found that several scoping reports were difficult to read and to follow due to irrelevant, redundant and dispersed information, which could reflect the youth of the SEA practice and/or the blurred and complex profile of the Portuguese guidelines, framed by national regulations that provide insufficient information. Even though DGOTDU (2008) has guidelines for the presentation of some points of the scoping report, Partidário (2007) does not include a systematic approach to deal with scoping information. In effect, this author often defends a flexible approach to SEA (see, for instance, Partidário 1996, 2000), and a structured frame for presenting a report may not be compliant with this view. The dearth of structure-oriented approaches may lead to a lack of transparency and poor public participation in later stages of the SEA, as argued by several authors. Fischer and Gazzola (2006) state that depending on the political practices and cultural variations, sometimes a rigid approach is best. Furthermore, a rigid approach may be a technique for achieving consistency, and creating routines for developing SEA capacity in practitioners (McLauchlan & João 2012).

Meaningless information and reports that are difficult to read undermine the educative role of SEA and the transparency of the process. SEA should promote new and creative ways of thinking about sustainability in planning systems (Mulvihill & Jacobs 1998; Therivel 2004; McLauchlan & João 2012). This may be happening due to the lack of quality control by national bodies with responsibilities in this area and the lack of political commitment to SEA. As showed by Jackson and Illsley (2007), Kelly et al. (2012) and McLauchlan and João (2012) for the Scottish practice, it is essential for the effectiveness of the scoping, as well as the whole SEA process, that there is commitment and engagement from the Government.

5. Conclusions

This research set out to develop a framework for the evaluation of the procedural effectiveness of the scoping stage in a specific planning and institutional context – Portugal. At the same time, it was expected to help enhance the Portuguese SEA practice, through new insights on scoping weaknesses and strengths.

The 21 criteria developed for the framework covered the entire scoping process, taking into account national and international guidelines and academic literature focused on scoping. It was noted that almost half of the identified criteria relate to communication and public participation, which is considered to be crucial for scoping effectiveness. There are some criteria that should be better explored and understood if they are to be placed in the scoping phase (e.g. criterion 10, identification of baseline information), or if they are applicable to all types of strategic actions (e.g. criterion 14, expert consultation), and thus should be reconsidered in future assessments.

The evaluation carried out for 20 Portuguese scoping reports showed that what is mandatory was done, and features that integrate the Portuguese guidelines were also taken into account. However, what creates proximity between SEA and stakeholders was not met, e.g. topics concerning extended and effective public participation were not achieved in the reports evaluated. From the evaluation review it is perceived that scoping is mainly considered only to comply with regulations, becoming a purely administrative procedure leading to a waste of resources.

The outcomes of this research show the need to evaluate the opportunity to develop more systematic and structured approaches and maintain a focused assessment, with effective participation methods. These evaluations should be used as a fundamental step to rethink current practices and promote the integration of SEA with the planning process at all stages, beginning from the scoping phase. Furthermore, scoping reports need to reflect what was done during the whole phase, especially relating to stakeholder inputs.

The proposed framework constitutes a tool to evaluate the scoping procedural effectiveness and to establish a guidance tool for the scoping phase. This is most important since the SEA methodologies are still being developed, and hence the criteria may be adapted and improved, integrating guidelines for the evaluation of future SEA scoping effectiveness reviews. The adaptation to other planning and institutional systems should be done by integrating context-specific features, weighting tailored versus common criteria.

Further work should be developed to explore the different aspects of SEA scoping effectiveness mechanisms, including the role and type of engagement of different stakeholders in the evaluation, the institutional/governance framework of the review, the type of data collection process (e.g. self-assessment, public authority or independently controlled) and in which stage the effectiveness evaluation should be conducted. Also, future research should focus on recently published scoping reports to compare with this research, to understand what changes, and how, at the same time, they could have a bearing on other types of effectiveness such as substantive and transactive.

Acknowledgements

The authors would like to thank the two anonymous reviewers for the valuable comments and suggestions, which improved the paper. The authors would also like to acknowledge the valuable collaboration of Clara Cintrão from the Portuguese Environmental Agency.

Notes

^1. This law was recently replaced by the new Environmental Basis Law (Law n. 19/2014, of 14 April).

^2. Amended by the Decree-Law n. 58/2011, adding minor changes.

^3. Amended and republished by Decree-Law n. 46/2009, amended by Decree-Law n. 181/2009 and Decree-Law n. 2/2011.

^4. These public authorities include not only environmental authorities but also health authorities, regional authorities and local authorities

^5. As shown by Fischer and Onyango (2012), there are three main journals which publish on SEA: EIA Review, IAPA and JEAPM. These three journals appear at b-on (www.b-on.pt).

^6. Nomenclature of territorial units for statistics (Eurostat 2011).

^7. Special plans (Partidário 2005) or special plans for spatial planning (Cardoso & Breda-Vázquez 2007) or spatial environmental planning (Fidélis & Sumares 2008) are plans that protect natural resources.