Strategic environmental assessment-related research projects and journal articles: an overview of the past 20 years

Abstract

This paper provides an overview of strategic environmental assessment (SEA) related research projects as well as publications in international refereed English language journals of the past 20 years. This is based on a survey with international SEA researchers and the results of a content analysis of 263 SEA papers, published in the three most preeminent international refereed English language journals for SEA: Environmental Impact Assessment Review, Impact Assessment and Project Appraisal and Journal of Environmental Assessment Policy and Management. It is concluded that, while more empirical evidence has been produced than is frequently acknowledged, the main challenge is to raise the level of awareness of what is already out there and to make this the baseline for the future SEA research agenda.

Keywords:

• Strategic Environmental Assessment
• research
• publications
• systematic review

Introduction

While the term ‘strategic environmental assessment’ (SEA) has been in use for the environmental assessment of policies, plans and programmes above the project level since the late 1980s (Lee & Walsh 1992, Fischer 2001), the need for such assessments was stressed much earlier, for example, by Lee and Wood (1978) and O'Riordan and Sewell (1981). Research on SEA has evolved in particular over the past two decades along with a voluminous written output, covering various aspects of theory and practice. Related publications from refereed journals have included, for example, the evaluation of performance and effectiveness (Thérivel & Minas 2002, Fischer 2002a,b, Fischer & Gazzola 2006, Retief 2007a-c, Kis et al. 2010, Eales & Sheate 2011); cumulative impacts (Bonnell & Storey 2000, Thérivel & Ross 2007, Noble 2008, Gunn & Noble 2009, Franks et al. 2010); environmental justice (Connely & Richardson 2005, Jackson & Illsley 2007, McLauchlan & João 2011); integration of assessment aspects and instruments (Stinchcombe & Gibson 2001, Emilsson et al. 2004, Morrison-Saunders & Fischer 2006, Gibson 2006, Bond & Morrison-Saunders 2009); SEA and biodiversity aspects (Balfors et al. 2005, Treweek et al. 2005, Uprety 2005); trans-boundary aspects (Albrecht 2008, Canter & Ross 2010, Marsden 2011); learning through SEA (Fischer et al. 2009, Jha-Thakur et al. 2009); SEA systems (Dusik & Sadler 2004, Fischer 2007); SEA frameworks and contexts (Bao et al. 2009, Fischer 2005); SEA procedures, methods and techniques (Fischer 2007, Sheate & Partidário 2010); SEA integration with decision-making and planning (Fischer 1999, Dalkmann et al. 2004); SEA guidance (Thérivel et al. 2004, Fischer 2006, Croal et al. 2010); SEA theory (Noble 2000, Kørnøv & Thissen 2000, Nitz & Brown 2001, Fischer 2003, Morrison-Saunders & Thérivel 2006, Wallington et al. 2007, Bina 2007); SEA follow-up (Partidário & Arts 2005, Persson & Nilsson 2007, Gachechiladze et al. 2009); indicators (Donnelly et al. 2006, 2008); climate change and SEA (Larsen & Kørnøv 2009, García-Montero et al. 2010, Slootweg & Jones 2011, Fischer et al. 2011, Lyhne 2011); health in SEA (Kørnøv 2009, Fischer et al. 2010); democratization through SEA (Bonifazi et al. 2001); different perceptions on SEA (Fischer & Xu 2009). Despite the range of publications produced, suggestions have been made that empirical evidence has remained scarce (see e.g. Emmelin 2006, Bina 2007). However, in the absence of any comprehensive and systematic account of research activities and outputs to date, such claims have remained largely unchallenged and unanswered.

With this paper, we aim to contribute to the advancement of knowledge on the comprehensiveness of SEA research and outputs by presenting the results of (a) an international questionnaire survey of SEA researchers and (b) an analysis of journal articles on SEA from the most preeminent refereed English language SEA journals. In this context, experts' opinions on past research, trends and suggestions on future priorities are established and the coverage of SEA in the literature is evaluated.

The original reason for conducting the research underlying this paper was to generate evidence for the second special International Association for Impact Assessment (IAIA) meeting on SEA in Prague: ‘SEA Implementation and Practice: Making a Difference?’ in September 2011. While a session on ‘Emerging priorities for the evolution of SEA and future research on the field’ (conference programme; see http://www.iaia.org/specialmeetings/prague11/) and an associated call for papers had been announced early on, the authors of this paper felt that, in the absence of a clear baseline, the identification of research needs would be rather problematic.

This paper summarizes the survey results presented at the IAIA Prague meeting. Furthermore, the results of the literature review, also presented at the same meeting, are described. Both survey and literature review results were subsequently updated and now reflect information on SEA research projects and SEA papers up to the end of 2011.

SEA experts to be contacted for the survey were identified, based on (a) an initial SEA document search and (b) information found on the website of the IAIA, in particular IAIA Connect. A broad understanding of SEA was used, covering various forms, including, for example, European plan and programme environmental assessment (following European Directive 2001/42/EC), policy assessment (see e.g. Chen et al. 2011), sustainability appraisal (as used in the UK; see e.g. Thérivel & Walsh 2006), ‘EIA for large projects’ (as conducted, for example, in the Netherlands; see e.g. Fischer 2006), Programmatic EA (as used in the United States; see e.g. Ma et al. 2009) and others.

Methodology and analytical framework

Questionnaire survey

Based on a review of (a) English-language1 SEA-related publications and (b) publicly available information provided by the IAIA (via its web site www.iaia.org), over 2000 potential SEA researchers from over 50 (English and non-English speaking) countries were contacted by email and asked to contribute to a questionnaire survey. This was administered online. The survey consisted of a list of standardized, as well as some open-ended, questions and was subdivided into the following five areas:

•	respondent's background, including institutional affiliation, education, country of professional activity, and experience in SEA research;
•	research sector, geographical focus, research approach, theories and methods;
•	research environment, including drivers for research, availability of reference materials, outputs, costs, data sources, cooperation and support;
•	weaknesses/challenges and strengths/opportunities defined within SEA research, including questions on funding, dissemination, application and utilization of findings;
•	future research outlook, including priority research areas, knowledge gaps and unresolved research issues.

The survey was open from 19 July 2011 to 29 November 2011, and resulted in 50 respondents completing the questionnaire. Two respondents had to be excluded, as they had not actually done any SEA research, but were expressing opinions only. As the authors of this paper initially considered 48 to be a small number, other SEA actors identified through the initial literature review were also contacted and asked to contribute to the survey. However, no further research projects were identified. It is acknowledged here that there is likely to be a large number of additional PhD and master-level university degree-related SEA research projects. However, while most PhD projects will often result in at least one publication in the professional literature, not many master-level dissertations find their way into publications in international refereed journals.

Content analysis

Content analysis of English language SEA papers was conducted, covering the following aspects:

•	publication details, including year of publication, names and number of authors, country and institutional affiliation of authors;
•	sector of application, including spatial/land use, industry, trade, transport, energy, waste and others;
•	main topic or focus of the paper, including process, methods and techniques, benefits of SEA application, comparative analyses, concepts, paradigms, perspectives, SEA frameworks, aspects of integration and others;
•	research approaches and method(s) used, including case study research, literature/document reviews, discourse analyses, experiments, surveys/interviews/questionnaires and others;
•	geographical focus and level of socio-economic development, including continent and country as well as development status.

Relevant publications were identified, using keyword searches, based on the three most widely used multidisciplinary citation databases; ‘Google Scholar’, Thomson Reuter's ‘Web of Knowledge’ and Elsevier's ‘Scopus’. ‘Strategic environmental assessment’ was used as a search term, resulting in 13,200 hits on Google Scholar. However, many of these were found to relate not to refereed journal articles, but rather to SEA reports/statements, books, book chapters and other reports. On the ‘Web of Knowledge’, 324 and on Scopus, 479 refereed papers up until the end of 2011 were identified. It was subsequently decided to base the literature review on the Scopus database results, as (a) this was focusing exclusively on the academic SEA literature (i.e. opposite to Google Scholar), and (b) it was less strict in terms of journal selection than the ‘Web of Knowledge’, which, for example, did not list IAIA's journal Impact Assessment and Project Appraisal. Furthermore, (c) the Scopus citation index is now widely used in academia.2

Reading the abstracts of the 479 SEA papers thus identified, it became evident that a keyword search approach on its own was not likely to result in a reliable overview, as some of the papers identified did not actually cover SEA, but were focusing on other (albeit often related) issues, including, for example, project EIA, life-cycle assessment, cost–benefit analysis, and spatial and sectoral planning. Finally, some publications the authors of this paper were aware of being SEA-related were not identified through the keyword search method. For this reason, it was decided to read all abstracts and, if this was inconclusive, additional parts of all papers published between 1990 and the end of 2011 of the three journals that were identified as, together, being the preeminent platform for SEA publications. These comprise Environmental Impact Assessment Review (EIA Review), Impact Assessment and Project Appraisal (IAPA) and Journal of Environmental Assessment Policy and Management (JEAPM). In total, 191 of the 479 papers (40%) were from these journals, 86 or 18% from EIA Review, 60 or 12% from IAPA and 45 or 10% from JEAPM. Considering the time and resources available, it was not possible to systematically go through the papers of all 40 journals that were identified to have occasionally included SEA papers.

Other journals with more than 10 papers included European Environment, with 16 papers (of which 10, however, were published in two special issues in 2004 – 14:2 and 14:3 – on ‘Progress towards meeting the requirements of the European SEA Directive’) and the Journal of Environmental Management, with 13 papers. Other English language journals with between five and 10 SEA papers included Environment and Management, Journal of Planning and Environmental Law, Journal of Environmental Planning and Management and Chinese Geographical Science.

Going systematically through all paper abstracts of EIA Review, IAPA and JEAPM resulted in 263 SEA related papers (i.e. 72 more than were identified based on the key word search) being included for further analysis. All of these are listed either in the references at the end of this paper, or the supplementary bibliography which is hosted as supplementary material on the article's webpage: http://dx.doi.org/10.1080/14615517.2012.740953.

Data analysis

Data analysis was based on the categories introduced in the previous two sections. Qualitative data were generated, using content analysis, while data were quantitatively analysed, using SPSS (Statistical Package for Social Sciences, a widely used statistical analysis computer programme). This involved calculating frequency distributions and statistical associations. Data were not normally distributed. Therefore, nonparametric tests were used. These included chi-square tests for assessing whether there were any statistically significant relationships (in terms of, for example, differences and similarities). Cross-tabulations were also used in order to establish relationships between variables. The results are subsequently presented, using a range of tables and graphs.

Results

Results are presented here in two main sections. The first reports on the findings of the questionnaire survey. The second then presents the results from the content analysis of the peer-reviewed SEA publications.

Questionnaire survey

Respondents from universities and international consultancies accounted for 73% of the sample. The most frequent educational background held by the researchers was natural sciences (50%), followed by engineering/other technical subjects (28%) and social sciences (22%). Eighteen countries were represented, with Canada and the UK each providing 28% of the survey participants (Figure 1). While one reason for this is undoubtedly language (the survey was conducted in English), another is that much of SEA research globally is actually happening in research institutions in these countries. Regarding academic research, these include in Canada the universities of Saskatchewan (Department of Geography and Planning), Waterloo (Department of Environment and Resource Studies) and McGill (UNEP Collaborating Centre on Environmental Assessment). In the UK, they include the universities of Liverpool (Department of Geography and Planning), Manchester (mostly research from the former EIA Centre), Oxford Brookes (Department of Planning), Imperial College (Centre for Environmental Policy) and University of East Anglia (School of Environmental Sciences). Regarding research driven by major international consultancies, they include, for example, Levett–Thérivel, Collingwood Environmental Planning, URS (including the former Scott Wilson), TRL and others. In terms of experience, 50% of the researchers (i.e. 23) had completed between one and three SEA research projects and 22% (i.e. 10) had completed 10 or more. Those who had been involved in between four and nine research projects made up just above a quarter of the sample (28% or 13).

Figure 1 Geographical representation of 48 SEA researchers.

Out of the 53 occupational roles the 48 respondents said they fulfilled (some have had more than one role in their careers), 45% were researchers, 45% consultants and 10% government officials. About half of the researchers dedicated between 1 and 25% of their research time; about a quarter between 26 and 50%; one-tenth between 51 and 75%; and about one-seventh between 76 and 100% of their research time to SEA research.

SEA research – sectors and costs

A total of 69 SEA related research projects were identified. 20% of these were said to be spatial/land use/town and regional planning related. Transport planning related research made up about 14% and both, energy and water related research together 13%; 10% was said to be non-sector related, and the remaining 43% were being conducted largely in natural resources management-related fields, including fishery, forestry, agriculture and biodiversity. While ‘industry’ made up about 5%, climate change and indigenous people (‘indigenous knowledge’) related research made up only 1.6 and 2.4% of all activities, respectively. On average, each researcher was found to cover at least three research areas (Figure 2).

Figure 2 Research distribution according to policy sectors (38 researchers giving 123 answers).

In terms of the costs of the research projects, only five out of 48 researchers provided data, covering a total of 22 projects. These included two UK researchers, representing 17 projects, one South Korean researcher, representing four projects, and one Italian researcher representing one project (see Figure 3). The largest research project of those had a budget of £300,000 and the smallest of about £3000. While the average cost was £60,1163, most projects tended to be fairly small scale. For 12 projects, information on the duration of the research projects was obtained. Of these, seven lasted a year; one two years; one four years and one eight years. Thirty-seven researchers indicated that the single largest proportion of costs in the overall research budgets was personnel.

Figure 3 Research costs per project.

In terms of geographical coverage, most projects focused on Europe, followed by North America and Asia (Figure 4). From 29 research projects undertaken in 11 European countries, 12 focused on the UK, five on the Netherlands and four on Italy. Finally, one each focused on Hungary, Denmark, Ireland, Portugal, Poland, Estonia, Greece and Germany. Since not all respondents indicated the exact number of research projects they had been involved in, this distribution is considered indicative only. Twelve projects focused on North America, 10 on Asia, four on each, South and Central America, five on Africa, three on the Middle East and two on Australia.

Figure 4 Geographical areas covered by researchers.

In terms of the topics covered, most research projects focused on SEA frameworks and the application of SEA within different contexts. Others included SEA methods/techniques, SEA procedures, SEA effectiveness and evaluation as well as theoretical and conceptual issues. Overall, research interests largely reflected a concern with how SEA is applied and how it works. When asked what theories were underlying their research, 31 researchers indicated that more than half of the projects were using rational theory related approaches. A smaller proportion used what they called ‘post-modernist’ approaches (just over a third). In terms of methods used in this context, frequently these included, for example, discourse analysis. Finally, a couple also said they applied ‘other’ approaches. Research tended to be more qualitative than quantitative, with relatively low applications of causal, and hypothetic–deductive approaches to research design being used.

The largest drivers for SEA research were found to be of an external nature, including in particular, government initiatives and research calls. A few were academic degree-related (in particular PhDs and some research master level dissertations4). Some were also driven by business/monetary interests, including in particular consultancy projects. About one-quarter of the projects were problem-led (Figure 5).

Figure 5 Key drivers for SEA research (37 researchers, 82 responses).

Regarding the methods used, qualitative approaches were the most commonly used, based on, for example, the review of case studies and interviews. Quantitative approaches, including statistical analyses and GIS were rarely applied. Various topics were covered, including in particular SEA frameworks/systems as well as regulatory processes. Finally, some studies focused on the evaluation and review of performance and effectiveness of SEA (Figure 6). Overall, the research agenda to date has clearly been biased towards applied research.

Figure 6 Aspects of SEA research (based on 37 researchers, who were covering 143 research topics).

SEA research outputs mainly took the form of academic publications, disseminated through a wide range of outlets, in particular peer-reviewed journals, books and book chapters. Other means of dissemination have included conference presentations (including posters) and project websites. Academic qualifications/degrees were other important results of SEA research.

Challenges, weaknesses and strengths of SEA research

Some challenges and weaknesses of SEA research to date were identified by the researchers taking part in the survey. About two-thirds of them indicated that funding for SEA research was hard to get and collecting relevant data was often difficult. Over half of them also indicated that dissemination of research findings to the wider SEA community was not effective enough and needed to improve. Nearly half of the respondents suggested that ‘application and use of SEA research results was good and encouraging’. A majority of respondents had no idea as to how other research communities perceived SEA research results.

In terms of strengths, it was suggested that, generally speaking, the SEA research community was acting in a collaborative way and nearly four-fifths of the respondents agreed that support from fellow SEA researchers was readily available. SEA research is receptive to ideas outside the immediate SEA community. In this context, more than four-fifths of the researchers were of the opinion that the SEA community was not insular and did not suffer from ‘silo’ mentality. While about half of the SEA respondents did not think that SEA concepts were hard to establish and apply, one in four disagreed with this statement.

Priority areas for future SEA research

When asked to indicate what specific areas SEA research should prioritize over the next 10 years, over 75% indicated that there was a need for more evaluation/effectiveness studies as well as for more research in certain policy sectors. Furthermore, over 70% indicated that more case study research on SEA effectiveness was needed. Nearly 70% of the respondents argued in favour of more inter- and trans-disciplinary research, more research on techniques and methods, as well as a more effective dissemination of research findings (Table 1).

Table 1 ‘Net balance of opinions’ on research needs and research environment.

Issue	Net balance of opinions	Consensus (yes/no)	More than 20% undecided (yes/no)
More research needed on SEA theory	28.1%	Yes	Yes
More research needed on techniques and methods	69.8%	Yes	No
More inter- and trans-disciplinary research needed	68.8%	Yes	Yes
More effective dissemination of findings needed	68.7%	Yes	No
More research on certain policy sectors needed	75.1%	Yes	No
More case study research on SEA effectiveness needed	71.8%	Yes	No
More research into SEA evaluation/effectiveness needed	81.3%	Yes	No
Need for increased research in SEA costs vs benefits	56.3%	Yes	No
More research in SEA outcomes	87.5%	Yes	No
Collecting direct and relevant data often very difficult	− 9.7	No	Yes
SEA community insular and suffers ‘silo’ mentality	− 16.1	No	Yes
Dissemination of findings poor and needs improvement	16.1	No	Yes
Bold indicates that over 75% of the respondents agreed; italics under 50%. ‘Net balance of opinions’ is a reflection of the dominant opinion and is arrived at by subtracting the ‘against’ votes from the ‘for’ votes.

When asked to make further suggestions on what specific areas SEA research should focus on over the next 10 years, a list of 81 items was produced from 31 responses. A majority of the suggestions were categorized as revolving around issues of SEA effectiveness and the use of SEA in certain areas, for example, climate change, renewable energy and trade. Furthermore, more research into SEA costs and benefits was seen as being of importance for various applications and levels of PPPs, especially policies and legislation. There were also suggestions for more research into SEA application in developing countries. This included in particular water management-related research in arid areas, as well as climate change, agriculture, ecosystem services and energy management-related research. Research priorities were also directed at specific application scales, including neighbourhood and trans-boundary scales.

Content analysis

Content analysis of 263 peer-reviewed papers that were published between January 1992 and December 2011, was conducted. These papers represent 626 authors that were associated with 439 institutions across 38 countries, representing all continents. They are listed in the references at the end of this paper, or the supplementary bibliography which is hosted as supplementary material on the article's webpage: http://dx.doi.org/10.1080/14615517.2012.740953. Three peaks in annual frequencies of publications were found (Figure 7). The first one was in 2000 and was mainly triggered by authors from Europe. This is most likely connected with work revolving around the then forthcoming SEA Directive. A second peak was established, lasting from 2004 to 2007. This is interpreted as a follow-up to the SEA Directive taking effect. Finally, a third peak has just occured in 2011. This comes with increased activities around Sino-Asian related papers, as well as a renewed interest into issues of SEA effectiveness and outcomes internationally.

Figure 7 Number of papers published by year.

Nearly half of all publications were published in EIA Review (124 or 48%), and about a quarter of the remaining each in IAPA (71 or 27%) and JEAPM (66 or 25%) (Figure 8). Chi-square test results reveal that there is a significant statistical difference in the distribution of the papers in the three journals over time, that is, the distribution is not as would be expected by chance (see Figure 9). One of the reasons for this are the various SEA-related special issues and changing editorships.

Figure 8 Distribution of SEA papers by the three journals covered in this paper.

Figure 9 Distribution of papers by year in the three journals.

The three journals have different histories, publication frequencies and paper volumes. EIA Review was founded in 1980 with Lawrence Susskind as the first editor. While initially there were four issues and up to 30 papers published each year, this had increased to up to six issues and over 50 papers per year in 2011. The current editor is Eric Johnson. IAPA is the official publication of IAIA and is published four times a year with up to about 30 papers. It was established in 1998, as a result of a ‘merger’ of two existing journals: Project Appraisal (1900–1997) and Impact Assessment (1982–1997). It was originally edited by Chris Wood and Carys Jones. Editorship is now with Angus Morrison-Saunders and Francois Retief. JEAPM was founded in 1999 by Bill Sheate. There are four issues a year with up to 30 papers. Current editor of the journal is Thomas Fischer.

Sixty-eight percent of the papers had multiple authors (Figure 10). Furthermore, 40% of the papers were written by authors representing several institutions. About a quarter of the papers had three or more authors and just less than a third had one author only.

Figure 10 Percentage of papers being the result of author and institutional collaboration.

Figure 11 shows that about half of the SEA papers did not focus on any specific sector. Most of those that did focused on spatial/land use, development planning and transport/other infrastructure-related planning. Other areas included natural resource management and biodiversity, trade, social, health, culture and waste management.

Figure 11 Distribution of sectors covered in the papers.

Regarding the research approaches and methods used, the four most common were case study research, discourse analysis, document review and survey method (Figure 12). The least common methods were ‘experiment’ and ‘comparative analyses’. The three most widely covered topics included SEA application and practice (45%), SEA reviews/evaluation/analysis (31%), and theories and concepts within SEA (21%). In terms of the geographical focus, 37% of the papers included no country reference. Nearly 50% of the papers covered European SEA practice with papers focusing on the UK making up 10% of the whole set. Chinese practice was reflected in 8% and Canadian practice in 7% of all papers. Those publications which were judged to be ‘generic’ were largely written by European authors, usually referring to European issues and perspectives.

Figure 12 Research methods used.

Most papers were written by authors from developed countries (Figure 13). While most publications from China and other Asian countries have been prepared over the past decade (and here mostly over the past two to three years), about 90% of these publications have been in collaboration with authors from developed countries.

Figure 13 Publications in terms of socio-economic level of development of the countryies the authors are based in. ‘Other’ refers to the group of papers having several authors from more than one level of socio-economic development.

Topics covered have changed over time. When looking at 5 year intervals (Table 2), two main sets of topics emerged, including (a) SEA application, implementation and contexts, and (b) SEA effectiveness and outcomes. This goes hand in hand with issues such as SEA procedures, systems and contexts.

Table 2 Five-year intervals showing most frequently covered topics (ranked 1–3).

	1992 – 1996	1997 – 2001	2002 – 2006	2007-2011
Most covered topics	1. SEA quality reviews	1. SEA systems	1. SEA system	1. SEA quality reviews and SEA effectiveness
	2. SEA procedures	2. Environmental integration/sustainability	2. Environmental integration/sustainability	2. SEA procedures
	3. SEA systems; benefits of SEA	3. SEA procedures	3. SEA practice, application and review; SEA effectiveness	3. SEA practice and application; SEA systems

Overall, and based on a further amalgamation of the main topics covered/the main emphasis of the papers, it is found that there is a bias towards more descriptive and practice evaluation papers. Theoretical and conceptual papers make up < 5% of all papers (Figure 14).

Figure 14 Main topics covered in papers (i.e. the main emphasis).

Discussion and conclusions

Two main questions are addressed here: (a) where we are with respect to SEA research, and (b) what we want from future SEA research. Furthermore, (c) reflections on the strengths and weaknesses of current SEA research are provided and (d) concluding remarks are made.

Where are we with respect to SEA research?

Overall, 69 SEA-related research projects were identified and evaluated in this paper, based on survey results with international SEA researchers. Furthermore, results from the literature review suggest that there are likely to be over 500 English language publications in refereed journals focussing on SEA. Some 263 of those, published in EIA Review, IAPA and JEAPM, were evaluated. Related publication activities in refereed journals have increased substantially over the past 20 years. While early publications focused on the reporting of pilot studies and comparative analyses, particularly in spatial/land use and infrastructure planning, more recently a much wider range of topics has been covered, including aspects of, for example, learning and long-term behavioural changes. However, to date certain sectors, including waste and tourism, and topics such as health and social aspects, have not yet received much attention. While some SEA publications are the outcome of funded research, generally speaking accessing funding for SEA type research appears problematic and much research currently revolves around PhD and master-level degree dissertations. Furthermore, refereed journal articles often refer to practical experiences, rather than empirical research results. In terms of the research approaches applied, there is clearly a preference for case-study-type research. As a consequence, there is a significant amount of ‘descriptive’ and ‘documentary’ type of research happening, illustrating SEA applications in various contexts. While this can provide for some benchmarks, generalizing associated results may be problematic.

Most of the funded SEA research has been driven by government initiatives and some research calls. Furthermore, the current research agenda appears to be heavily focused on (certain) developed countries. While numerous publications from China in particular have emerged over the past decade, other developing countries, as well as some developed countries, are still under-represented. Overall, and considering the amount of SEA practice that is out there, with 1000s of SEAs being conducted every year globally (Fischer 2010), the number of research projects identified in the survey underlying this paper is remarkably low.

What do we want from future SEA research?

Overall, while the survey and literature review results allow us to draw some conclusions regarding future research needs, there clearly is no overall consensus among the research community on what should take priority. Many of the researchers taking part in the survey underlying this paper suggested that more research on SEA effectiveness was needed, in particular with regards to outcomes and value added to PPP decision-making. Furthermore, more research on certain policy sectors was said to be of importance. Respondents also underlined the need to do more research on the costs and benefits of SEA. Finally, what capacity development should consist of was said to require some closer attention.

There clearly is a question mark over some of the suggestions made in the survey. Considering the plethora of case studies already available, one is left wondering why some respondents were asking for even more case studies to be published. Here, it appears that the existing literature is not known well enough, even among researchers. In this context, it is of interest to note that only half of the researchers surveyed said that the consideration and uptake of the results of empirical SEA research was actually good.

Strengths and weaknesses of current SEA research

Based on what has been said above, there are several weaknesses and strengths of current SEA research. An important weakness is clearly the unsatisfactory uptake of the results of empirical research by other researchers and those writing on SEA. Another weakness is the apparent lack of funding for SEA research. While there clearly is a geographical bias of SEA research activities, this does not necessarily indicate a weakness, as this may simply reflect where most of the SEA practice is currently taking place.

An apparent strength in current research is the level of collaboration amongst authors and institutions. A further strength is clearly the openness of SEA researchers regarding cross-disciplinary research. Finally, a multitude of practical and theoretical contributions have been made, which together provide for a comprehensive SEA research baseline.

Concluding reflections

While our paper has shown a number of trends in SEA research over the past 20 years, we believe that the SEA research agenda needs to catch up with the existing baseline and knowledge. Currently, even some researchers appear to have a poor understanding of what is already ‘out there’. In this context, political attempts to steer the research agenda may be counterproductive in the absence of a clear understanding of existing knowledge. This may in fact lead to the reinvention of already known issues and concepts. Our findings indicate that many of the areas researchers are asking for more research to be conducted are actually covered by the existing literature quite well. In this context, a compendium of key findings would be useful. This paper provides for a first step by reflecting on research activities and English language journal articles published over the past 20 years.

Notes

^1. The authors are aware that there are numerous non-English language SEA literatures. However, the purpose of this paper is to reflect on the international SEA literature in the academic lingua franca, that is, English.

^2. In Australia, it is applied as the sole citation provider for the Excellence in Research for Australia assessment. In the UK, it is used by the higher education funding bodies as the data provider for the 2014 Research Excellence Framework.

^3. Currency conversions were based on the average annual rates found on the website http://www.x-rates.com as of February 2012.

^4. The number of SEA-related PhD dissertations is difficult to estimate. Regarding the number of master-level dissertations prepared every year, conversations of the authors with university staff from various countries indicate that, globally, these may be several hundreds.