Literature syntheses to inform marine ecosystem management: lessons learned from stakeholder participation

ABSTRACT

Systematic literature syntheses are a key element in the scientific realm, considering the steadily growing amount of available knowledge. Involving stakeholders in the research process brings a wide range of advantages, like broadening the perspectives on the problem in question, increasing the relevance of results for policy- and decision-making, the public and other end-users and thus enhancing the impact and acceptance of research. While participatory approaches are on the rise, reflections on stakeholder involvement in systematic syntheses on environmental management are scarce. We reflect on the process of involving stakeholders with expertise also from outside academia during three literature syntheses with different foci of marine and coastal ecosystem services in the Baltic Sea. Our analysis is based on notes, e-mails, minutes and recordings of internal project meetings, interviews and workshops involving both researchers and stakeholders. We discuss the challenges the participatory approach introduced and develop lessons learned to support the planning of stakeholder engagement for future literature syntheses. We conclude that stakeholder identification, communication, collaboration and knowledge translation are highly time- and resource-intensive processes. Furthermore, appropriate training and experience are necessary for the design, execution and evaluation of participatory methods tailored to each project stage. Therefore, we underline the importance of adequate consideration of the required resources during project planning and implementation. To encourage and support valuable stakeholder engagement and knowledge exchange between the research community and actors of policy and practice, more appreciation of such efforts by funding institutions and within the wider scientific community is needed.

EDITED BY:

• Sebastian Villasante

KEYWORDS:

• Evidence synthesis
• systematic review
• systematic map
• stakeholder participation
• science-policy interaction
• policymakers
• HELCOM region

1 Introduction

Sustainable environmental management, policy and decision-making depend on the integrated consideration of healthy ecosystems and human wellbeing. The European Union’s Biodiversity Strategy for 2030 (European Commission 2020) states that biodiversity considerations need to be better integrated into public and business decision-making at all levels. It also states that the criteria and standards relating to ecosystems, the services they provide and their sustainable use will be developed based on existing studies and data. Thus, identifying the relevant scientific knowledge, e.g. in relation to biodiversity and ecosystem services, is key to ensuring policy- and decision-making considers environmental and social concerns alike. Before the broader application of literature reviews and the growing call for evidence-based decision-making, practitioners were not necessarily making use of the steadily growing volume of information available and consequently the impact of research on practice has often been limited (Pullin et al. 2004; Thomas-Walters et al. 2021).

In 2017, the joint Baltic Sea research and development programme BONUS (2007–2020) launched a call to address these issues. The request called for a review and synthesis of research outputs to identify knowledge gaps and future research needs for the sustainable management of the Baltic Sea ecosystems (see Figure 1 for geographical location) (BONUS 2017). The research call aimed to support evidence-based decision-making and the application of available scientific knowledge. Aiming for policy-relevant results, the BONUS ROSEMARIE project (Project title: Blue health and wealth from the Baltic Sea – a participatory systematic review for smart decisions) carried out three participatory systematic syntheses: i) of the marine and coastal ecosystem services (ES) research in the Baltic Sea, ii) of ES and their contribution to human health and wellbeing, and iii) of the methods and models used to estimate monetary and non-monetary values of Baltic Sea ES.

Figure 1. Map of the Baltic Sea and its littoral countries.

The Collaboration for Environmental Evidence (CEE) is an open community of scientists and managers created to promote the use of systematic syntheses to inform environmental policy and practice. They developed methods in 2018 to systematically synthesise environmental research literature (CEE 2018). The CEE distinguishes two types of systematic evidence synthesis: systematic maps and systematic reviews. Systematic maps aim to describe the evidence base of a specific topic in terms of distribution and abundance, while systematic reviews aim to synthesise the outcomes of studies, compiling the impact effects of specific measures (James et al. 2016; CEE 2018).

We apply the term ‘syntheses’ as an umbrella term for both systematic maps and reviews. Systematic literature syntheses are based on pre-defined methods and criteria to reduce hidden bias, increase transparency and enable updating as and when new research becomes available (CEE 2018). According to Haddaway (2017, p. 1), systematic evidence syntheses are the ‘gold standard’ of synthesis methods and ‘should be transparent, comprehensive, repeatable and user-friendly’. A wide range of guidance for syntheses have been published for application in environmental sciences. These cover the whole synthesis process (e.g. Pullin and Stewart 2006; James et al. 2016; Haddaway and Crowe 2018a) or specific aspects like search sources (e.g. (Livoreil et al. 2017; Gusenbauer and Haddaway 2020) or software opportunities (Thomas et al. 2017; Westgate et al. 2018).

The potential of end-user involvement has been at the heart of initiatives promoting systematic approaches in reviews since their inception (e.g. Bastian 1994). Engaging policymakers and other relevant stakeholders in the synthesis process aims to ensure policy-relevant development, and fosters the efficient and timely communication of the results (Gough et al. 2013). Apart from increasing the relevance and impact of the synthesis, involving stakeholders can eventually enhance trust in the results and facilitate science-policy communication (Haddaway and Crowe 2018b). Engaging stakeholders in evidence syntheses originates in the medical and health policy field (Harris et al. 2016) and has since been adopted in environmental research (Pullin and Knight 2001; Sutherland et al. 2004). However, methodological guidance and frameworks for broader stakeholder involvement in literature syntheses more generally, have only fairly recently been developed (Pollock et al. 2019), especially in environmental management (Haddaway et al. 2017; Land et al. 2017). Although stakeholder involvement in environmental management research is well-established (Reed 2008; Reed et al. 2018), and there are expectations that stakeholders should be integrated into the synthesis process (CEE 2018), there is still little reflection on the experience available (e.g. Collins et al. 2019). Additionally, the stakeholder engagement description in systematic reviews is often insufficient to guide or support future research planning (Pollock et al. 2018).

This paper takes one step in closing this gap by reflecting on the stakeholder interaction process in three participatory systematic syntheses. We present the participatory procedure and decisions taken during the syntheses processes to provide guidance for stakeholder involvement planning in future evidence syntheses. The paper is framed around the following two key research questions:

• What challenges were encountered during the syntheses processes concerning the participatory approach?

• What are the lessons learned that might foster successful stakeholder involvement in systematic literature syntheses?

This study concentrates on the processes and outcomes of the participatory activities, rather than the actual systematic literature syntheses outputs, which have been reported separately (Håkansson et al. 2020; Storie et al. 2020, 2021; Kuhn et al. 2021). We draw on the combined findings of these three syntheses, deliberating on the stakeholder involvement and concentrating on the processes and outcomes of said participatory approach. Oliver et al. (2015) provide a framework to evaluate and structure stakeholder involvement in research based on three interrelated dimensions. We adapted their framework (Figure 2), basing the structure of this paper upon it. In Section 2, we outline the stakeholder involvement approach, including the drivers of involvement and the process of involvement. Section 3 presents the impact of involvement on the literature syntheses. Sections 4 and 5 present reflections upon the approach used and discuss the challenges encountered, lessons learned and drafts implications for future research.

Figure 2. Framework for stakeholder involvement in research: applied to study the involvement activities in three systematic syntheses of ES in marine management (adapted from Oliver et al. 2015).

Our analysis is based on material from the BONUS ROSEMARIE project about the preparation, execution, and evaluation of the different stakeholder events, in addition to team discussions that drew out the tacit knowledge acquired by the multi-disciplinary project team. The material consists of notes taken during internal project meetings, e-mails and invitations used to contact the policy advisors, audio recordings and interview transcriptions and written notes taken by researchers and stakeholders during workshops and meetings. While the objective to reflect explicitly on the stakeholder involvement was developed during the project, this analysis was carried out following the project’s completion.

2 The participatory approach

An overview of the BONUS ROSEMARIE project’s participatory approach, including drivers, processes and stakeholder involvement impact, is presented in Figure 2. The diagram shows drivers of involvement on the left and impacts on the right. The drivers are split in two, with the top row describing the research team characteristics, and the bottom row those of the stakeholders. Influenced by these drivers and running from left to right through the diagram’s centre, are a set of three involvement processes (bringing the people together, supporting the debate and decisions, and how people interact and how their ideas are used), which in turn influence different kinds of involvement impacts. The content illustrated in Figure 2 is described in more detail throughout Sections 2 and 3.

2.1 Drivers of involvement

Our aim was to identify the policy advisors’ knowledge needs, so as to produce tailored, transparent, and therefore more trustworthy syntheses of scientific knowledge relevant for these practitioners’ specific stakes. The research team included scientists from a variety of countries, career stages and disciplines, such as geography, environmental social science, marine management, environmental economics, sustainable development, data management, pharmacology, and chemistry. Furthermore, the project’s advisory board included experts on systematic syntheses methodologies and participatory methods. The research team had diverse experience with the participatory approach in environmental sciences but had not applied a transdisciplinary approach to systematic syntheses before.

We define stakeholders as people or organisations that hold a self-interest in results of these syntheses, either because they might apply the findings for decision-making, or they might be affected by decisions influenced by the outcomes of the research (Rees and Oliver 2017). Stakeholders are therefore potential ‘end-users’ that include researchers, research funders, subject experts, decision-advisers, -enforcers and -makers from governmental and non-governmental organisations, as well as practitioners, business people and/or citizens (Haddaway et al. 2017).

Bearing this in mind, members of the HELCOM¹ (Baltic Marine Environment Protection Commission; Helsinki Commission) GEAR (Group on the Implementation of the Ecosystem Approach) group were identified as key stakeholders. They have a high interest in healthy marine ecosystems and in reaching the goal to attain a good environmental status of the Baltic Sea ecosystems, while having the power to shape the updates of the relevant marine policies.

2.2 Process of involvement

The stakeholder identification and invitation used purposive selection from groups of stakeholders already known to the researchers. Familiar groups of stakeholders can be more responsive and easier to engage, and with a purposive approach, target numbers can be more easily met. Both considerations were important, given our need to use time and resources efficiently (Haddaway et al. 2017). A purposive approach meant that different stakeholders were recruited according to their different areas of expertise so as to support the three distinct syntheses (Hanlon 1998). Different stakeholder sets were involved at different project stages.

The communication with the policymakers was mostly undertaken by the project leader and the environmental social scientists. In general, all meetings and workshops were facilitated by the project researchers and started with short presentations of the methodology and the current state of the syntheses and their interim developments. This was followed by a time to reflect and provide feedback on the results. The approach of the participatory systematic syntheses was emphasised throughout, to underline the opportunity for actively influencing the research scheme as well as to disseminate the syntheses methods and its aims. Apart from the methodological approach, the focus of the stakeholder events rested on the exchange concerning the syntheses topics around the Baltic Sea ES, their valuation and the interrelation with human health and wellbeing.

2.2.1 Syntheses stages

This section describes the stakeholder involvement processes throughout the syntheses process from question formulation to the narrative syntheses. Figure 3 illustrates the literature syntheses stages and indicates the associated stakeholder interaction. The interaction consisted of two main steps: After the project stakeholders identification (as described above, Section 2.1), policy advisers participated in the early stage of the syntheses as advised by Haddaway et al. (2017). For the first stage, four semi-structured interviews with five participants were carried out in spring 2019. The aim of the interviews was to define the scope and research questions of the syntheses. In a second step, two workshops were organised in autumn 2019. The first workshop, ‘Future Outlooks on Oceans and Human Health for the Baltic Sea’, was organised together with the researchers of the review project BONUS SOPHIE to present and discuss the synthesis findings so as to enhance understanding of the current status of research and policy discussions related to human health and wellbeing impacts of the Baltic Sea. Second, a working lunch was carried out with twelve participants from the HELCOM GEAR meeting to present mid-term results, discuss the nature of the research identified by the searches and deliberate on the form of communication of the final results.

Figure 3. Workflow and stakeholder involvement during the three systematic syntheses.

2.2.1.1 Semi-structured interviews

Before initiating the stakeholder involvement process, the first draft versions of the syntheses questions, defined on the basis of the BONUS ROSEMARIE project proposal, were revised and formulated with help from experts in systematic evidence synthesis. The first steps of stakeholder involvement took place in February and March 2019, when four interviews were conducted with four representatives from the HELCOM GEAR group in Finland, Sweden, Estonia, and Germany and one expert for the economic valuation of nature conservation. The interviewees were selected from the HELCOM GEAR group and contacted based on the nationalities and previously established contacts of project group members. This allowed easy access and face-to-face interviews either in their mother tongues, if preferred, or in English. The communication around the interviews was mainly organised by each interviewer, but all interviews were organised within a common timeframe. The semi-structured interviews were conducted using pre-defined thematic questions, similar in all the interviews, together with complementing open questions. To enable all project partners to utilise the interview material, the four interviews were audio-recorded, then transcribed (into English if necessary) and summarised.

The interviews served to identify topics and corroboration. They mapped the base and gaps in the use of knowledge on Baltic Sea ES, their valuation and human health impacts, together with further knowledge needs and expectations from the syntheses. For example, whilst endeavours have been made to integrate human health and wellbeing with ES into marine policies, interviews with HELCOM GEAR members highlighted that research exploring these linkages was either unknown or was thought to be insufficiently strong to inform policymaking. It was therefore identified as a priority for the research syntheses of the topic to show what is known and to identify the knowledge gaps. The interviewees also proposed relevant literature sources and other stakeholders to be involved at later project phases and highlighted relevant policy processes to be targeted in communication. After the interviews and drafting the three protocol abstracts and elaborated syntheses questions, the five interviewees were consulted via email and asked to revise the abstracts and the questions and suggest possible amendments. Incoming reactions were approving and the minor amendment requests were considered to revise the texts accordingly.

2.2.1.2 Mid-term feedback

During the syntheses process, the identified key stakeholders were reached in a targeted manner to inform on the mid-term results and gather feedback on the upcoming steps and means of results’ communication. A workshop with seventeen participants of various disciplinary backgrounds (geology, marine ecology, life science, law, environmental economics, ecosystem services and marine bio-geology) was held in October 2019 in Helsinki, Finland on the relationship of the Baltic Sea ES and human health, in collaboration with the BONUS SOPHIE project. The aim was to identify possible trends in research and policy that need to be considered regarding oceans and public health in the Baltic Sea region. Further, during the 21st HELCOM GEAR meeting, a working lunch with twelve participants from the HELCOM GEAR group was organised to reflect on ES and the valuation syntheses (November 2019 in Helsinki, Finland). Participant representatives were from Finland, Estonia, Lithuania, Denmark, Germany and the European Union. They had not attended the workshop on human health and wellbeing. Both events started with a presentation of the methodological approach, the syntheses questions, and syntheses findings to date.

In the Baltic Sea and human health workshop, the presentation by a member of the health synthesis team was followed by a short time to reflect and ask questions. This helped to set the scene for the workshop, which the BONUS SOPHIE team ran with input from the BONUS ROSEMARIE team in the preparatory stages. The workshop drew on the views of the experts on the future of the Baltic Sea from their own perspectives and related it to known, emerging and potential impacts on human health in the discussions. The participants were divided into small groups to get acquainted and to discuss general impacts and trends the Baltic Sea faces from their viewpoints and expertise. The insights from the discussions were gathered as post-it notes onto boards around the room. A poster was presented that focussed on Helsinki’s demographic and economic trends to prepare the participants to discuss the interdependence of the trends identified earlier and the impacts on public health. Finally, the most topical impacts and trends relating to public health were identified by the participants and discussed as a whole group. Based on the outcomes of these discussions, it was possible to see the health synthesis search strategy covered all the topic areas mentioned and able to identify the relevant literature.

The working lunch focused on Baltic Sea ES and their valuation. The participants were assigned and divided into ES and valuation syntheses related small groups based on their backgrounds and interests. In both groups, there was a facilitator and a note-taker in addition to the participants. The participants were asked to write down their responses to the topics on post-it notes. These were gathered anonymously and then used to structure the discussion. The working lunch followed the approach of structured go-arounds, where people took turns and were instructed not to interrupt the respective speaker (e.g. Schuman 2006). The method is equalising in the sense that it ensures people have the opportunity to speak even if there are implicit or explicit hierarchies among participants. Additionally, the suspension of judgement, when participants are instructed to neither paraphrase nor draw people out (Kaner et al. 2014) allows for an open, relaxed and judgement-free atmosphere since people are free to speak without risking unconstructive criticism.

The workshop outcomes of the workshop discussions were used by the research team to support decisions taken during the article screening and data extraction phase. Additionally, based on this stage of the stakeholder involvement, the ‘extended peer review process’ (see Section 2.2.3), the different means of results presentation and communication were drafted. As informing decisionmakers was set as the key objective and purpose of the syntheses, it was an essential task to ensure policy relevance of the syntheses and enable an iterative and participatory development.

2.2.2 Communication of syntheses results

A variety of media and publication types were used with the intention of fostering science communication towards a diverse audience. To reach a wider audience, an animated whiteboard video was made freely accessible online at an early project stage. The video explains the participatory systematic approach and aims to also reach non-academic or non-specialist audience with no previous experience with the methodology (BONUS ROSEMARIE 2020).

The target groups for the communication of the results in addition to the involved stakeholders were:

• The scientific community;

• Persons and working groups involved in updating the HELCOM Baltic Sea Action Plan;

• Persons and working groups responsible for the Programmes of Measures in the national marine strategies.

The three syntheses were made available through three scientific publications, targeting the research community (Storie et al. 2020, 2021; Kuhn et al. 2021). Additionally, three policy briefs (Håkansson et al. 2020; Kuhn et al. 2020; Storie et al. 2020), addressing decisionmakers and political actors in particular, presented the syntheses results in a condensed and accessible format. As the communication of the results was mainly focused on scientists and policymakers, participating in relevant national and international scientific conferences was an essential way of promoting and communicating the syntheses results and the methodological approach. A webinar in June 2020 combined the project results, along with two other BONUS projects, to inform decisionmakers, research funders and developers of research programmes and projects.

2.2.3 Characterising our approach

This project followed principles aimed at involving both scientific and non-scientific experts in the research process. Others in the environmental management field have characterised this as an ‘extension of the peer community’ (Dankel et al. 2012). It has been argued that such an approach is essential when decisions need to be taken on complex and multicausal issues, where values are in dispute, stakes are high, and facts are incomplete; also that such circumstances are increasingly unavoidable, all of which adds to the need for a ‘post-normal’ science (Funtowicz and Ravetz 1993). Furthermore, the plurality of the perspectives in such situations is seen to call for extended participation that combines extended facts as well as tacit knowledge to produce insights that can support decision-making (Buschke et al. 2019). We argue, in addition, that the approach we describe here gives structure to and increases the transparency of processes identifying evidence. This should further support decisionmakers involved with the complexities of marine governance who need to justify choices between alternative intervention and outcome options (Jentoft et al. 2010; Varjopuro et al. 2014). In planning and initiating the extended peer review process, we considered the following aspects and aims characteristic of the process:

1. In terms of initiating involvement, as Figure 2 suggests and as the Section above describes, researchers and stakeholders were brought together by the research team’s identification of key stakeholders (this can be compared with alternative approaches where stakeholders take the initiative and invite researchers to work with them).

2. The project had a work package dedicated to transdisciplinary dialogue and stakeholder involvement, in which all project partners were included and were given resources. This dialogue work package was designed and coordinated by environmental social scientists, but the implementation process was truly a joint effort. The transdisciplinary approach is characterised by knowledge generation across the conventional academic boundaries of natural, social and health sciences, involving researchers from multiple academic disciplines and different traditions, to address policy audiences in a more holistic way (Choi and Pak 2006).

3. The research team aimed for stakeholder collaboration, offering the opportunity to directly influence the research scheme and share decision-making regarding the syntheses processes. The policymakers involved, however, were neither familiar with the synthesis methodology nor familiar to a participatory approach of this degree. Therefore, the stakeholder involvement throughout the project then classifies mostly as consultation, seeking for the opinion and feedback on proposed action points and possible ways to move forward with the syntheses.

4. Another point important to the syntheses processes was to utilise the limited time of researchers and stakeholders in the most efficient way. This is reflected in the choice towards the purposive selection of stakeholders in comparison to an analysis of all potentially interested stakeholders. Likewise, pre-scheduled meetings of policymakers involved were also utilised, to reduce the effort to participate and therefore increase their likelihood to take part.

5. Furthermore, a key aspect of science communication was to tailor the communication to the target group, known as the need, to ‘know your audience’ (Cooke et al. 2017; Wilson et al. 2017). This means identifying the information stakeholders need to understand and participate in the syntheses processes, the simplification of complex interrelations and their explanation with a sparse use of scientific terminology. In order to allow a quick and graphical access to the syntheses’ method, different aspects were illustrated in a number of figures that were used during the stakeholder events (see for instance Figure 4).

   Figure 4. Example of the illustrations used to describe the project and it's methodological approach.

   

3 Impact of involvement

3.1 How participation added to the syntheses

The stakeholder interactions were a particularly important aspect of the development of the syntheses. The four interviews were used as a tool to further explore and specify previously identified knowledge gaps. The interviewees clarified the need for, and policy relevance of, the proposed syntheses topics. The stakeholders agreed there was sufficient information available on the state of the Baltic Sea ecosystems and the provision of ES but emphasised that understanding of their interrelations and connection to human wellbeing tended to be rather general. The interviews showed how awareness of the complexity and multiplicity of these relations differed depending on stakeholders’ expertise, especially in the context of sociological and psychological aspects of human health. The points, broached during the interviews, fostered the development of questions and discussion points for the two workshops in the later course of the project. While the stakeholders confirmed the initial scope and the drafted syntheses questions for all three syntheses, they did not give specific input to the following steps of defining search strings, search strategy and eligibility criteria. While this would be the ideal, as explained below, stakeholders have limited available time to commit to such processes. The extra time and resources needed to include stakeholders in projects are often emphasised (Cottrell et al. 2014; Haddaway et al. 2017) but less appears to have been said about the time expected of stakeholders in the process.

During the mid-term workshops, the researchers outlined the potential of participatory evidence syntheses for policymaking and presented mid-term results to raise awareness of the available scientific knowledge on the three syntheses topics. An important aspect of the workshops was to identify the policymakers’ demands and expectations of results communication. They explicitly expressed the need for short and concise policy briefs with meaningful figures, due to dense schedules that do not allow for the reading of long and complex scientific publications. The policy advisers also emphasised the timing of the result outputs in accordance with the timeframe of internal policy development to maximise the interest and impact. However, predefined research project timetables may not always coincide with the often relatively short time slots for information gathering in policy preparation.

The regular interaction with stakeholders throughout the syntheses process allowed the research team to tailor the method for each step to the aim of the meeting or workshop and stay flexible in the approach. Through the consistent planning and/or evaluation of stakeholder events during the project the researchers’ awareness for the policy advisors’ interests and needs in regard of the syntheses was constantly raised and brought to focus. It therefore supported the aim of reaching policy-relevant results to be communicated toward policymakers in a form that aided their consideration in future policy making. The stakeholder interaction was therefore important for the researchers’ mindset and approach towards the syntheses, even independently from concrete contribution to different steps of the syntheses process.

3.2 Generating mutual understanding

During the stakeholder involvement it was necessary to find a common language and develop mutual understanding of the syntheses process and the syntheses topics alike. The involved policymakers pointed out that the ES concept is not yet well-established in marine management. Often ES were understood as a theoretical approach with little application in practise. The very specific nomenclature and complexity of classification systems like CICES, the Common International Classification of Ecosystem Services (Haines-Young and Potschin-Young 2018), made the policymakers critical of the potential of the concept for practise and policymaking. The stakeholder interaction made clear, that while there is understanding of market price-based valuation like abatement costs to some extent, there is little understanding of non-market valuation, and especially non-monetary valuation methods. The workshops helped to foster understanding of relationships along the ES cascade (ES are produced from biophysical (ecosystem) structures and processes, via ecosystem functions, services and benefits to multiple values for humankind; see Potschin and Haines-Young (2011) and underlined the relationships of marine ecosystems and their impact on human health and wellbeing. The anthropogenic impact marine uses and policies can have on ecosystems was, in general, well understood by the environmental management experts, while they were not necessarily aware of the reciprocal interrelations of ecosystems and human wellbeing. The participants did not causally link and grasp the extent of all aspects consolidated in the term ‘wellbeing’ and tended to only think of physical health, neglecting, for example, the emotional connection to the sea. Values and connectivity to place can be highly emotional but also well-known to be connected to health and wellbeing (Nisbet et al. 2011). This did mean care needed to be taken to draw out these results in an understandable format for those not (yet) familiar with the more sociological aspects/terminology.

4 Discussion

4.1 Reflection on the stakeholder involvement process

The conceptualisation and execution of the participatory approach was fundamentally affected by two core aspects: 1) the conditions framing the project, especially the associated time frame, and 2) the selection of stakeholder groups. Both aspects were closely interrelated.

4.1.1. Conditions framing the project

Existing research indicates that an average systematic map (providing an overview of the distribution and abundance of evidence in regard to a specific topic) following the CEE guidelines takes approximately 211 person days to complete, excluding the previous development of protocols (Haddaway and Westgate 2019). Combined with project management and other project deliverables (e.g. progress reports, policy briefs), the project consortium aimed to complete three topically different syntheses and the corresponding protocols within 18 months, respectively 700 person days. Time had to be assigned for studying the application of the systematic synthesis method, as most team member were novices to the approach. The identification of the synthesis questions, communication, event planning and the capacity development and understanding between researchers and stakeholders alike, were identified to be very time- and resource-intensive. It needs to be taken into account that stakeholder involvement requires additional time and resources; therefore it might divert resources away from conducting an actual synthesis (Cottrell et al. 2014). In retrospect, it can be said the timeframe for the intended goals was too ambitious without long-term research programme resources, resulting in tasks being finished only after the project.

In response to these constraints, resources in terms of time, skills, know-how and the access to networks had to be optimised to make space for meaningful stakeholder engagement. For instance, use was made of the research group members well-developed contacts with key policy actors and their familiarity with the workflows at HELCOM. These previously existing networks provided the possibility to directly contact and engage with HELCOM GEAR members that were interested and in favour of the approach, in a way that was convenient for their time schedules. Being familiar with the people in question aided in building trust and mutual understanding between the researchers and stakeholders alike. In this way the research team was able to meet the project’s timeframe concerning all deliverables but had to target the stakeholder involvement mainly to policy advisers, without the identification of all stakeholders potentially interested in or related to the syntheses outcomes. Applying a systematic stakeholder analysis prior to the synthesis process helps to ensure the identification of key organisations and groups, that are most important to the particular project and lays the ground to specifically target their interests (e.g. Reed 2008). Therefore, conducting a systematic stakeholder analysis at the outset of the process would have potentially identified a more diverse compilation of key organisations and target groups. Furthermore, the involvement of selected and well-known stakeholders might increase the risk of a biased stakeholder composition and an under-representation of minorities (identification bias) (Leventon et al. 2016). It is therefore a fine line between a biased sample of stakeholders and a well-informed decision to target a specific stakeholder group that is highly dependent on the syntheses’ objectives. The bias towards the HELCOM GEAR group was considered justifiable as it enabled the researchers to directly address the knowledge needs and interests of a relevant part of the policy/end-user community which in turn updates the policies in question. We argue, especially in the context of limited project resources, that the simplified choice through purposive selection of the target group suited the syntheses aim to collect and collate scientific evidence for marine management.

4.1.2 Selected stakeholder involvement methods

As mentioned before, stakeholder involvement in systematic syntheses can have multiple goals, e.g. developing more policy-relevant questions, informing the stakeholders about the research, or higher likelihood that the syntheses results are used in practice (Haddaway and Crowe 2018a). Throughout the syntheses processes, a set of different stakeholder involvement methods were utilised (e.g. semi-structured interviews and workshops) targeting the different aims. Applying a variety of methods allowed a more flexible approach concerning the organisation of stakeholder events and therefore constituted a more efficient resource use. Furthermore, the inherent strengths and weaknesses of each method are to be considered in their selection, dependant on the intended goal (Reed et al. 2009). The use of different methods enabled the involvement of different people within each of the involvement stages. By enabling participation without a request to engage in the whole process we strove towards more balance in stakeholders reached and the reduction of commitment and resource bias. As people take part for different reasons, a different set of activities can be helpful to meet heterogeneous interests (Haddaway et al. 2017).

Due to the limited time and capacity of stakeholders, a highly effective use of these resources is needed. We advocate to organise stakeholder events alongside their work schedule, in the sense of going where the stakeholders are. This pragmatic approach acknowledges the busy agendas stakeholders might have and so may assist in addressing time constraints as a barrier to participation. Nevertheless, the level of flexibility required when involving stakeholders in the synthesis process should not be underestimated (Keown et al. 2008). One example of using stakeholders’ time efficiently was to gather mid-term feedback during a workshop that took place over lunch during a previously scheduled meeting, thus participants had little to no extra effort to attend the event.

4.1.3 Selected key stakeholder group

The policy-advisers were mostly unfamiliar with the systematic synthesis method and its potential for directly influencing a research scheme. Stakeholders have different reasons to get involved (Haddaway et al. 2017) and it must be clear to them why they should invest their time and resources. Time and resources are needed to reach the persons of interest, pull their attention towards the research objective, and explain their potential role and influence. Stakeholder expectations should be identified and managed early in the synthesis process (Taylor et al. 2017). Through the pre-existing contacts, a level of trust and understanding of intentions was established from the beginning of our review process. While the responsibility to carry out the three syntheses laid in the hands of the research team, the policy-advisers were invited to contribute to their development to reach more policy-relevant results.

4.2 Implications for future systematic syntheses

4.2.1 The role of participatory systematic syntheses

The potential of participatory systematic syntheses has been broadly discussed within the scientific community engaged with systematic maps and reviews (e.g. Dicks et al. 2017; Haddaway et al. 2017; Pullin et al. 2020). Taking the example of environmental research of the Baltic Sea region, substantial volumes of knowledge have been generated (e.g. Andersen et al. 2017; Reusch et al. 2018), especially fostered by BONUS, the joint Baltic Sea research and development programme 2010–2020. Systematic syntheses have the potential to identify knowledge clusters and can point out lacunae of knowledge where no or insufficient information is currently available. While knowledge clusters can be the source for more detailed synthesis approaches, they also represent data availability and access, as well as previous research scheme and funding foci. Concerning such clusters in the Baltic Sea region, the focus has predominantly been on biophysical assessments and comparatively easy to assess aspects like eutrophication mitigation, food provision (fishery) and recreation (Kuhn et al. 2021). To facilitate a holistic and integrated approach towards sustainable marine management, more focus needs to be given towards the manifold interrelationships of the marine environment and human wellbeing (Storie et al. 2021). However, arriving at a perfectly comprehensive understanding of natural conditions is an unreachable goal and the demand for more accurate data should not become an excuse for inaction. The creation of participatory systematic syntheses encourages the use of the best available knowledge and data to support evidence-based decision-making. Engaging stakeholders in literature syntheses raises their awareness for the topic in question and the exposure to the rigorous systematic approach can increase their trust in the results. The participatory approach thereby represents an opportunity to draw attention towards the available scientific knowledge and the applicability of synthesised evidence. However, to this point, the capacity of researchers to commission meaningful participatory systematic reviews and understanding of stakeholders of the method are still rare but constitute a growing field. With experience of conducting participatory systematic reviews, both researchers and stakeholders will become more aware of the methodological opportunities and reflect on their needs and the time necessary to accommodate them. In this way every stakeholder engagement becomes an opportunity for training and capacity building (Eales et al. 2017).

4.2.2 Communication at the science-policy-society interface

There are two levels of communication to consider when working with stakeholders: firstly, communication is needed to organise the interaction process; secondly, communication is needed to generate mutual understanding with regard to the topics and the methodology. The design and execution of stakeholder intervention requires expertise in communications and preferably transdisciplinary collaboration involving social and natural scientists that understand each other’s research topics and methods alike. Involving researchers from multiple academic backgrounds allows the combination of different ways of thinking when approaching challenges and finding solutions. Furthermore, environmental systematic syntheses profit highly from the participatory methods and approaches established in the social sciences. Their additional value should not be underestimated by more traditionally trained natural scientists. Appropriate education and training or involving people specifically trained in applying participatory approaches and science communication, would be an important support for adequate interaction and result communication. These so called knowledge brokers aim to bridge the gap between the two communities of knowledge producers and users (Caplan 1979). They would particularly coordinate and manage the interaction between scientists and practitioners, as well as transfer and translate research for use by decision-makers. Knowledge brokers, often also referred to as boundary organisations or individuals, are expected to remove barriers to effective knowledge exchange (i.e. the boundaries) and facilitate ‘a culture that values the use of the best available science in policy and practice’ (Guston 2001; Cvitanovic et al. 2016).

Successful interaction and knowledge exchange between scientists and practitioners needs effort on both sides to find a common language on the research subject. Appropriate science communication and the simplification of complex and multi-layered issues is needed from the researchers (Fazey et al. 2012). An important aspect is the use of a consistent terminology for the same concepts and the translation of approaches and methods across different fields of expertise. Otherwise, scientific terminology and the extensive use of acronyms can act as a barrier rendering knowledge inaccessible, especially in transdisciplinary research teams and towards stakeholders (e.g. Barnett and Doubleday 2020). The complexity of scientific content needs to be communicated and were required, simplified to an extent that keeps policymakers involved in the process and still provides sufficient input for evidence-informed decisions as a result. For example, in our experience, the complexity of the ES concept needed translation and as such our scientists were acting as knowledge brokers. While the ES concept provides an integrated approach towards a more holistic management of complex adaptive social-ecological systems, it must be clear there are seldom simple or straightforward solutions for the complex ‘real world problems’ that need to be approached through transdisciplinarity (e.g. Jax et al. 2018).

Bearing this in mind, policymakers might not necessarily have to understand every detail and aiming for such is neither realistic nor efficient. It is helpful to identify to which extent stakeholders are aware of the terminology and the scope of their input. A major goal at the beginning of the stakeholder involvement process is to understand the stakeholder’s relevant knowledge gaps and to identify the time and effort needed for science communication. This applies especially for diverse stakeholder groups. Otherwise, communication and identification of knowledge gaps between science and decision-making and vice versa might become emphasised more during the involvement than intended. In our case more time than originally planned by the research team had to be assigned towards the description and communication of the different aspects of the ES cascade (Potschin and Haines-Young 2011). This was because the policy makers involved were not necessarily aware of the potential of the ES concept when combining ES mapping and assessment, valuation and the integration of environmental and human needs through the environmental and social science implementation. In this way we experienced some of the key challenges in stakeholder engagement: additional time and resources are required (Ward et al. 2009), and their distribution needs to be planned well as stakeholder engagement may otherwise divert resources from the conduct of the review (Cottrell et al. 2014; Haddaway et al. 2017).

4.2.3 Researchers’ needs

This aim to develop participatory systematic literature syntheses for the research community and policymakers alike is ambitious. Researchers need to step out of the solely expert-position and foster dialogue to enable stakeholders to bring in their expertise (Jacobs and Burkhard 2017). As knowledge brokers researchers must incorporate numerous skills in the sense of understanding the research topic and its methodologies, as well as the expertise and specialist skills to carry out a systematic synthesis and translate its findings towards the knowledge users (Gough et al. 2013). Furthermore, the stakeholder identification, the design of the involvement activities, the communication, and the process of translating information from practise to science and back are resource-intensive, especially in terms of time. Resources, as well as the skill set required (e.g. communication, mediation, networking, teaching, mentoring and interpersonal skills) constitute one of the main challenges within the role of knowledge brokers (Ward et al. 2009). Therefore, a dedicated person or group experienced with stakeholder participation and a firm understanding of the evidence syntheses methodology is key and should be part of the research team and/or the advisory board.

To implement effective transdisciplinary research and further support evidence-based decision making, the participatory efforts to evidence synthesis processes need to be better recognised and valued more by research funders and within the scientific community. Funding institutions should consider strengthening the focus on the science-policy-society interactions in research schemes and support knowledge brokerage by supplying funding to identify relevant stakeholders, identifying their interests and finding a common language. Another relevant aspect is funding travel costs in order to increase stakeholder mobility and fostering personal interactions to grow mutual trust and understanding (e.g. EU COST Actions). Furthermore, a crucial point is the timing of research funding regarding ongoing policy processes. This is more in the hand of funding institutions than researchers, who are not necessarily able to set their own emphases but often adapt research plans to third-party fund requirements. While depending on funding for extended project preparation, it can be helpful to integrate the expertise and stakeholder interests prior to project application (Görg et al. 2014; Spangenberg et al. 2015). Another point in the hand of funding agencies, is the option to modify and adjust research projects in the course of the project depending on stakeholder groups and their needs (Durham et al. 2014).

Academia, however, plays an equally important role in fostering and valuing a culture of participatory contributions to the science-policy-society dialogue and in literature syntheses within the scientific community. A higher professional appreciation of stakeholder cooperation needs to be rewarded as an important aspect of scientific careers that requires not only resource contributions, but also appropriate training. This is especially needed for early career scientists, who require valuable outputs in measurable form such as peer-reviewed publications or acquired third-party-funds. Therefore increased scientific recognition is needed to endorse the time and effort dedicated towards resource intensive participatory and communication processes. The provision of sufficient resources from academia and research funders alike to carry out policy- and practise-relevant systematic syntheses, that meet the requirements of being transparent, objective, and repeatable, is needed.

5 Concluding thoughts

The approach of this study was to reflect on the stakeholder involvement process during the compilation of three systematic syntheses on: marine and coastal ES in the Baltic Sea; their relationships to human health and wellbeing; and their valuation. Involving stakeholders throughout the syntheses process increased awareness for the available research on a specific topic and its limitations. The rigorous systematic synthesis method has the potential to build additional trust. The participatory approach aimed at achieving policy-relevant results to support evidence-based marine policy decisions and the update of the key policies managing the Baltic Sea ecosystems. We reflected on our experience and discussed implications for future research to support the stakeholder interaction process in future systematic literature syntheses. From our experience we can draw the following key recommendations:

5.1 Awareness of available resources for the participatory approach in systematic syntheses

Scientists need to become aware of the available resources for the participatory approach in syntheses projects, ideally from the early project planning and application phases. Time, in the sense of project duration and person-hours, and the availability of other resources like education and training of the participating scientists or access to experts with the corresponding expertise, have been identified as the key limiting factor that should not be underestimated when involving stakeholders in the synthesis process. If insufficient prior scheduling is built into the project, then adequate stakeholder engagement might be the aspect that gets cut first.

5.2 Identification of relevant stakeholders and the communication throughout the synthesis process

The stakeholder involvement can be an encouraging step towards co-creating meaningful results through direct feedback. To reduce bias in the synthesis, efforts need to be made to ensure a wide a range of stakeholders are incorporated. However, the extent of stakeholder analysis and engagement will depend on the available time and resources. A time-saving approach can be to utilise pre-established contacts to key stakeholders interested in the subject and method, in cases where the capacity to carry out a full stakeholder analysis is not given. Such pre-existing contacts also hold a higher likelihood of engaging people and providing a mutual trust from the beginning of the process. Another recommended course of action is to look for opportunities to connect with stakeholders in the sense of previously scheduled meetings, conferences, and travel efforts. In this way, time resources of stakeholders can be used in a more efficient manner. It is important to build capacity among stakeholders, in terms of their interests and the available time required accordingly.

5.3 Application of a set of diverse methods for participation throughout the synthesis process

This enables researchers to find the best method for the agreed goal for each specific situation and purpose. It furthermore ensures the engagement of different stakeholders that participate for varying reasons. The systematic design of the stakeholder engagement steps, and their transparent communication allow participants the equal opportunities of involvement. While this study mainly presents the researchers’ perspective, it would be valuable to similarly reflect the stakeholders’ experiences during the synthesis process, as analyses and reflections from this point of view do not exist.

5.4 A dedicated person or team with experience in stakeholder engagement

A dedicated person or team with experience in stakeholder engagement methods and the suitable academic background and training should lead the conceptualisation and organisation of the participatory process. It is valuable to have team members that oversee the participatory process and take the responsibility for organisation and communication around stakeholder events. However, engagement from the whole research team and their multidisciplinary background benefits the process.

5.5 Translation of knowledge between researchers and stakeholders

A key aspect of stakeholder engagement in systematic syntheses is the translation of knowledge between researchers and stakeholders. Scientists must ensure their terminology is accessible for and tailored towards their participants, i.e. reflecting also on their use of scientific terminology. Where possible, the subject matter of the research must be presented in a simplified and comprehensible, but still sufficiently comprehensive manner. At the same time, there is no need for stakeholders to understand every detail. Both parties can learn from the knowledge and experience of the other and it is a team effort to find the right means of interaction, understanding and mutual respect of each other’s expertise and knowledge.

Acknowledgements

We thank all members of the BONUS ROSEMARIE project team for the fruitful collaboration. Also, we would like to thank the anonymous reviewers for their diligent work and thorough feedback on an earlier version of this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This study was supported through the BONUS ROSEMARIE project, which received funding from BONUS (Art. 185), funded jointly by the EU and the Swedish Research Council Formas and the Estonian Research Council. The publication of this article was funded by the Open Access Fund of Leibniz Universität Hannover.

Notes

1. HELCOM is an intergovernmental organisation consisting of the nine countries bordering the Baltic Sea (Denmark, Estonia, Finland, Germany, Latvia, Lithuania, Poland, Russia and Sweden; see Figure 1) and the European Union. Its goal is to protect Baltic Sea ecosystems from the environmental pressures of human activities. The GEAR group, consisting of delegated representatives of HELCOM contracting states, works towards incorporating ecosystem-based approaches and management in the implementation of the EU Marine Strategy Directive (MSFD). GEAR focuses on the relationship between the MSFD and activities of HELCOM, such as the BSAP (Baltic Sea Action Plan) and HOLAS (Holistic Assessment of the Ecosystem Health of the Baltic Sea) and proposes means to improve synergies and coordination between HELCOM and the national implementation of the MSFD.