Interactive reflection trainings on RRI for multiple stakeholder groups

ABSTRACT

The article builds on the perceived gap between ‘RRI in theory’ and ‘RRI in practice’ and demonstrates the need for RRI training for different stakeholder groups so as to overcome this gap. The RRI Tools project has developed different kinds of materials and tools that can be used in RRI trainings. Based on the umbrella notion of RRI, as elaborated in the RRI Tools project which spans the six key dimensions of RRI, its process requirements, outcomes and stakeholders, the training illustrates exemplary exercises that would serve a different purpose: to raise awareness for RRI, to enable mutual understanding of different stakeholder groups’ perception on RRI, to reflect on RRI and to implement RRI in daily practices. Experiences gained from the various training exercises are described, which result in recommendations and guidelines on how to set up a multi-stakeholder workshop in terms of setting, methodology, content and participants.

KEYWORDS:

• Multi-stakeholder workshop
• RRI training
• RRI implementation
• mutual learning
• reflection and anticipation

1. Introduction: implementing which RRI

The idea of emphasising responsibility in research and innovation processes by reflecting societal needs has been on the table for quite a while now. However, by formally introducing the term of Responsible Research and Innovation (RRI) as a cross-cutting issue of the European Commission’s Horizon 2020 programme (European Union, European Commission, and Directorate-General for Research and Innovation 2013), the debate of how to define ‘socially responsible research’ has gained momentum within the European Research Area. Hence, a variety of different interpretations and operationalisations have been brought to the fore and are under constant examination (cf. EPSRC 2015; European Commission 2013; European Union, European Commission, and Directorate-General for Research and Innovation 2013; Owen, Macnaghten, and Stilgoe 2012; Stahl 2013; Stilgoe, Owen, and Macnaghten 2013; Sutcliffe 2011). Focusing on the European H2020 programme, RRI comprises long-standing perspectives such as public engagement, ethics, science education, open access, gender equality and the governing of these aspects in order to exploit the potential of offering new opportunities and addressing societal challenges. RRI is meant to increase the quality of research and – in the long run – to contribute to boosting the European economy and its growth. It aims at better aligning both the process and outcomes of research and innovation with the values, needs and expectations of the society (Wickson and Carew 2014). As a result, the conceptual debate is ongoing while the concept’s strategic significance for researchers, projects and innovators is becoming clearer.

The here presented RRI tools project¹ has taken up one of the probably most prominent definitions of RRI, the one provided by René von Schomberg (2011): ‘RRI is a transparent, interactive process by which societal actors and innovators become mutually responsive to each other with a view on the (ethical) acceptability, sustainability and societal desirability of the innovation process and its marketable products’ and advanced it further. As agreed within the project, another definition of RRI is that of ‘a dynamic, iterative process in which all stakeholders in research and innovation become mutually responsive and share responsibility for both the process and its outcomes’ (Klaassen, Kupper, and Broerse 2014). This means that research and innovation will only be regarded as ‘responsible, if they are aimed at particular outcomes and also meet certain process requirements’.

When it comes to implementation of RRI, however, it becomes obvious that RRI expertise so far is mainly theoretical and practical experiences are still lacking (Smallman, Lomme, and Faullimmel 2015). In a practical context, the vagueness of the concept gives rise to difficulties when it comes to implementing it into daily (research) practice.

As RRI is a concept with a very broad scope combining the aforementioned key topics, it simultaneously addresses research processes, stakeholder groups and all other actors affected by it. Hence, RRI is not only meant for researchers but for all different stakeholder groups such as representatives of science education, industry and business, research, civil society actors and policy-makers. Although it has become apparent that RRI as an umbrella term and concept offers a major opportunity to bring together all different stakeholder groups and as well as a holistic view on the research and innovation process, this broad and all-encompassing nature of RRI can at the same time also be considered one of its biggest challenges (Callon and Lacoste 2012).

When implementing RRI in practice, not only do the needs and environments of the specific stakeholder groups have to be taken into consideration, but its application as well. Keeping this in mind, the RRI tools project was therefore aimed at consulting stakeholders on current understandings of RRI as well as to relating the theoretical concept with the specific needs and working situations of the different stakeholder groups. Accordingly, the project chose an approach which derived its rationale from the following aspects:

• First and foremost, we identified a strong need for trainings and practice support for all stakeholder groups.

• Beside these stakeholder group-specific trainings, the exchange between the different groups is a major contributor to the enabling of RRI. It follows that multi-stakeholder workshops would be the most appropriate form of training as they provide room for exchange and understanding of the role and relevance of RRI for the various actors involved.

• Face-to-face training workshops making use of interactive methods and hands-on exercises help to break down the theoretical concept of RRI to a more practical understanding for the individual working environments. Such an understanding should ideally lead to an integrated RRI approach embedded in all fields of research and innovation rather than just adding on a particular singular task.

In this contribution, we present trainings for multiple stakeholder groups, which describe and reflect on the applied methodologies, the reactions and feedback of the participants and lessons learned.

The following chapters will focus on the stakeholder specific trainings conducted by the Austrian and Slovenian hub coordinator, the Centre for Social Innovation in Vienna (ZSI) (see next chapter). We will discuss methods and experiences in order to exemplify how to set up RRI trainings, what is needed to provide most effective spaces for meaningful encounters and learning, which actors should be addressed and which methods to be applied to work with multi-stakeholder groups. A special focus is given to the training workshops held only by the ZSI team², which exemplarily demonstrates how the RRI concept can be explained in trainings, which methods could be used and finally which of these worked. After summarising the experiences and recommendations gathered in the trainings, we reflect on the lessons learned as a result of the trainings presented.

1.1. The RRI-Tools project – a practical approach for RRI

The RRI tools project, funded by the European Commission under Framework Programme 7, aims at guiding stakeholders to implement RRI into daily practice. Twenty-six partner organisations organised in 19 so-called hubs – dispersed all over Europe – attempted to reach out to different target groups, to disseminate the concept of RRI and to involve stakeholders in trainings. The stakeholder groups identified by the project are: policy-makers, educators, researchers, civil society organisations, and business and industry representatives. Following a bottom-up approach by involving more than 400 members from all aforementioned stakeholder groups in nearly equal shares right from the very beginning, the RRI tools project succeeded in developing a common understanding of the concept of RRI. To build up a community of practice in Europe, gain a common understanding of RRI and identify ways how to implement RRI within stakeholders working environments, a range of activities were carried out such as

• Consultation Workshops

• Collection and Development of Tools Trainings

• Communities of practice

• Trainings

1.1.1. Consultation workshops

During the first third of the project duration, 27 consultation workshops in 30 European countries were conducted. The consultation workshops aimed at identifying future stakeholders’ needs and constraints when putting RRI into practice and at delivering input for the development of training materials and methodologies. Furthermore, the workshops served to deliberate the working definition presented, to arrive at a mutual understanding of RRI and to exchange information on the practical, and hopefully inspiring, implementation of RRI in different fields. A total of 411 stakeholders participated.

The full day workshops followed a standardised procedure with precise guidelines on how to conduct and document the sessions, including stakeholder identification and invitation policy (Marschalek et al. 2015). Hub coordinators also received full-day training on the consultation workshops. Finally, the consultation process achieved almost equal composition of the five stakeholder groups. Mainly, the workshops consisted of four sessions and nine interactive exercises in which the participants were provided with the working definition of RRI as suggested by the project and related it to their own experiences and working environment. They also discussed what characterises ‘promising practices on RRI’ and worked on the stronger and weaker points of the concept. Concerning implementation of RRI in the own working environment, participants worked on obstacles and opportunities and identified actions and solutions for their needs. Each workshop provided a compilation of needs for stakeholder actions and solutions and ended with a feedback session.

Broadly speaking, in terms of RRI, results showed that there was little knowledge thereof and RRI as a concept was too abstract and difficult to engage with. Furthermore, participants of the consultation workshops envisaged poor reception as a great obstacle in the implementation of the RRI concept. With regard to the content, a total of seven clusters of the main opportunities and nine clusters of the main obstacles of implementing RRI were identified as a result of these workshops. Participants of the workshops saw the greatest opportunities in bringing science and society closer together and improving innovation. While these two aspects are essential when it comes to finding incentives to encourage different stakeholders to attend trainings and to start their RRI implementation process, obstacles and challenges were of more interest when developing training materials and methodologies.

One of the biggest obstacle clusters was ‘culture’, addressing disciplinary boundaries in the scientific realm, lack of innovation culture and lack of collaboration culture across stakeholder groups, which can specifically be addressed through multi-stakeholder group trainings. Another main obstacle, which needs to be taken into account and actively addressed within trainings, was found to be the cluster of ‘attitude’, addressing the lack of ‘buy in’. This describes the willingness of different stakeholders to taking up the RRI principles. To augment this willingness, potential benefits and incentives must be established, and clearly defined career benefits, or requirements for RRI need to be created, as resistance to change is a major challenge. This is especially vital in large systems or hierarchies where there is not much interest or lack of confidence in change. Additionally, unwritten rules and norms of the respective culture are clearly challenging in implementing RRI throughout all stakeholder groups (Marschalek 2015).

Another big obstacle cluster was knowledge of RRI. Based on the lack of a common understanding of the concept of RRI, the question of how and when to implement RRI within one’s own working environment is still a great challenge due to absence of norms, regulations and examples of good practices. Furthermore, augmented personal skills and an understandable language, which helps communication between all stakeholder groups, are needed (Marschalek 2015).

These and more challenges served as input for setting up the trainings, deciding which methods should be used and what material fits best.

1.1.2. Tools

Good practices, showcases and case studies can be of practical use in this respect. For this purpose, the RRI Tools project offers implementation tools like a Self-Reflection Tool, a database of resources and several How-To application guidelines, so-called ‘How To’s’ that help to guide the process of changing towards a more RRI-driven research approach (see in Section 2.3.1). These tools are of great help when implementing RRI, although they still need to be adapted to a specific working environment and individual approaches to exactly fit the distinctive and diverse needs of researchers and other stakeholder groups.

1.1.3. Communities of practice

The project also aimed at establishing an RRI community of practice involving members from the consortium, from the hubs and from national stakeholder representatives in the wider sphere. Via two train-the-trainer workshops (T3), representatives from all hubs were invited to receive training on RRI, while resources were allocated to them to organise trainings for interested stakeholder groups in their respective countries. In the final year of the project, the wider community of practice had gained more than 900 members and more than 5000 followers on its social media channels, newsletter and online forum.

1.1.4. Trainings

To enable hub coordinators to carry out stakeholder trainings in their countries or regions, two internal train-the-trainer (T3) trainings were organised. Both T3 workshops were comprehensive workshops, lasting three days, with theoretical insights and practical hands-on exercises. The purpose of the first T3 workshop was to get a clear understanding of RRI from different stakeholder perspectives, to get to know the RRI toolkit and to work on showcases illustrating the potential of RRI in practice. The T3 workshop was interactive in nature, comprising exercises in smaller groups and also reflection and discussion rounds in the plenary as an inspiration for training to be developed by the hubs in their own countries.

Following these T3 workshops, the 19 hubs have organised more than 90 training sessions, around 100 advocacy meetings and 200 dissemination events with more than 3000 people eventually trained all over Europe (cf. Figure 1).

Figure 1. Training sessions organised by Hub members (green) and international organisations (red) (Garcia et al. 2016).

In this article, we will focus on the series of workshop and trainings the ZSI team – as Austrian and Slovenian hub – carried out, including 7 trainings with multi-stakeholder groups of 10–20 participants. The following sections are based on these experiences.

1.2. Learning for all stakeholders

Based on the complexity of the RRI concept and multi-stakeholder approach of the RRI Tools project, as previously described, we decided to create trainings for all stakeholder groups identified within the RRI project to be applied in one setting. Besides some outcomes intended specifically for different stakeholder groups, the goal was that all participants would be able to understand and explain the concept of RRI and its component parts. They would be empowered to contextualise RRI according to their own practices and reflect these individually and as a group. After the training, they would be able to identify the opportunities and the obstacles of the implementation of RRI and gain some ideas on how to address them and which changes are required in their working environment. Finally, they would be able to use the RRI Tools toolkit effectively and become encouraged to use it in real-life situations.

To achieve this in the trainings, the different perspectives and needs were considered within the training methodology. Not only do different stakeholder groups matter, individuals with different kinds of pre-knowledge and readiness for RRI implementation do too. Therefore, aspects of individual and peer learning needed to be implemented in the trainings. To be able to make use of the wisdom of the crowd and to broaden the mirroring perspectives, techniques had to be selected and adapted accordingly.

2. From theory to practice: RRI trainings

RRI trainings for stakeholders are a first step towards the implementation of RRI in all future research and innovation activities. To understand the concept and answering questions such as ‘why should it be part of my research process’ or ‘am I not already doing responsible research’, support and trainings for all different stakeholder groups of different levels of pre-knowledge of RRI are needed.

2.1. Approach: building up one setting for different stakeholder groups

Based on the agreed-upon RRI definition of the RRI tools project (Klaassen, Kupper, and Broerse 2014), the complex concept of RRI was broken down to four dimensions, encompassing existing different approaches, namely the six keys identified by the European Commission which are seen as powerful policy agendas (Schrammel et al. 2016), four process requirement clusters based on the research of Stilgoe, Owen, and Macnaghten (2013) and Wickson and Carew (2014) and outcomes comprising learning outcomes, research and innovation outcomes and solutions for societal challenges. Stakeholders complete this multidimensional approach as shown in Figure 2.

Figure 2. This graphic shows the Policy Agendas, Process Requirements and Stakeholder groups based on the RRI Tools concept definition (source: www.rri-tools.eu).

Based on this elaborated notion of RRI with its four dimensions, we created a training concept, which we implemented and tested in a series of workshops in Vienna. We invited participants representing the five different stakeholder groups (as identified by the RRI tools project – see Figure 2 and chapter 1.1), following the same recruitment criteria and processes as outlined for the consultation workshops, to take part in a half days training workshop to learn and reflect on RRI. With the ‘from theory to practice’ approach, we aimed at reflecting the rather abstract concept of RRI through the lens of their own work and experiences. The locations offered a nice, light and friendly room for structured training activities including networking and informal encounters for exchange and reflection on lessons learned.

The multi-stakeholder approach supports the idea of cross-fertilisation, bringing different groups to one table to come into closer contact with each other. Training groups consisted of equally composed groups of stakeholders. Already within the training situation, participants had the opportunity to get to know the perspectives on RRI of other stakeholder groups and thus learning from each other. According to our experience, new knowledge as well as new pathways for collaboration are generated together.

2.2. How to train RRI: training concept

In general, the aims of the RRI trainings were to raise awareness for RRI, to become familiar with the topic and to build up a common understanding of the concept. In the long run, our intentions were that the trainings should lead to a commitment to RRI and the future inclusion of its principles. The trainings would equip participants with basic knowledge of RRI, specific tools for implementation as well as arguments for its relevance and an understanding of how RRI can be successfully implemented in their environment. The applied methodological approach was explicitly tailored to the RRI concept addressing main RRI aspects such as inclusiveness, anticipation or reflexivity. It thus had a strong focus on reflection and self-reflection. The workshop format was applied because workshops are a good forum for debate and discussion of complex issues and can prove also good educational experience for the participants (cf. Human and Davies 2010).

The format and applied techniques had an interactive and energising character, which allowed for discovery and experience of problems and concerns, as well as for work on possible solutions (Creek et al. 2014). The theory that non-verbal formats, especially, such as drawing or scene works which reveal details and findings that usually escape verbal activities (Höykinpuro and Ropo 2014), supported this approach. They followed sociometric and theatre pedagogical approaches, amongst others, as introduced by, for example, Boal (1999), Diamond (2013) and Johnstone, Schreyer, and Schreyer (2014). Accordingly, the trainings had the format of very interactive workshops, following a pre-defined structure of a series of participatory techniques applied in the entire group or in small break-out sessions. These techniques, as described below, can be regarded as modular components for the composition of an RRI training workshop.

2.3. Details of the training

To start with, it might be useful to get an overview of participants’ existing knowledge and experience of RRI within the training group. One way is to visualise it on a board with two scales from 0 to 10, asking them one by one to indicate with a sticky dot on these scales how much (a) knowledge and how much (b) practical experience they already have. This helps them to reflect on what they already know or how many opportunities they have so far had to implement RRI principles. The visualisation also shows differences between these two scales, which are expected to occur. As the participants talk about their choices, the training group learns about considerations and arguments behind these self-estimations. The final picture shows levels of knowledge and experience of the whole group and existing gaps between theoretical knowledge and practical experiences.

After this initial stocktaking, the participants were given a very brief presentation of RRI. The theoretical concept of RRI so as to understand its practical implementation for the different stakeholder groups was explained to them. The contextualisation of RRI in the working environment was the next step. The applied training element was based on appreciative inquiry (Cooperrider and Srivastva 1987). In pairs or small groups, participants interviewed each other about experiences related to RRI, talking about a specific context, which comes to their mind. They were given guidance to listen out for specific issues and to give feedback on what they heard. They were asked to draw their attention precisely to who was involved, what the important moments were and what had led to changes but also why the process had been undertaken. After these mutual interviews, they were asked to write down which aspects they had in common and define which they regard as being positive or negative. In doing so, both the narrators as well as the listeners, were able to reflect and learn. Afterwards, they shared the lessons they learned with the whole training group. The clustered group results show the most important aspects as brought together by the participants. This common knowledge pool builds a good basis for further discussion.

Another technique used to introduce the topic of RRI in the training and to understand others’ views and needs was an interactive game with mixed pairs of stakeholders. Each participant was handed over a paper hat (or alternatively, coloured sticky dots on name tags) in a colour which represented one specific stakeholder group. They were then asked to build pairs of different coloured hats and tell their vis-a-vis why the RRI concept is of importance and relevance for them and how they could address it specifically. The challenge of having two minutes to explain to the representative of another stakeholder group why RRI is relevant and what are the main benefits for their peers, brought up ideas on when and how to collaborate with other stakeholders. It also opened up ideas for various implementation fields for RRI. After three rounds of mixed pair’s conversations, participants gathered in small groups of hats of the same colour. They were now able to compare what they heard in the previous rounds and discuss it within the group. They then wrote all aspects they collected on coloured cards. All groups present their cards and findings to the plenary afterwards. They were thus able to get a comprehensive overview of arguments and requirements concerning all stakeholder groups. They experienced how expectations differ between stakeholder groups. Furthermore, they gained a better understanding of concerns and difficulties, but also some ideas for needs and implementation requirements. Especially for a multi-stakeholder group, this method proved to be important for reflecting on RRI’s relevance for other stakeholders (Figure 3).

Figure 3. Clustered results of training workshop (source: Marschalek 2017).

One technique to help participants experience how to implement different aspects of RRI within their own practice and to make them visible also for others, was the creation of theatre scenes or living sculptures. These techniques, however, needed a little warm up and preparation as not all individuals were ready to become openly engaged. Addressing a specific RRI issue, for instance, ‘a visionary scene on future public engagement in RRI’ on the topic of Public Engagement, participants performed what they commonly created in small groups. Performers as well as the audience afterwards reflected on what they experienced and observed and together they arrived at conclusions on what they learned. This also helped them to understand their own attitudes and assumptions better, as well as the roles of others – in this case of those who should be invited to public engagement activities. This way they could check and experience there and then if the envisaged methodology for Public Engagement was appropriate and likely to be successful and accepted by all actors.

For a deeper understanding of the four dimensions of the RRI concept, as described above, a suitable approach is to work on a case study which is a detailed description of one practical RRI implementation example. We took, for instance, one case description from the RRI-tools online repository on inspiring practices. Although any other case could have been taken, the interesting aspect was the fact that this case had already been selected as one of the RRI Tools activities as an inspiring practice for RRI and thus addressed different aspects of RRI. It was in actual fact, the case called ‘KlimaAlltag’³, a German compound project researching how different social conditions can affect a more climate friendly live in urban areas. Amongst other methods, a field study integrated 80 so-called climate households who collaborated over a period of more than six months with the project team. The task of the training was to find out if the four RRI dimensions were addressed (or not) and how or to which extent. To discuss the four different dimensions, small groups had to be formed accordingly, meaning one group working on process requirements, one on policy agendas, one on outcomes and finally another one on stakeholders to have a look at the actors involved, to find out who was addressed or engaged in the case and who was not. Participants were asked to discuss and detect pros and cons, possibilities and limits of the case study from the perspective of the respective RRI dimension. Furthermore, they were asked to have a look at the actors involved, to find out who was addressed or engaged in the case and who was not. The groups were also provided with different material for visualising their discussions, such as different coloured sticky dots, markers, pens, playing pieces, etc. (see one flipchart results as shown in Figure 4). The plenary discussion afterwards, with presentations of all small groups clearly demonstrated that there are many overlaps between the different aspects and view-points, including also the complexity of the RRI concept and how the different dimensions are interwoven so that no single definition would satisfy the understanding of the concept. Participants were also able to critically reflect where and how RRI dimensions were addressed in the show case and where they were not. They were also able to collaboratively discuss options for improvement.

Figure 4. Showcase group work on Process Requirements, RRI tools Hub training workshop November 2016.

2.3.1. Supplementary material: the RRI-tools online toolkit

Within the training, we also introduced the RRI-tools online toolkit⁴, which provides a wide range of useful tools for practical implementation of RRI. The tools can be filtered according to categories, topics or stakeholder groups to ease the search for an appropriate tool for the specific implementation context. The different kinds of tools within the toolkit were:

• Library elements to inform of RRI and its various facets; the library compiles all sorts of relevant resources that inform of RRI and its various facets. This includes academic articles, reports from projects or institutions at all levels, books, opinion pieces, videos, presentations, as well as dedicated journals and blogs that cover RRI or related aspects.

• Projects on RRI and closely related fields to build upon and collaborate with; Projects’ resource pages offer a simple summary of past or ongoing initiatives dealing with RRI or any related aspects (policy agendas, societal challenges, etc.).

• Good practices to inspire and adapt to other contexts; and ‘How-To’ application guidelines which should help users understand how they can solve a given challenge through examples and the contextualised use of specific resources of the toolkit;

• Tools to plan, implement, evaluate and disseminate a more socially RRI.

One of the tools was the self-reflection tool⁵ for individuals or teams. The Self-reflection tool is meant to support the idea of continuous reflection on RRI by prompting discussions on RRI with the overall aim to contribute to the paradigm shift from traditional science and innovation towards RRI. It is an online tool for all identified different stakeholder groups (researchers, policy-makers, educators, civil society organisations, business and industry) and there is also a printed version available as a small booklet. It comprises questions that match the different keys of RRI as well as process requirements. The tool is adaptive in the sense that users can choose which policy agenda they would like to work on and additionally which set of questions they would like to answer as not all will be relevant for their context. Moreover, there are open text fields in each key that allows the users to add questions that they find relevant for them. After finishing the work on the questions and answers, the system creates a distinct reflection sheet as an instrument for further use. The self-reflection tool is applicable for different knowledge levels and stages within the research process. Within a training unit, it can function as an information and awareness raising tool, introducing the different RRI concept dimensions and sensitise users to its meanings. It can also function as very practical tool, raising questions to be considered for implementation, as well as an evaluation tool to reflect on the own work or even as a basis for comparison and evaluation after a certain period of time.

Furthermore, the toolkit offered a development sheet to guide participants towards a practical implementation of RRI in their work and projects. This template is a structured step-by-step guideline considering important aspects towards achieving defined RRI goals. The sheet can be downloaded from the toolkit. Participants were provided the sheet as part of a training activity. They were able to work on the sheet immediately during the training or then later on. Ideally, the work on the development sheet would be part of an advanced training on RRI.

Finally, participants were given the opportunity to suggest further tools. They could be shared with the training group and later uploaded to the RRI-tools online toolkit as well.

2.4. Feedback and evaluation

All training was concluded with feedback gathered from either with a questionnaire or verbally. This feedback is summarised below:

The feedback gathered through questionnaires was rather positive. By and large, participants appreciated the training and felt that they had reached the goal of the workshop, that is, to deepen their understanding of RRI and to receive practical guidance on the implementation of RRI in their work. The training had succeeded in clarifying the goals of RRI for them and also in explaining the different aspects of RRI. Generally, participants stated that the trainings had an energising effect. The workshop had animated them so they were more actively engaged in the topic. They also learned about their own attitudes, besides being able to find out more about their own and other’s needs. Besides this, they felt encouraged to develop their own RRI strategy. More than half of them stated that the topic of RRI was relevant to them, although one third thought that the concept still was undeveloped and that there is still a need for a common definition of RRI.

As part of the exercise, we also collected the participants’ ‘likes’ and ‘wishes’. They were given a chance to talk about what they liked about the workshop and what they would wish for or what they felt was missing. All these aspects were noted by the moderators on flip charts. What they liked about the trainings was the composition of the group, the interactive nature of the training and the variety of workshop formats, which they also found entertaining. They appreciated the nice location and the atmosphere during the workshop. Finally, they were able to gain further insight, food for thought and much information for the purpose of implementing RRI in their future work. Most participants wished for RRI would to become a natural and sustainable practice. They also wished to integrate more stakeholders and to be able to dedicate sufficient time and effort to the topic.

3. Reflections and lessons learned

According to the experiences described previously and the feedback expressed by the participants, we reflected on the methodologies they applied and their findings. Based on these, we put together the following specific requirements regarding setting, methodology, content and participants, integral to the process.

3.1. Setting

Beforehand, it has to be clarified what exactly could be expected of the training. To avoid disappointment on the part of the participants, it is vital that clear goals are defined. Depending on their different knowledge and experience levels, these goals could vary from more basic information and awareness to concrete implementation planning of one’s own Research and Development practices. The formats suggested here are most appropriate for newcomers to those with slightly advanced levels. The activities suggested could easily be completed in subsequent trainings. However, one overriding training goal is to make it clear that besides concrete activities, it is the very spirit of RRI that needs to be implemented. RRI is not just a two-hour activity but a long-term undertaking.

Reflection and self-reflection processes are crucial for RRI implementation and should thus be integral elements of RRI trainings. A friendly atmosphere of good will and an open character to allow participants to get actively engaged with their pre-knowledge, experiences and concerns, is very favourable to this end.

Language therefore is of high importance. Participants need to find a common ground and understanding of an abstract concept. Therefore, clear and simple wording, avoiding subject particularities, which allow for a quick uptake and replication outside of the training setting, is crucial.

Last but not least, the trainings should consider adequate ‘space and beauty’ (Marschalek 2017), offering nice and friendly atmosphere in pleasant locations, visually appealing materials and an agreeable supply of refreshments which make participants feel good and appreciated.

3.2. Methodology

The suggested training activities involves aspects of inquiry-based learning by identifying and research issues and questions (Dostál 2015; Freire 1984) around RRI to bring forward their knowledge or approaches to issues and possible solutions. On the other hand, a peer-learning approach (O’Donnell and King 1999) was followed up, that stimulated the sharing of knowledge, ideas and experience between the participants on concrete practical examples by interacting with each other (Boud, Cohen, and Sampson 1988 ). By doing so, the participants were enabled to exploit the existing knowledge of the group, highlighted from different perspectives. Additionally, they offer scope for further learning and mutual exchange to also allow for creating new ideas. Still, the methodologies used need to be aligned with the individual preferences of those involved since many of them require openness, reflection and a willingness to share.

Besides this, a strong relation is needed between the very theoretical concept of RRI and its practical requirements and possibilities for implementation. To this end, methods and tools introduced during the activities should support participants so that they are able to immediately start planning their own RRI implementation.

3.3. Content

Besides providing some basic information to help understand RRI, different exercises to enable participants to better grasp the whole concept is crucial; slipping into other stakeholder roles helps towards an appreciation of different views. Discussing highlights and lowlights help them to be prepared for argumentation and sceptical positions against the RRI concept, which can occur often within one’s own working environment. Elaborating more on the advantages of RRI should support this argumentation. Therefore, showing very practical examples and working on actual cases are strongly advised.

In addition, clarification as to the collective efforts must be addressed not only in terms of resources, but also regarding the institutional and governance support. It should be noted that to emphasise the governance aspect, and the embedding of the RRI concept in the whole Research and Development system with its ongoing changes could be recognised as a paradigm shift (Owen 2014).

3.4. Participants

To work with multi-stakeholder groups is a good starting point to enable exchange and mutual understanding, enriching perspectives and possible collaborations with other stakeholder groups. It also offers a good opportunity to look at the range of different actors being addressed within the RRI concept. Having different stakeholders in one group supports the exchange between different groups already in training and allows for wider perspectives for collaborations in future implementations of RRI.

However, for participants to work on their own RRI implementation and relevant questions, further in-depth trainings or exchanges are needed. This gives direct guidance to the individual implementation of RRI. This could also be the starting point for ongoing mentoring programmes to accompany teams along their own pathway to RRI.

4. Challenges and consequences

Results show that trainings and applied methodologies address the needs of the stakeholders and help to understand the RRI concept. Furthermore, participants were encouraged to reflect on RRI within their own practice and to work on individually targeted strategies for implementation. They were able to exchange this information with others. Although the approach was applied and practical, they were able to reflect on a meta level as well.

From a trainer’s perspective, there are few challenges that need to be addressed more specifically. From an organisational point of view, experience has shown that the recruitment of the different groups needs particular attention. The same invitation is not suitable for all groups. There are different reasons of participation in such training sessions, which need to be addressed accordingly.

Also, language needs to be adapted within multi-stakeholder settings taking into consideration how RRI could be presented in a nutshell and made understandable for different stakeholder groups. To this end, the RRI tools trainings approach of applying fewer verbally active techniques proved to be suitable. The technique used of tandem interviews and fun aspects such as the different coloured hats contributed to a relaxed atmosphere and thus proved to be a more appropriate approach to overcome linguistic challenges.

At the same time, it was important that those techniques be applied which all stakeholders could accommodate and feel comfortable with. As a result, techniques were adapted to the needs of the individual groups according to their level of readiness. We do not consider the training to be ‘RRI-propaganda’, which is why we very consciously avoided preaching about RRI but rather focused on actively encouraging a lively discussion that would allow for constructive criticism of RRI or to use an analogy fill ‘an old wine in a new bottle’. This would enable us to identify concerns that might be raised.

A further challenge was to find and prepare inspiring examples with strengths and weaknesses to be discussed by the participants. The RRI tools library offers a wide range of cases, which could serve this purpose. As the moderators need to be very well informed on the case study and all stakeholders need to be able to relate to the topic, a selection of an appropriate one was difficult.

Trainings targeted at the implementation of RRI need a very personalised approach, which our trainings served well, but for a broader uptake and an institutionalised approach, other forms of activities might be considered. Our experience showed that one-day trainings could be a good start for such a process. Nevertheless, ongoing activities to support individuals and institutions to implement and improve their RRI practices are needed. For instance, a series of trainings within institutions could be offered to this end.

What was also challenging was that RRI is a long-lasting process that can potentially be expanded in several directions and formats. An evaluation of the success of the training or a monitoring process within participating organisations would require several years. It follows that, this aspect cannot always be fulfilled due to restrictions in time and resources, yet at the same time carries a lot of potential for further research including questions such as, how are individual strategies developed and implemented?, who is taking decisions on RRI implementations and how are key stakeholders embedded?

Developing a sound methodology for evaluation and monitoring the practical RRI implementations in all its different aspects could result in further insights. Meeting this challenge would, as a result, generate further recommendations used by other stakeholders for a successful implementation.

Acknowledgements

We would like to thank our consortium partners for the great work in RRI Tools project.

Notes on contributors

Ilse Marschalek is a sociologist with several years of experience in international studies in FP 5th, 6th and 7th of the EC. At ZSI she is project coordinator at the technology and knowledge department, carrying out a range of projects at the interface between technological and societal innovations. Recently she was the coordinator of the NanOpinion project (www.nanopinion.eu), a multichannel communication and dialogue project which aimed at discussing and assessing ethical and societal aspects of nanotechnologies, involving different stakeholder groups and the hard to reach public. Currently, she is engaged in the RRI tools (www.rri-tools.eu) project elaborating a self-reflection tool for active reflection processes on Responsible Research and Innovation practices.

Maria Schrammel studied Social and Cultural Anthropology at the University of Vienna. Her research interests include responsible research and innovation (RRI), public engagement in research and innovation, gender studies and migration and integration. Due to her educational background and her experience in actual and former projects, she specialised on interdisciplinary and transdisciplinary as well as qualitative social science research methods.

Elisabeth Unterfrauner holds a master degree in Psychology and a post-gradual degree in Clinical Psychology and Health Psychology and gained her PhD in interdisciplinary fields of pedagogy and social sciences. Unterfrauner has been a lecturer at the University of Vienna for several years in the psychology department and works as senior researcher at the Centre for Social Innovation in the Department of Technology and Knowledge.

Dr Margit Hofer is senior researcher at ZSI and has coordinated several FP6 and FP7 projects at ZSI and the European Schoolnet (EUN). She consults the Austrian Federal Ministry of Education and Women’s Affairs on the implementation of ICT, concepts of TEL and the use of social media tools for communication and outreach. She holds a PhD in Education and Training from the University of Graz and is specialised on user need analysis, evaluation, stakeholder engagement and the implementation and evaluation of Responsible Research and Innovation. Recent projects she has been involved in are the RRI Tools, SeeingNano and NanOpinion.

Additional information

Funding

This work was supported by the European Commission under FP 7 grant agreement no: 612393.

Notes

1. http://www.rri-tools.eu/

2. ZSI team was responsible for building up a hub community on RRI in Austria and Slovenia (more information about hubs can be found in chapter 1.1). Besides this, ZSI was responsible for all kinds of evaluation in the project and developed the self-reflection tool, which can be used in trainings and more (www.rri-tools.eu).

3. https://www.rri-tools.eu/-/kilmaalltag_insp_practice

4. https://rri-tools.eu/#

5. https://www.rri-tools.eu/self-reflection-tool