Experimentation, learning, and dialogue: an RRI-inspired approach to dual-use of concern

ABSTRACT

Responsible Research and Innovation is promoted by research funders and scientific communities as a way to place societal needs and values at the centre of research and innovation. In practice, however, legal compliance still tends to dominate the RRI agenda. In order to move beyond the dominance of legal compliance and address a broader societal agenda, this article argues that RRI requires: (1) a productive intertwining of research and practice; (2) the integration of anticipation, reflection, engagement, and action (AREA) in a non-linear process; and (3) an experimental approach. Based on this framework, this article draws on our experience of developing and institutionalizing an RRI-inspired approach to address dual-use and misuse issues in the EU-funded Human Brain Project. Our experience suggests that the four dimensions of the AREA framework work better not as separate stages but rather being flexibly intertwined to enable experimentation, learning, and dialogue.

KEYWORDS:

• Responsible Research and Innovation
• dual-use
• AREA framework
• experimentation
• learning
• dialogue

Introduction

Over the past decade, Responsible Research and Innovation (RRI) has been promoted by research funders and scientific communities as a way to put societal needs and values at the centre of research and innovation (Doezema et al. 2019; Frahm, Doezema, and Pfotenhauer 2021; Owen, von Schomberg, and Macnaghten 2021; Stilgoe, Owen, and Macnaghten 2013). However, experience shows that in practice, legal compliance still tends to dominate the RRI agenda (Stahl et al. 2021). This raises concerns about the implications of narrowing the scope of RRI and highlights a need to resituate it within a broader purpose that attends to the social, political, and economic context as well as implications of science and technology (Aicardi, Reinsborough, and Rose 2018; Salles and Farisco 2020; Stahl et al. 2021). This article examines the following questions: how can we mobilize RRI to address a broad range of societal questions and concerns beyond legal compliance? What does this entail, and what challenges and opportunities arise in the process? To address these questions, we examine our experience of developing an RRI-inspired approach to address dual-use and misuse issues in the EU-funded Human Brain Project (HBP). We argue that in order to move beyond the dominance of legal compliance and address a broader societal agenda, RRI requires: (1) a productive intertwining of research and practice; (2) the integration of anticipation, reflection, engagement, and action in a non-linear process; and (3) an experimental approach.

Launched in 2013, the HBP is one of the first big science projects to formally adopt RRI as a framework for addressing ethical and societal concerns (Stahl et al. 2021). Created to support and develop multidisciplinary Information and Communication Technologies (ICT) for neuroscience research, the HBP aims to achieve a more integrated understanding of the human brain, leading to both medical and technological applications (Amunts et al. 2016, 2019). Bringing together approximately 500 researchers and engineers based in more than 100 universities, research institutes, and hospitals in some 20 countries, the HBP is one of the biggest EU-funded research projects.¹ The project is developing a digital research infrastructure that gathers data and tools for brain research and provides a range of services such as brain simulations, neurorobotics and neuromorphic computing.

From the beginning, the HBP included a dedicated Ethics and Society ‘sub-project’ tasked with exploring the social, political, philosophical, and ethical foundations and implications of the HBP’s research, with the aim of shaping the project’s direction to serve the public interest.³ In order to do so, the Ethics & Society team developed a broad range of processes and mechanisms (Aicardi, Reinsborough, and Rose 2018; Aicardi et al. 2018b, 2020; Salles, Evers, and Farisco 2019; Salles et al. 2019; Stahl et al. 2019a, 2019b, 2021) informed by an RRI framework. This article focusses on dual-use and misuse issues as one area where the Ethics & Society team has developed and institutionalized an RRI-inspired approach that goes beyond legal compliance in order to consider a broader range of societal concerns. The EU Framework Programme adopts a narrow understanding of dual-use as research that can be used in both civilian and military domains, and its terms of funding require that all research has an exclusive focus on civil applications. The Ethics & Society team, however, used an RRI approach to develop a broader understanding of dual-use of concern that focuses on socially beneficial and harmful uses of research in areas such as politics, security, and intelligence in addition to military uses (Aicardi et al. 2018a).

This article draws on our experiences of developing and institutionalizing an RRI-inspired approach to dual-use and misuse issues in the HBP. The article is structured as follows. The first section discusses attributes of RRI which we see as crucial for going beyond legal compliance: the importance of intertwining research and practice; integration of anticipation, reflection, engagement, and action; and experimentation. The second section discusses the development of an RRI-inspired approach to dual-use and misuse issues in the HBP. The third section looks at the integration of anticipation, reflection, and engagement in RRI-framed activities. The fourth section considers the institutionalization of an RRI-inspired approach to dual-use. Finally, drawing on our experiences, we provide some insights on how to move beyond compliance to address a broader range of societal aspects.

Challenging the standardization of RRI

RRI is a multiply defined concept and set of practices, consistently being rethought amidst changing social, economic, and political contexts and scientific and technological developments (Doezema et al. 2019; Fisher 2020; Owen, von Schomberg, and Macnaghten 2021). Even within the HBP, RRI is conceived of and implemented in multiple ways (Stahl et al. 2021). We maintain that the lack of specificity in defining RRI is in itself a strength – as a set of loosely held together orientations and practices, it allows for the fluidity needed to work across a range of scientific and technological domains while remaining coherent enough to attract and consolidate people and resources. It is important to retain the fluidity and multiplicity of RRI and challenge attempts to standardize and operationalize its concepts and practices via toolkits and indicators (Owen, von Schomberg, and Macnaghten 2021). In this section, we highlight the following closely related aspects of RRI that we consider to be key in moving beyond the dominance of legal compliance in order to address broader societal issues at the project level: (1) the importance of intertwining research and practice; (2) the integrated dimensions of anticipation, reflection, engagement and action (AREA); and (3) the need for a collective experimental approach.

Intertwining research and practice

The HBP Ethics & Society team’s approach aligns with the view that the implementation of RRI in the neurosciences requires not just normative discussion but also conceptual analysis and a deep empirical knowledge of the research, clinical, cultural, and social contexts to be intervened in (Gardner and Williams 2015). Indeed, since its inception, the Ethics & Society team has used historical research and methodologies, ethnographic research, and in-depth interviews to engage with the HBP research communities while developing background reports for its joint publications. Empirical grounding and theoretical reflection have formed the basis of engagements with researchers within and outside the project to encourage reflexivity about the social, political, and economic contexts and implications of their research, and have been foundational to implementing a number of the recommendations that we later discuss.

The mechanisms proposed to intertwine the Ethics & Society team’s research and practice have included: (1) developing joint research documents known as ‘Opinions’ on relevant issues, in collaboration with internal and external stakeholders; (2) developing concrete recommendations to the HBP governing bodies and others; and (3) following-up on these recommendations in collaboration with relevant stakeholders within and beyond the project. To facilitate RRI practices and implementation within the HBP, the Ethics & Society team developed and put into effect additional processes and structures such as the Ethics Rapporteur Programme and the Ethics Advisory Board (Aicardi, Reinsborough, and Rose 2018; Rainey et al. 2019; Stahl et al. 2019a, 2019b, 2021) that are particularly relevant to our work on dual-use (for more information, see the section on institutionalizing an RRI-inspired approach).

Integrating anticipation, reflection, engagement and action

The work of Stilgoe, Owen, and Macnaghten (2013) led to the development of the AREA framework for responsible innovation (anticipation, reflection, engagement, and action)⁴ which was subsequently endorsed and adopted by the HBP’s Ethics & Society team. While the European Commission’s RRI framework centres on six keys – public engagement, gender, open access, ethical issues, education, and governance (Owen, von Schomberg, and Macnaghten 2021) – these represent isolated themes rather than a coherent approach. By contrast, the AREA framework offers a broader, integrated vision rooted in the social study of science and technology (Owen and Pansera 2019). The emphasis on the need for public engagement, researcher awareness, and a conceptual analysis of HBP research in the HBP’s first proposal to the EC in 2012 ‘lent itself especially well, a posteriori, to the broad AREA framework for RRI proposed by the EPSRC in the UK, rather than mirroring the “six keys” of the EC framework’ (Aicardi, Reinsborough, and Rose 2018, 27–28). The HBP was, therefore, one of the first large-scale science and technology projects to formally adopt an RRI approach in the early stages of its design and development and the first to adopt the AREA framework to specifically address ethical and societal issues within international neuroscience research.

Moving from anticipation, reflection, and engagement to action – ‘closing the AREA loop’ – and ensuring that the recommendations made by interdisciplinary groups such as the HBP’s Ethics & Society team influence the governance of the projects they are embedded in, is particularly challenging. Initially, the Ethics & Society team organized its activities to reflect the AREA framework: anticipation of the potential future impact of new knowledge and technologies; reflection on the conceptual, social, ethical, and regulatory implications of neuroscientific research and emerging neurotechnologies; engagement and dialogue with diverse stakeholders on issues of immediate relevance to the HBP; and action based on insights gained through anticipation, reflection and engagement. These research-based activities were carried out by four work packages: the Foresight Lab; Neuroethics and Philosophy; Engagement; and Ethics Support. As we demonstrate in further sections, Anticipation, Reflection, Engagement, and Action have not been separate processes but rather enmeshed in non-linear ways within the ongoing collaborations between the Ethics & Society team members, and the scientists and engineers in the HBP. The Ethics & Society sub-project is now distributed between a dedicated RRI work package and a number of philosophy, neuro-ethics and RRI-related tasks embedded across different HBP work packages.

Collective experimentation

Several authors have highlighted the importance of experimentation with and within RRI, emphasizing its open-ended and inventive nature as well as its productive potential to generate unexpected questions and insights (Delgado and Åm 2018; Stilgoe 2016; Stilgoe, Owen, and Macnaghten 2013). Ongoing experiments with and within RRI are particularly important due to the highly uncertain nature of research and innovation and the challenges involved in foreseeing their future developments and impacts, especially during the early stages of research. This is known as the ‘dilemma of control’, namely, at early stages of technology development it is difficult to predict its social consequences but by the time undesirable consequences are discovered, technology is already so deeply embedded in society that change is extremely difficult (Collingridge 1980). Sheila Jasanoff suggests the concept of ‘technologies of humility’ to describe the methods or institutionalized habits of thought that try to come to grips with the unknown, the uncertain, the ambiguous and the uncontrollable (Jasanoff 2003). According to Jasanoff, technologies of humility acknowledge the limits of prediction and control, the possibility of unforeseen consequences as well as the need for plural viewpoints and collective learning. Due to the high uncertainty of emerging technologies, it has been suggested that they need a ‘tentative governance’ that is more flexible and provides spaces for probing (Kuhlmann, Stegmaier, and Konrad 2019). In this context, collective experimentation (Stilgoe 2016) can be seen as an element of governance that provides space for the contestation and negotiation of emerging technologies early on. Experimentation goes hand in hand with approaching RRI through the lens of collective improvisation (Sauer and Bonelli 2020) and a mutual learning process that is reflective and dialogue-based (Egeland, Forsberg, and Maximova-Mentzoni 2019).

Closely related is the dialogical approach developed by some of the Ethics & Society team members (Stahl et al. 2019a). Building on discourse ethics and RRI, this approach suggests an open, inclusive, and responsive dialogue as a way to identify and address societal issues. It recognizes that in the case of novel technologies, these issues cannot be known a priori – ongoing and continuous dialogue and reflection is needed. Thus, instead of a box-ticking mentality, a dialogical approach aims to open up and critically scrutinize the societal and ethical aspects of research. Common features of approaching RRI as a collective experimentation and improvisation, mutual learning and dialogue are openness, exploration, and flexibility which are crucial considering the uncertainty of research and technology. Moreover, an experimental, learning-based, and dialogical approach to RRI challenges assumptions about RRI as a standardized method, tool, or recipe (Egeland, Forsberg, and Maximova-Mentzoni 2019).

To sum up, we suggest that in order to address a broader societal agenda, RRI should not be perceived simply as a linear process where research and engagement are followed by action. These are all continuous, overlapping, and interlinked processes that resist the structure, formulism, and standardization of RRI according to a set of keys, indicators, and measurements. Thus, it is fruitful to conceive RRI and its AREA framework as sources of inspiration for developing and advancing novel approaches to addressing societal issues in research and innovation rather than recipes, checklists, and toolboxes to be followed. Here we agree with Owen and colleagues about

the need to resist attempts to reify RRI as a set of disparate keys, including the institutional impulses (well-intentioned though these may be) driving such attempts in the guise of making RRI pragmatic, actionable and measurable. In part, we have argued instead the need to return to and regain some of the substance of the original visions made by RRI’s early protagonists. (Owen, von Schomberg, and Macnaghten 2021, 227)

We now turn to the development and institutionalization of an RRI-inspired approach to dual-use issues in the HBP where the above-mentioned aspects of experimentation, an integrated AREA framework, and the interlinking of research and practice have played a crucial role.

Developing an RRI-inspired approach to dual-use

While in the process of developing the Ethics & Society proposal for the HBP, members began working on a ‘map’ of ethical and societal issues raised by its proposed research (Stahl et al. 2019b). The map included a number of topics, such as clinical translation, mental and brain-related disorders, consciousness, community-building, data governance, research integrity, open science, artificial intelligence, and dual-use. Given the impossibility of working effectively on multiple topics simultaneously, a decision was made during the first phase of the HBP to focus on four or five main topics for joint research. Priority was given to those that, after extensive discussions within the HBP and with the Ethics Advisory Board, were selected as being most immediately relevant for HBP research. Dual-use and data protection were identified as highly relevant to HBP research while also addressing the EU Framework Programme’s ethics issues (e.g. European Parliament and the Council of the EU 2013). Thus, the first Ethics & Society sub-project’s Opinion on Data Protection and Privacy was followed by the second one on Dual-Use.

Dual-use and misuse are among the key societal and ethical issues raised by the neurosciences (Butorac, Lentzos, and Aicardi 2021; Giordano and Evers 2018; Global Neuroethics Summit Delegates et al. 2018; Ienca, Jotterand, and Elger 2018; Mahfoud et al. 2018; Nuffield Council on Bioethics 2013; OECD 2019; The Royal Society 2012; Tennison and Moreno 2012; Ulnicane 2020; Voarino 2014; Whitby and Dando 2019). More specific dual-use concerns raised by the neurosciences include:

brain inspired neuro- and ICT technologies that are already in use or in advanced stages of development, for example, in warfighter ‘enhancement’, intelligence gathering, image analysis, threat detection, manipulation of emotional states, incapacitation of adversaries, and the development of autonomous or semi-autonomous weapons, or weaponized robots using artificial intelligence technologies and machine learning algorithms for target detection an elimination. (Aicardi et al. 2018a, 5–6)

The concept of dual-use has many definitions and connotations which have changed over time and which vary across different fields (Aicardi et al. 2018a). Traditionally, research and technology have been considered to be dual-use when they have current or potential civilian and military applications (see, e.g. Molas-Gallart 1997). This definition of dual-use focusing on the civil–military dichotomy is still used in the EU Framework Programme (European Commission 2019). However, dual-use experts have recognized the limitations of the traditional civil–military dichotomy and highlighted the need to understand dual-use research and technology more broadly as having beneficial as well as harmful purposes (see e.g. Ienca, Jotterand, and Elger 2018; Kavouras and Charitidis 2019; Oltmann 2015; Ulnicane 2020) in a broad range of domains such as political, security, military, and intelligence (Giordano and Evers 2018; Mahfoud et al. 2018).

While in some countries, such as the US, brain research receives funding from defence agencies,⁵ all research activities funded by the EU Framework Programmes (FP), including the HBP, must comply with the FP regulation and ‘have an exclusive focus on civil applications’ (European Parliament and the Council of the EU 2013). In the Ethics & Society team, this focus was seen as too limited to address concerns about emerging brain research, therefore < in 2015 we started to work on an Opinion to develop a broader, RRI-inspired approach to dual-use. RRI, and more specifically the three aspects we outline above (experimentation, integrated AREA framework, and intertwined research and practice), have played a key role in the conceptual advancement of dual-use as well as in the process of the development of an approach to tackling dual-use at the project level.

The HBP Ethics and Society approach to addressing dual-use is encapsulated in the title: ‘Opinion on Responsible Dual-Use: Political, Security, Intelligence and Military Research of Concern in Neuroscience and Neurotechnology’. According to the Opinion, dual-use of concern refers to:

neuroscience research and technological innovations, and brain inspired developments in information and communication technologies, for use in the political, security, intelligence and military domains, which are either directly of concern because of their potential for use in ways that threaten the peace, health, safety, security and well-being of citizens, or are undertaken without responsible regard to such potential uses. (Aicardi et al. 2018a, 5)

The Opinion recognizes that the identification of research of concern is far from straightforward and will remain a matter of debate. It also points to the crucial role that RRI and the AREA framework play in enabling the identification of dual-use of concern, building capacity for debate, and engaging researchers and other stakeholders. In line with much RRI literature (see e.g. Stilgoe, Owen, and Macnaghten 2013), the Opinion understands responsibility to be based in collective and institutionally governed processes and practices, rather than as the responsibility of any individual researcher. While RRI has been mentioned in the context of dual-use before (e.g. Rychnovská 2016),⁶ the Opinion places the RRI framework at the centre of the HBP approach to dual-use.

To advance this approach to dual-use, the Opinion ends with a number of recommendations for the HBP, EU policymakers, and other stakeholders. These recommendations include suggestions to establish processes to continuously scrutinize brain research from the perspective of dual-use of concern and to develop educational activities on dual-use issues. To follow-up on these recommendations, the HBP governing bodies decided to establish a project-wide working group (see below section on institutionalizing an RRI-inspired approach to dual-use).

Adapting the RRI framework to dual-use issues in the Opinion enables us to address dual-use issues at the project level via engagement with a broad range of stakeholders within and outside of the project. Moreover, it allows for the anticipation of and reflection on potential uses and implications at the early stages of research. In combining ‘dual-use research of concern’ and RRI, our work goes beyond existing approaches to dual-use which largely focus on macro-level policies and regulation of technologies. While those approaches sometimes recognize the need for ethics guidelines, awareness raising, and education, they tend to pay little if any attention to broader engagements with researchers and stakeholders at the laboratory and project level to the extent the HBP has.

Between anticipation, reflection, and engagement

The development of the dual-use opinion entailed undertaking a broad range of activities – from an analysis of the concept of dual-use to the organization of workshops and webinars with HBP researchers, dual-use experts, and policymakers; to citizen consultation and educational activities. These research and engagement activities resulted in reports which analysed existing understandings of the notion of dual-use, critically examined relevant policies and regulations, identified dual-use issues in neuroscience and computing, and articulated HBP-relevant issues as well as expert and citizen views (Aicardi et al. 2018a).

The process of preparing the dual-use opinion was informed by the RRI and AREA frameworks. The range of activities undertaken illustrates how anticipation, reflection, engagement, and action can inform research and practice on a specific topic. These activities have been well received by dual-use experts who, for example, saw a webinar organized by the HBP as ‘a first promising step in the direction of awareness-enhancing strategies’ (Ienca, Jotterand, and Elger 2018, 273).

To illustrate the activities that shaped the approach developed in the Opinion, we provide insights into two dual-use workshops that contributed to the way the Opinion was developed, what case studies it focused on, and which recommendations were made. These workshops demonstrate how anticipation, reflection, engagement, and action were intertwined throughout the process of working on the Opinion and in engagement with the scientists and engineers. While bureaucratically these processes were divided along the AREA framework in order to allocate resources across different partner institutions within HBP Ethics & Society team, in practice, they were much more integrated, and necessarily so in order to develop a project-based understanding of dual-use and RRI issues.

While some scientists and engineers are eager to reflect on ethical and societal aspects of their work, it can be challenging to encourage others to do so – particularly when they are under increasing time pressures and when there aren’t any institutionalized mechanisms that support and reward discussions of ‘ethics and society’ issues (Aicardi, Reinsborough, and Rose 2018). Reflection is an important dimension in RRI, and in the AREA framework specifically. RRI can be achieved in part through institutional reflexivity ‘in which the value systems and theories that shape science, innovation and their governance are themselves scrutinised’ (Stilgoe, Owen, and Macnaghten 2013, 1571). This ‘role’ – the ‘reflexivity inducer’ – has often been assigned to social scientists working within post-ELSI frameworks (Balmer et al. 2015). Embedding social scientists in the laboratory, and using ethnography as a method, can enable laboratory work to become more sensitive to its potential societal implications (Fisher 2007).⁷ Working in this spirit, the HBP Ethics & Society sub-project developed its Researcher Awareness programme to engage HBP researchers with the Ethics & Society team’s research process – not just as the ‘recipients’ of our research results. While researcher awareness activities have covered a range of topics and mechanisms (see e.g. Aicardi et al. 2020), here we focus on two dual-use workshops that took place in 2017 and 2018 to illustrate in detail one particularly productive RRI approach that we developed.

Rather than instituting a project-wide researcher awareness programme, we identified laboratories whose research was more likely to raise dual-use of concern issues and worked closely with members of those labs to organize informative and exploratory workshops. The aim of the workshops was to inform the researchers about the research conducted by the Ethics & Society team on dual-use issues of concern, and to encourage reflection on the social, political, and economic implications of the research they were conducting and the technologies they were developing. We made a few methodological choices based on previous and ongoing ethnographic engagement with various labs participating in HBP research: (1) to work closely with PhD students and early career researchers since they are the members of the lab conducting the majority of the research work, (2) develop the content of the workshops in close collaboration with members of the labs so as to encourage ownership of the conversation, (3) limit the number of participants to around 20, and (4) ensure that the content of the discussion that took place during the workshop would be confidential, and would not be shared outside of the meeting through reports or publications.

The workshops followed a similar format. The first session included short presentations by the Ethics & Society team’s members that gave more background to the draft Opinion which was pre-circulated to workshop participants. In the second and third sessions, we discussed scenarios which had been prepared in advance in collaboration with the lab members. While scenarios are a common methodological tool used in RRI (Aicardi, Reinsborough, and Rose 2018), it was actually the scientists we collaborated with who suggested we use science-fiction-like scenarios as a basis for our discussion. The scenarios were designed to be hypothetical so that they would not directly reference their own research. But we made sure to develop scenarios that were inspired by their research as well as historical knowledge and current events relating to developments in the life sciences and engineering. This made the scenarios ‘strange’ enough to feel distant from them to critically engage, but familiar enough to have a provocative reaction. This framing also allowed for the scenarios to be used as an opportunity to reflect on how past and current events can be used to understand and intervene in thinking about and planning for the implications of their research. These scenarios served as frameworks and stimuli for evaluating the possible consequences and the social and ethical implications the technologies and research may have. In this way, these workshops helped to build anticipation and reflection capacities that are necessary for RRI.

The workshop discussion sessions have been generative. They have led to changes in how involved teams report on the potential implications of their work in their Ethics Rapporteur self-assessment of ethical and societal issues, and have pushed the ‘ethics and society’ discussion beyond mandatory compliance and research ethics, towards a much broader discussion of how their research is both shaped by the political, social and economic contexts they work within, and how their research contributes to shaping these, in turn. Some of the topics discussed at these workshops also informed the work of the Dual-Use Working Group – for example, the concern many researchers had about the tensions between developing open-access research infrastructure and security concerns about how that research would be used and by whom. The workshops also led to a list of recommendations for individual labs to formalize a yearly discussion with all members of the team to discuss ethical concerns people have, and what they can do about them, and to include a course or session on RRI in Masters and PhD curricula, and in early career researcher training.

Furthermore, we have worked closely with the HBP Education programme to provide training on dual-use and RRI. While laboratory-based workshops have been a successful format, they are very time-consuming to prepare for and conduct, and cannot be used to reach all of the 100+ participating labs in the HBP. The Ethics & Society team members have, therefore, offered lectures and workshops at student-run conferences, and have recorded online lectures that are available to all HBP researchers. We have found that without a deep empirical knowledge of the research in the HBP, it would have been impossible to identify what is relevant for researchers to learn in order to better reflect on the implications of their research. While some RRI work has focused on developing standardized toolkits, a toolkit cannot address the variety of interconnected issues that arise in interdisciplinary projects (Owen and Pansera 2019). In the HBP, we have developed general as well as more targeted approaches, working closely with laboratories on particular problems related to their work instead of directly applying general RRI and dual-use toolkits which would not be as relevant to the particular needs of HBP researchers.

Institutionalizing an RRI-inspired approach to dual-use

A key milestone in the institutionalization of a broader RRI-inspired approach to dual-use of concern in the HBP was the establishment of a project-wide Dual-Use Working Group (DUWG) in December 2018. This group was established by one of the main HBP governing bodies (the Directorate) which adopted the mandate for the DUWG that defines its purpose, composition, expected activities, and resources. The DUWG was established to develop and implement follow-up activities to the Dual-Use Opinion and to ensure that the HBP acts responsibly with regard to the potential dual-use issues of concern raised by and through its research and innovation activities. The DUWG brings together representatives from all parts of the project to scrutinize HBP activities in relation to potential dual-use of concern, to prepare for the EBRAINS infrastructure, and to plan and execute other necessary activities such as raising awareness and education.

Building on and expanding the HBP RRI infrastructure

The DUWG was established approximately half-way through the 10-year project, which has allowed it to draw on already established structures and processes for addressing ethical and societal issues in the HBP. One of these is the already mentioned Ethics Rapporteur Programme which is an important resource in helping us to identify and address ethical and societal issues arising from HBP research. Each sub-project (now work package) appoints one or more Ethics Rapporteur from among their researchers, engineers, or managers. Ethics Rapporteurs communicate regularly with representatives of the Ethics & Society team and with members of the independent Ethics Advisory Board to deepen our understanding of existing and emerging ethical and societal issues in their respective sub-projects. In particular, they are tasked with preparing annual reports (‘one-pagers’) where they outline issues which are then discussed during annual trilateral meetings that bring together Ethics Rapporteurs, sub-project leaders and managers with representatives from the Ethics & Society team and the Ethics Advisory Board. These meetings illustrate the dialogical approach introduced above (Stahl et al. 2019a).

Already in the Opinion and the DUWG mandate, the Ethics Rapporteurs have been assigned a key role in evaluating dual-use of concern issues in their respective work packages. The majority of DUWG members are the Ethics Rapporteurs. Building on the existing RRI infrastructure in the HBP, the evaluation of dual-use of concern has been incorporated in the annual process of the preparation of Ethics Rapporteurs one-pagers and trilateral meeting discussions.

The DUWG has virtual meetings once every two months. These meetings are usually attended by around 15 participants and typically last for an hour. When needed, they are followed up by individual meetings to discuss issues related to specific work packages. In addition, to reach out to other stakeholders within and beyond the HBP, the group and its representatives organize sessions at the annual HBP summit which are open to all HBP members; workshops at the HBP student conferences; early career researcher events; as well as internal and external webinars and talks.

While being able to draw on existing RRI structures and processes in the HBP for addressing dual-use issues is beneficial, it also often requires challenging entrenched understandings and perceptions about ethical and societal aspects of science and technology, which still often focus on legal compliance. In conversations with researchers, it becomes obvious that their primary ethical concerns often focus on getting ethics approvals, which in their experience are becoming more complex and demanding, due to the requirements of the EU General Data Protection Directive. Moreover, in a large-scale EU project like the HBP, scientists and managers often associate ethics with mandatory regular reporting requirements, which again largely focus on legal compliance. In this context, an invitation to think about broader potential societal concerns is not a straightforward task.

Experimentation, learning, and dialogue

Accordingly, one of the first steps to start a conversation about broader potential concerns and misuses of research is a range of awareness-raising activities. This requires experimentation with various approaches to engage researchers to consider potential concerns and misuses of their own research early on, for example, discussing how RRI and the AREA framework’s dimensions can help here. Whenever we run workshops at student and early career-researcher focused conferences in the field of brain science within and beyond the HBP, we often encounter participants who are highly interested in learning more about the societal and ethical aspects of their research. But they also tell us that they have had little if any training or education on these issues, and some of them also admit that they have never considered the potential misuse of their research. These training and awareness-raising activities aim to change the culture of research by sensitizing researchers to societal concerns and the potential misuse of their research. They are also intended to support and prepare researchers for one of the key tasks of the DUWG, namely, the identification of dual-use of concern and misuse issues in their work.

As mentioned earlier, an approach the DUWG has chosen to pursue the identification of dual-use of concern and misuse issues is to include these issues in the short self-assessment (one-pagers) of the ethical issues prepared by Ethics Rapporteurs for each work package. These one-pagers are internal documents intended for discussions and follow-up actions within the project. In the preparation of the dual-use of concern and misuse part of the one-pagers, the DUWG has collaboratively prepared a living document with key definitions and questions to support anticipation and reflection on these issues. Ethics Rapporteurs are invited to share their potential concerns at the DUWG meetings, allowing them to exchange their reflections, comment on each other’s approaches, and learn from each other. It is intended that the DUWG serves as a safe space where any potential concerns can be shared for initial discussion and potential follow-up actions.

To identify dual-use issues in their work packages, Ethics Rapporteurs are encouraged to experiment with different methods of anticipation and reflection, and to afterwards consider the advantages, disadvantages, and results of these different methods. It is suggested that when preparing their one-pagers, Ethics Rapporteurs follow a deliberative approach of discussing any issues with their colleagues in their respective work packages. However, each rapporteur has their own approach for doing that. Some of them undertake a survey in their work package to identify ethical issues, others consult with colleagues at meetings or over email, while some might have to do most of the reflection themselves. This is followed up by discussion, for example, on whether surveys are a useful tool for facilitating reflection on dual-use of concern issues, which are sometimes misunderstood as the civil–military dichotomy. This can lead to predictable survey answers that there are no dual-use issues in the respective research.

As pointed out in the Opinion, the identification of potential dual-use of concern and misuse issues is far from straightforward. Moreover, going beyond legal compliance and immediate risk assessment often means that potential concerns and misuses are vaguer and less concrete, requiring expanded vocabulary and practices of anticipatory governance to address these more uncertain issues (Rychnovská 2021).

A lot of research in the HBP is in its early stages when it is still difficult to predict potential uses and related concerns. This can lead to some standard answers that any possibility of dual-use of concern is very remote and highly unlikely. This is in line with the findings of Isobel Butorac and colleagues that researchers find it difficult to predict long-term effects and risks, and to be what they perceive as ‘fortune tellers’ (Butorac, Lentzos, and Aicardi 2021). On the other hand, researchers also point out that anything can be misused, and it is difficult to avoid that. Hesitation to talk about potential dual-use of concern and misuse might also stem from concerns about how it could influence the future of their specific research.

In this context, it is crucial to explain and follow our approach that in emerging sciences and technologies such as brain research, ethical issues, concerns, and misuses often cannot be identified upfront and require continuous reflection and dialogue as research develops (Stahl et al. 2019a). Our dialogical approach also involves resisting requests for checklists and reminding researchers and managers that the identification of potential concerns and misuses is a dialogue and not a checklist. According to this dialogical approach, the DUWG is intended to serve as a network that supports and facilitates a process of mutual learning and experimentation. At the core of this network is the DUWG, but it is an open and collaborative formation that regularly interacts with others within and beyond the HBP including other HBP groups such as the Data Governance Working Group and HBP student initiatives as well as policymakers, ethics and dual-use experts, and social scientists interested in the governance of technologies. Opportunities to draw on this DUWG network for discussing and sharing any concerns and ways of addressing them are appreciated by HBP colleagues and students. Richard Owen and colleagues have suggested that community building and socializing might be some of RRI’s most enduring legacies (Owen, von Schomberg, and Macnaghten 2021). The effects of community building and socializing are also strongly present in the DUWG work of bringing together researchers for experimentation and mutual learning.

To summarize, the institutionalization of an RRI-inspired approach to dual-use of concern in the second half of the HBP has benefited from RRI processes and structures established in the first half. However, this also came with some challenges. In addition to the need to challenge entrenched, narrow understandings of ethics, it was also necessary to consider that a systematic introduction of dual-use issues should not ‘overload’ existing structures and processes. Ethics Rapporteurs, who typically are researchers or managers and can only devote a limited share of their time to their Ethics Rapporteur activities, already have a lot of responsibilities. Similarly, their one-pagers and trilateral meetings already have a range of ethical issues on the agenda. Therefore, it was decided to take a ‘light’ approach, for example, to meet only once every two months rather than more often. This can also limit how deeply it is possible to engage with experimentation and reflection. However, institutionalizing dual-use of concern within an existing RRI infrastructure has allowed us to develop it further by expanding the range of issues discussed and the activities undertaken as well as experimenting with more ambitious approaches to address broader societal issues that encourage reflection on societal concerns and potential misuses that might emerge in a longer-term and various ways of addressing them.

Conclusions

In this article, we have explored our experience of developing and institutionalizing an RRI-inspired approach to dual-use that goes beyond legal compliance and that considers a broader range of societal concerns. We have argued that it is essential to maintain RRI as a flexible, fluid, and open concept and challenge attempts to standardize and instrumentalise it. We highlight the importance of three closely connected aspects of RRI: (1) a productive intertwining of research and practice; (2) the integration of anticipation, reflection, engagement, and action in a non-linear process; and (3) an experimental approach. Our experience suggests that the AREA framework works better when its four dimensions are used in a flexible and intertwined manner that enables experimentation, learning, and dialogue rather than when trying to artificially separate them into four distinct stages.

Developing and institutionalizing an RRI-inspired approach to dual-use has required extensive interdisciplinary research on topics such as the governance of technologies, neuro-ethics, and scientific collaboration drawing on Science and Technology Studies, philosophy, and other disciplines. This research has played an important role in anticipation, reflection, and education activities and has supported experimentation and mutual learning. Research and practice in our approach to RRI continue to mutually shape each other: ideas and insights from research influence practices, while practices shape future research questions and focus. Rather than reducing RRI to standardized practices, the combination of research and practice allows us to enrich and deepen the potential of RRI to tackle a broader set of societal issues. However, the success of RRI in addressing societal issues by increasing understanding and changing culture and practices remains difficult to measure (Stahl et al. 2021).

In our experience, RRI works best when it is neither conceived nor followed as a set of principles. While the HBP Ethics and Society sub-project has adopted the AREA framework, in practice our work does not so neatly fit into the separate ‘boxes’ of anticipation, reflection, engagement, and action. In fact, we suggest that the AREA framework should not be interpreted in a linear fashion but should rather demonstrate the messiness of research and innovation itself. Because of the fluidity and flexibility of the RRI framework, the HBP Ethics & Society sub-project has been able to interpret RRI in ways that work for the specific context of the HBP. Because RRI does not give priority to some principles or values over others (Salles and Farisco 2020), we have been able to work closely with researchers to identify particular areas of concern and reflect on possible solutions for addressing them. We have focused on particular concerns that arise across the intersection between neuroscience, computing, and medicine without the ease of relying on already existing dual-use or RRI toolkits. While this has made our reflection and practice more demanding, it has also been generative and productive.

Researchers who have studied RRI in other contexts have observed a reluctance to address dual-use issues because they are sometimes perceived as controversial (Frahm, Doezema, and Pfotenhauer 2021). The development of an RRI-inspired approach to dual-use in the HBP encountered similar reactions and questions about the relevance of the topic, which have also provided opportunities for discussions and awareness raising about dual-use of concern issues. This work on developing and institutionalizing an RRI-inspired approach to dual-use has attracted interest within and beyond the project. It has been highlighted as one of the scientific achievements of the HBP (Amunts 2020) and recognized by experts as an important step forward (Garden et al. 2019; Ienca, Jotterand, and Elger 2018; NIH 2019). However, it remains a work in progress that requires a continuous challenge to standardized understandings of RRI and dual-use in order to open space for a more reflective approach to potential concerns that arise from emerging research and technology.

Acknowledgements

We are grateful to reviewers and editors for their valuable feedback and support. We would like to thank Kathinka Evers, Nikolas Rose, Christine Aicardi and Bernd Carsten Stahl for their helpful comments and suggestions on an earlier version of this article. This article has benefited from collaborations and conversations with many of our HBP colleagues over the years. Presentation at the 2021 virtual IPSA World Congress of Political Science helped to shape some of the ideas outlined in this article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The research reported in this article has received funding from the EU’s Horizon 2020 Research and Innovation Programme under Grant Agreements No. 720270 (HBP SGA1), No.785907 (HBP SGA2) and 945539 (HBP SGA3).

Notes on contributors

Inga Ulnicane

Inga Ulnicane has extensive international and interdisciplinary experience of research, teaching and engagement in the field of science, technology and innovation governance. Her scientific publications and commissioned reports focus on topics such as European research and innovation policies, international research collaboration, Grand societal challenges, Artificial Intelligence, and dual-use. She has worked at the University of Vienna (Austria), University of Twente (Netherlands), University of Latvia and Latvian Academy of Sciences, and has been visiting scientist at the University of Manchester (UK) and Georgia Institute of Technology (US). Currently, she is at De Montfort University (UK).

Tara Mahfoud

Tara Mahfoud is a Lecturer (Assistant Professor) in the Department of Sociology at the University of Essex. Before that, she was a Research Associate in the Department of Global Health and Social Medicine at King’s College London and a member of the Human Brain Project Foresight Lab. Her work explores the social and political contexts and implications of developments in the neurosciences.

Arleen Salles

Arleen Salles is a Senior Researcher in philosophy at the Center for Research Ethics and Bioethics (CRB) at Uppsala University (Sweden). She is the Deputy Leader of the Responsible Research and Innovation Work Package of the EU-flagship Human Brain Project where she is a researcher and leads the task Neuroethics and Engagement. She is also Board member of the International Neuroethics Society, Board member of the Institute of Neuroethics (IoNx) and serves as a member of the International Brain Initiative Neuroethics Working Group. Her current research focuses on neuroethics, particularly the normative, epistemic, and ontological implications of neuroscientific findings and its applications (including brain-inspired artificial intelligence).

Notes

1 The Project receives approximately 400 million Euros from the EU Framework Programmes (FP7 and Horizon 2020) over a 10-year timeframe (2013–2023), and between 4% and 5% of the HBP budget has been devoted to Ethics & Society research.

3 The HBP, now in its last phase, has been restructured from 12 sub-projects into 9 work packages.

4 https://epsrc.ukri.org/research/framework/area/.

5 https://www.darpa.mil/program/our-research/darpa-and-the-brain-initiative.

6 Additionally, the term ‘Responsible Dual Use’ has been used before but without linking it to RRI (e.g. Forge 2013).

7 The HBP in its final three-year period has, in a similar spirit, embedded philosophy and neuroethics tasks within all the science work packages, in addition to a work package devoted to RRI specifically.