Limiting open science? Three approaches to bottom-up governance of dual-use research of concern

ABSTRACT

Governing dual-use research of concern (DURC) in the life sciences has become difficult owing to the diversification of scientific domains, digitalization of potential threats, and the proliferation of actors. This paper proposes three approaches to realize bottom-up governance of DURC from laboratory operation to institutional decision-making levels. First, a technological approach can predict and monitor the dual-use nature of the research target pathogens and their information. Second, an interactive approach is proposed in which diverse stakeholders proactively discuss and examine dual-use issues through research practice. Third, a personnel approach can identify the right persons involved in DURC. These approaches suggest that, going beyond self-governance by researchers, collaborative and networked governance involving diverse actors should become essential. This mode of governance can also be seen in light of the management of research use. Therefore, program design by funding agencies and publication screening by journal publishers continuously contribute to governance at the meso-level. Bottom-up governance may be realized by using an appropriately integrated design of these three approaches at the micro-level, such as dual-use prediction and monitoring, stakeholder dialogue, and background checks. Given that the term ‘open science’ has been promoted to the research community as part of top-down governance, paying due attention on site to research subjects, research practices, and persons involved in research will provide an opportunity to develop a more socially conscious open science.

KEYWORDS:

• Biosecurity
• enhanced potentially pandemic pathogens (ePPPs)
• preprint screening
• genomic data management
• gain-of-function (GOF)
• responsible research and innovation (RRI)

Introduction

Dual-use research of concern (DURC) refers to research that is intended to provide a clear benefit but whose misapplication could negatively impact health and safety, agriculture, the environment, or national security. DURC is typically used to refer to gain-of-function (GOF) research in the life sciences, a field that has recently been subject to diversification of scientific domains, digitalization of potential threats, and a proliferation of actors, thereby making governance more difficult than ever [1]. In addition, with the growing threat from enhanced potential pandemic pathogens (ePPPs), such as host modification of the highly pathogenic avian influenza (H5N1) virus and the origin of the virus (SARS-CoV-2) responsible for novel coronavirus infections (COVID-19), researchers and policy makers have been involved in the debate on the definition and promotion of GOF research and DURC for more than a decade [2]. Meanwhile, the urgency of international governance of DURC is growing [3,4]. Currently, open science has gained increasing attention, particularly in genomic medical research, where research data are shared and managed through international databases to accelerate research progress while enhancing research quality. Owing to the growing concern on dual use in society, triggered by the current COVID-19 pandemic and the development of synthetic biology and genome editing, research governance comes to the fore in terms of how to strike an appropriate balance between security and open science.

As was originally hoped for in the Biological Weapons Convention, sovereign intervention by states and inter-state organizations is at the center of governance discussions as an approach to dual use and biosecurity. Specific measures of governance have been implemented and advocated, such as the development of legal frameworks focusing on technologies, substances, or users of technologies that could be considered potentially dangerous, and the strengthening of peer review systems by governments and journal publishers in light of the technologies’ inherent characteristics [5]. However, top-down interventions, such as legal regulations and institutional oversight, inevitably make it difficult to reflect technological developments and practical issues in the research field in a timely and flexible manner [6]. Recently, many scientists have protested against a major expansion of the United States government oversight of life science DURC, claiming that it interferes with their daily research [7].

In contrast, the Organization for Economic Co-operation and Development, United States, and Japan have begun to require not only governments, but also individual universities and research institutions to address ‘research integrity’ and ‘research security’ and envision a role for the research community [8–10]. Such efforts may need to align with DURC efforts in biotechnology. In fact, life science researchers in the United States prefer self-governance over DURC [11]. Currently, however, practical tools for the research community are limited to the formulation of voluntary guidelines, and education and training. Concrete efforts toward precaution and post-event response to unintended events are lagging behind. Although the World Health Organization has indicated how research institutions and other stakeholders should address DURC [12], the actual practice is undeveloped in individual research institutions and laboratories [13].

The recent examples of GOF research, research ethics committees, and biotech industries suggest that self-governance of the research community is difficult, and no single actor can manage life science research independently and autonomously [14–16]. This raises the question of how the research community can go beyond self-governance and proactively collaborate with other stakeholders in a bottom-up manner. In this paper, therefore, we discuss the technological, dialogical, and personnel approaches and their usefulness in going beyond the dichotomy of top-down governance and self-governance in dual-use issues, and expand upon widely developing bottom-up governance involving the research community and various other actors. Each approach is directed to research subjects, research practices, and persons involved in research, focusing on both local and individual activities, such as projects and laboratories, and systemic activities of their collaborations and networks. Although the concept of open science is not one to be opposed to, we dare to point out its limitations and question the methodologies for research evaluation and assessment, and responsible research and innovation (RRI) that are required to achieve a sound scientific research environment.

Approaches to realize bottom-up governance of DURC

Technological approach

The challenging part of DURC is identifying the research subjects with dual-use potential, and establishing a system to track and monitor the movement of such subjects. The system cannot be addressed only by laws, regulations, and human control owing to the transdisciplinary and transboundary nature of the issue. The key question is how to combine technical solutions effectively and efficiently. The development of safer gene drives [17] is one such solution; nevertheless, scientists and engineers have little incentive to work on research and development that is difficult to identify and reduce safety risks, or to assess the risks of existing substances and technologies. In such cases, risk estimation may be conducted only to justify technological development. Therefore, the technological approach in this paper refers to a technological tool, method, or system that improves the quality, reproducibility, and scalability of research to predict and monitor the dual-use nature of research, including the risk of the research subject.

In the case of predicting and monitoring the dual-use nature of the research subject, it may be more problematic now than in the past to identify and address the impact caused by the artificial modification of the agent. The challenge of how to manage substances that can be modified in part to have a similar function, or even a different function altogether, needs to be addressed. Even the same pathogen has different virulence in different hosts, and it is quite difficult to follow the nature of the virulence. The same is true for the introduction of point mutations whose effects are quite difficult to predict; mutations may reduce gene function or, conversely, activate it [18]. With recent developments in bioinformatics, technological solutions are being sought for risk identification. For example, attempts have been made to predict the function of mutants using the amino acid sequences of proteins from various organisms that are thought to have the same function due to high sequence similarity [19,20]. To realize this, it would be possible to design computer programs that determine the potential hazards for mutants to be synthesized. If the function of a domain within a protein is known, it may be possible to program for genetic manipulation on a domain-by-domain basis. The development of a program to predict the three-dimensional structure of a protein from its arbitrary amino acid sequence is underway [21]. New dual-use specific tools and methods have also been proposed, such as understanding the activity and consequences of sequences of concern (SoC) on the host during infection [22], and a web-based Dual-Use Quickscan that allows researchers to easily monitor dual-use potential [23].

However, these efforts may end up as self-governance proposals by a few researchers concerned about dual use, following top-down regulations and guidelines by governments and public funding agencies, and may not lead to major reform of academia involving other stakeholders. The expected change agent here is the life science industry, which promotes biofoundries and DNA synthesis. Combining standardized bioengineering workflows with automated design-build-test-learn cycles, biofoundries have helped to optimize products for viral testing and vaccine production during the COVID-19 pandemic and has served as an interface for a wide range of stakeholders and citizens. Such biofoundry automation could strengthen existing biosecurity programs and improve environmental and public health monitoring [24]. Meanwhile, the International Gene Synthesis Consortium (IGSC), composed of DNA synthesis companies, has published protocols for customer screening and sequence screening in compliance with United States government guidance and other international standards [25]. One of the consortium members, Twist Bioscience, has conducted an extensive red teaming (independent third-party audit) of its sequence screening system and shared the results with other IGSC members to further improve their respective systems [26]. Although these industry trends do not necessarily have an immediate impact on academia, automated facilities such as biofoundries and cloud labs can improve the quality, reproducibility, and scalability of research and are expected to provide access to science for a wide range of players [27,28].

Thus, peer reviewers, journal publishers, industry, private foundations, public funding agencies, and research institutions each provide resources and support for activities that gradually increase the robustness, security, transparency, and democracy of research governance, which provides incentives and opportunities for scientists to address dual-use issues and revise their own framings of science.

Personnel approach

Who is involved in the research needs to be viewed as an inseparable and continuous issue in DURC. This is not only a downstream issue, similar to the users of technology [5], but also a more upstream issue, such as what kind of applicability the research has and how knowledge and data obtained through research will be used. This paper thus defines the personnel approach in DURC as a process of recruiting and screening persons involved in research after checking and monitoring them from the planning, implementation, and use stages, thereby determining their eligibility for involvement.

Background checks and security clearances of individual researchers are regarded as major check points in the field of biosecurity [29,30]. However, as there is a limit to how accurately a single institution can confirm the identity of an individual researcher, improving screening efficiency at the time of hiring remains challenging. Nevertheless, having proven experts vouch for their identities in data sharing will be effective in enhancing security in future DURCs. The following are two examples of controlling data formatting for digital identity and access permission in platforms for genomic data management.

First, GISAID is a global data sharing platform that provides open access to genomic data on influenza virus and SARS-CoV-2, and requests identification from users who agree not to republish genomes from the site without the data provider’s permission. This allows for appropriate acknowledgment of data providers and builds trust through data sharing [31]. Some argue that GISAID does not allow re-sharing of sequences publicly, thus hampering efforts to understand SARS-CoV-2 and the rise of its new variants; conversely, others recognize that retaining the rights of data providers incentivizes real-time data sharing [32]. The three genome databases in the United States, Europe, and Japan, members of the International Nucleotide Sequence Database Community, are similar to GISAID, but anyone can access the data anonymously and use it freely. In this sense, as for security in DURC using genome data, actions only by GISAID entail many loopholes [33,34]. This demonstrates an evident trade-off between openness and security.

Second, one attempt that could resolve this tradeoff is the Global Alliance for Genomics and Health (GA4GH) initiative, an international consortium founded in 2013 aiming to advance medicine and medical science by developing standards for collecting, storing, analyzing, and sharing genomic data. It has established the Passport standard, defining a machine-readable digital identity in the form of a passport visa and conveying data user roles and data access permissions. These visas are used by technical infrastructure services to manage data access permissions of data users across repositories [35]. GA4GH integrates more technology-controlled approach and has stricter access policies than GISAID. Both GISAID and GA4GH have had many scientists taking a leading role since their inception, and carried out the task of access control as an activity that would be difficult for a single institution to undertake.

Dialogical approach

The argument that science should be advanced in society and for the benefit of society has become dominant in the last two decades, emphasizing the salience of open dialogue between scientists and a wide range of stakeholders and citizens about what and how they do their research [36,37]. At the international level, stakeholder dialogue has led to agreements and decisions on the governance of emerging technologies and direction of research. However, since the dual-use issue remains controversial, controlling who is invited to the dialogue may effectively eliminate debate on dual use [38].

Even if discussion and dialogue are inhibited by political factors, it is possible to promote discussion and dialogue among diverse stakeholders, especially in research institutions and laboratories. The dialogical approach in bottom-up governance aims to proactively engage researchers who are advancing individual research practices to discuss the direction of research in dialogue with diverse stakeholders on the research outcome and its social impact. The significance of this approach is two-fold. On the one hand, it deals with events that cannot be handled by standard practices conforming to laws, regulations, and guidelines. While there have long been arguments for the inclusion of citizen voices in DURC [39,40], the first step in on-site attempts is aimed at involving a wide range of stakeholders. For example, researchers, engineers, or managers appointed as Ethics Rapporteurs in the EU-funded Human Brain Research Project are engaged in an ongoing dialogue to advance research activities in consideration of not just legal compliance, but also dual-use issues [41]. On the other hand, discussions among people with different knowledge, experiences and values enhance insight and enable each to contribute to better governance. This can be more effective in an open forum for knowledge exchange that brings together diverse stakeholders. For example, the International Genetically Engineered Machine (iGEM), an international conference aiming to advance synthetic biology and improve undergraduate education, asks all participants to consider not only biosafety and biosecurity, but also human practices in their research activities, including ethical, legal, and social issues, for which outstanding teams receive awards [42]. The referee system gives another opportunity for wide-ranging discussion, where four to six referees with different backgrounds judge one team through a close exchange of their opinions and comments from an expert committee in complex cases. Despite these ongoing efforts, research in the iGEM has shown that definition-based education has a low effect on the widespread understanding of dual use [13]. This suggests that dual use should not be a temporary awareness-raising subject for competitions but should be integrated into daily practice in the laboratory. Scenario-based discussions and team-based learning in biosecurity education allow for more serious consideration of dual-use issues in the laboratory [43,44].

Proactive involvement of the industry is also important for open governance, as iGEM is sponsored not only by the Federal Bureau of Investigation (FBI) and foundations, but also by several companies. CRISPRcon is an international conference that discusses the agricultural and health applications of genome-editing technologies, bringing together international organizations, government agencies, NGOs, and business sectors to engage in open dialogue. Thus, initiatives for RRI are spreading in the biotechnology field [45,46]. This could be an opportunity to develop the traditional concept of social responsibility of scientists and engage them in DURC discussion participation at a higher level.

Rethinking research review and assessment

We have thus far discussed three approaches to bottom-up governance of DURC, and all these approaches evidently illuminate research activities as a multi-layered and networked system, rather than being limited to a single laboratory or research institution. However, in discussing the dual-use issue, it is also necessary to focus on aspects where research is used, that is, how academic and social impacts are generated. Rethinking the review of research results and assessment of research outcomes for more appropriate use of research calls into question the traditional academic research evaluation system, as well as the role of journal publishers and funding agencies.

Preprints are scientific papers distributed mainly via the Internet prior to formal peer review. They currently account for only 4% of all research articles; however, their number has skyrocketed 63-fold in the last 30 years. They are published, on average, 14 months earlier than journal articles and are cited five times more often [47]. Although the flawed preprint of an observational study showing that the antiparasitic drug ivermectin improved survival against COVID-19 was retracted in approximately a month, its impact was so great that policies recommending the use of ivermectin as a treatment for COVID-19 were adopted in several countries, including Peru. Whereas preprints support public health decision-making and accelerate research, they can also entrench attractive or dangerous ideas before the completion of proper scientific research studies [48]. Preprints allow authors to submit reviews without author permission or editorial invitation, which is why a number of preprint review platforms, such as Early Evidence Base, Sciety, and bioRxiv, have emerged [49]. Leading preprint servers, such as medRxiv and bioRxiv, have a policy of screening for dual-use research and biosecurity risk-related discussions before submission; however, researchers can also choose to submit to non-screening servers. Thus, proper preprint review will require a coordinated effort among research institutions, funding agencies, and journal publishers [50].

Risk-benefit analysis (RBA) is a well-known assessment of research outcomes and is often used for DURC owing to its potential social impact, and funding agencies have encouraged its proper implementation [51,52]. However, the benefits of ePPP studies are not commensurate with the high risks, and the general public mostly perceive that they are unacceptable [53]. Recently, GOF research, a DURC study, has been claimed to not necessarily help prevent pandemics, and its title as academic research for the benefit of society has been questioned [54,55]. The risk-benefit considerations by simple means of RBA are methodologically and ethically inadequate because of the incertitude of knowledge and urgency of decision-making, difficulty of ascertaining the motivations and capacities of users, and disobedience of the procedural principle of obtaining consent by the entities exposed to risk [56–58]. Moreover, the institutional review of DURC is a highly subjective process that differs according to risk tolerance, experience, and training of the reviewers [59].

Some argue that measures should be considered to avoid as much risk, cost, and unacceptable harm as possible, according to proportionality or No Means to Harm principles [60,61]. Alternatively, precautionary and participatory methods that promote dialogue and interaction while comprehending the limits of knowledge may also be effective [56]. The United States Department of Health and Human Services recommends consideration of ethical values, such as non-maleficence, beneficence, justice, respect for persons, scientific freedom, and responsible stewardship, when reviewing funding for research on ePPPs [62]. Funding agencies will be required to systematically and openly iterate on the design and redesign of research programs as experimental governance [14].

Discussion

This section discusses the implications of each of the three approaches described above for DURC governance. In the technological approach, the development of technological tools, such as programs that facilitate risk identification, provides an incentive for researchers to engage in DURC. If the potential risks of the target agents can be predicted in advance, general life scientists will be less likely to be involved in the dual-use issue, and their contribution to the development of bioinformatics will provide an incentive for their research performance and careers. In other words, rather than facing DURC in a defensive manner, it can be evaluated as a bottom-up, voluntary effort that aims to steer research to a more innovative direction.

Technological tools can also be used for screening before academic results are published and are widely used by society. The Automated Screening Working Group (ASWG) is an international and cross-organizational group of researchers and tool creators who are seeking to build a new review system for COVID-19 preprints on medRxiv and bioRxiv using automated screening tools. The application of these tools to submitted manuscripts and preprints is progressing, and they are expected to be able to quickly identify potential problems and complement peer review [63,64].

The dialogical approach may be effective for research management and raising awareness of the people involved to provide opportunities for ongoing dialogue, discussions, and training sessions at the lab level. iGEM participants may not be educated about the significance of dual use through participation in the competition; nevertheless, the organizers and the United States federal government may be able to provide a more effective means of education through the iGEM. Furthermore, the FBI is working with amateur biologists (DIY biologists) to prevent future biological risks that may arise from research through continuous dialogue [65]. Meanwhile, DIY biologists can disseminate their socio-technical vision through the FBI while participating in framing research ethics through institutional review boards [66,67].

Note that the participation and engagement of diverse stakeholders are necessary, where some criticize that the opinions of opponents to genome-editing technologies have been largely ignored at CRISPRcon [68]. For this, it would be necessary to first extend the consideration of dual-use issues in limited communities to other domains. For example, BIOMOD is a student competition, similar to iGEM, in the field of molecular robotics, and BIOMOD Japan follows iGEM in promoting ethical and social considerations in project teams [69]. Another example is a case of international initiatives, including a series of co-hosted workshops between the Johns Hopkins Center for Health Security and the Tianjin University Center for Biosafety Research and Strategy conducted from 2019 to 2021, which compiled the Tianjin Biosecurity Guidelines for Codes of Conduct for Scientists and disseminated them globally [70,71]. This is a notable example of bottom-up collaboration based on several years of continuous dialogue between universities in two countries with strained relations. The dialogical approach suggests the limits of self-governance by the research community. By working with other stakeholders through dialogue, an ‘extended peer community’ [72] is formed to build governance that enables more accurate issue identification from a broad perspective. Here, ‘opening-up’ elicitation and deliberation methods to employ pluralistic discourse in participatory appraisal, including multicriteria mapping, Q methodology, and repertory grid, are effective in building an open space for dialogue [73].

The personnel approach highlights the difficulty of the correspondence between the people involved in the research and their intentions for being involved in this era of open science. In addition to conducting as many background checks as possible at universities and research institutions, the appropriate data providers and data users in data sharing must be determined. Not only does this prevent the infiltration of suspicious individuals from a national security perspective, but it also provides a contingency plan by setting a range of access to data for each individual, even for legitimate researchers [74], such that the flow of data can be traced back in the event of an incident. This provides a solution to not only screening at the time of employment, but also the management of the removal of materials and knowledge by individuals who have left their institutions. To this end, universities and research institutions should examine their policies on research management and review their self-governance.

As Figure 1 shows, this paper adopts the traditional definition of governance as interacting, multilayered levels of macro, meso, and micro [75]. Macro refers to policy-making functions at the national or international level, meso to decision-making functions at the institutional level, and micro to operational issues in research institutions and laboratories. Here, top-down governance refers to the action of macro-level governance on mesos and micros, while self-governance refers to the action completed within the micro level. Bottom-up governance represents governance acting from the micro to the meso level but does not preclude a top-down approach in governance at the meso level, as macro-level policy formation also affects the meso level.

Figure 1. Three approaches to bottom-up governance of DURC.

In the bottom-up governance of DURC, it would be useful to take the technical approach for appropriately identifying the risk of the research target pathogens and their information, the personnel approach for identifying and managing the appropriate persons involved in the research, and the interactive approach to research practice for achieving collaboration between researchers and other stakeholders. However, this is a simplistic scheme, and in practice, DURC should be governed by an appropriately integrated design of these three approaches, such as GA4GH, which uses a combination of technical and personnel approaches to manage data users. In the international context, however, since even meso-level governance platforms, such as GISAID and GA4GH, are considered inescapably political to some extent [33], different actors in multiple levels and systems would act as gatekeepers and mitigate specific political agendas and their resulting inequalities.

Furthermore, governance should consider how journal publishers and funding agencies identify and disseminate research results and outcomes to society, as in the biosecurity review [76] and the aforementioned consideration of DURC during grant review. Individual researchers and research institutions can draw a more overarching governance picture by reviewing their collaboration and relationship with journal publishers and funding agencies while conducting research through the integrated practice of the three approaches. For example, various screening tools used by ASWG are supported by multiple research grants, including NIH grants, and are housed in a university-hosted knowledge base called SciCrunch. This activity directly or indirectly engages funding agencies and journal publishers with larger policy and social objectives and promotes the research direction at the macro level through an open international network of researchers. The project also looks at the direction of research at the macro level through an open international network of researchers. This working group is a good example of integrated bottom-up governance, in which dialogue among researchers and stakeholders takes place and screening tools are used to identify legitimate researchers and their activities.

Conclusion

The bottom-up governance of DURC is not fulfilled under the responsibility of each person involved in research. Building a culture of responsibility, as a technology of security, creates tensions between scientists and society, and as seen in the question of the social significance of GOF research, scientific self-governance can rekindle scientists’ interest in how to effectively secure science [77]. This is where the research community and research process need to be open and aim for an open system with a wide range of stakeholders. However, open science is not about making everything open. In the first place, it is undeniable that the very term ‘open science’ has been encouraged by the research community as part of top-down governance [78,79]. Openness requires adequate attention to each of the research subjects, research practices, and persons involved in research, as well as an appropriate check system, including data traceability [80].

Considering the openness of research practice, this paper examined the appropriate restrictions on research subjects and persons involved in research and proposes the need to design a new research system that effectively combines technological, personnel, and dialogical approaches. Beyond the self-governance of researchers, an expanded peer community can be achieved through researcher-led, bottom-up collaboration, and a more conscious open science can be developed while pursuing program-level objectives and outcomes devoid of individual project interests. DURC’s bottom-up governance is expected to redraw the boundaries of open science through the proactive involvement of the research community itself, thereby demonstrating the integrity of the research community and its potential to develop research and innovation.

Acknowledgements

This study has been very much inspired and stimulated by our closed webinar series, and we would like to thank the participants, including Malcom Dando, Nicholas G. Evans, Inga Ulnicane, Simon Wain-Hobson, and Dagmar Vorlíček, and Ayako Kiriyama and Masako Kuroda for their technical assistance.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Ministry of Health, Labour and Welfare under Grant Research on Health Security Control Program No. 21LA 2006.