Identifying and detecting potentially adverse ecological outcomes associated with the release of gene-drive modified organisms

ABSTRACT

Synthetic gene drives could provide new solutions to a range of old problems such as controlling vector-borne diseases, agricultural pests and invasive species. In this paper, we outline methods to identify hazards and detect potentially adverse ecological outcomes at the individual (genotype, phenotype), population, community and ecosystem level, when progressing Gene Drive Modified Organisms through a phased test and release pathway. We discuss the strengths and weaknesses of checklists and structured hazard analysis techniques, identify methods to help meet some of the challenges of detecting adverse ecological outcomes in experiments and confined field trials, and discuss ways to improve the efficiency and statistical rigour of post-release monitoring strategies.

KEYWORDS:

• Gene drives
• CRISPR/Cas9
• risk assessment
• hazard analysis
• monitoring
• stakeholder engagement

Introduction

Gene drive is a generic term for a variety of processes that in sexually reproducing organisms cause genes to be transmitted to successive generations at ratios greater than the classical Mendelian ratio (Champer, Buchman, and Akbari 2016). Natural gene drive processes, such as under dominance and Homing Endonuclease Genes (HEGs), have been known for many decades (Zimmering, Sandler, and Nicoletti 1970), and identified as mechanisms to suppress or modify populations of disease vectors (Wood and Newton 1991; Braig and Yan 2002; Burt 2003; Gould, Magori, and Huang 2006; Lindholm et al. 2016). Harnessing natural drive mechanisms to achieve vector control, however, has proven to be difficult (Smidler, Min, and Esvelt 2017), but this situation could change due to the development of RNA-guided endonucleases such as CRISPR/Cas9 (Jinek et al. 2012).

CRISPR/Cas9 is a gene-editing technique (Le Cong et al. 2013) that is easier and cheaper to implement than other gene-editing tools (Gaj, Gersbach, and Barbas 2014), and has consequently reduced the time needed to complete gene-editing experiments from years to days (Saey 2015). Furthermore, if constructs are developed with DNA sequences that are homologous with the DNA on either side of the cleavage site, then CRISPR/Cas9 systems can be programmed to act like HEGs, copying themselves into cleavage sites via homology-directed repair, and thereby creating a construct that, if copied into germ-line cells, is inherited at a super-Mendelian rate (Gantz and Bier 2015).

CRISPR/Cas9 has been used to successfully edit the genomes of a wide variety of organisms, (Sander and Joung 2014; Doudna 2015; Gantz et al. 2015; Hammond et al. 2015), and the potential applications of CRISPR/Cas9 gene-drive systems are diverse (Burt 2014; Esvelt et al. 2014; NASEM 2016a; Burt et al. 2018; Leitschuh et al. 2017; Scott et al. 2017). However, the novelty and relatively simplicity of gene editing with CRISPR/Cas9 systems, together with the potential for creating constructs that (at least theoretically) can spread to an entire population following the release of just a single organism (low threshold gene drives), introduces the possibility of unintended consequences to ecological and epidemiological endpoints (Webber, Raghu, and Edwards 2015).

Recognition of these potential risks has led to recommendations for researchers to act cautiously and implement multiple, independent confinement strategies to prevent unintended releases into the environment of organisms carrying gene-drive constructs (Araki, Nojima, and Ishii 2014; Akbari et al. 2015; NASEM 2016a). Ultimately, however, many of the potential benefits of gene-driving technologies are only possible if organisms carrying these constructs are intentionally released into the environment in a manner that is designed to allow them to spread through a target population.

Deliberate unconfined releases of gene-drive modified organisms (GDMOs) will likely be sanctioned within a risk-based decision-making process. This process will be influenced by ethical, socio-cultural, epidemiological, ecological and economic considerations, and the risk analysis process should include mechanisms that facilitate the effective engagement of stakeholders and help integrate these considerations within the overall decision-making process (NRC 1996; Renn 2008; NASEM 2016a). Risk analyses for living modified organisms (LMOs) are typically conducted on a case-by-case basis and we envisage the same approach will apply to GDMOs. These analyses should aim to: (1) identify hazards, defined here as the circumstances that may lead to adverse (harmful) outcomes; (2) evaluate the risks associated with these hazards; (3) identify strategies that help to reduce these risks and (4) design and implement monitoring strategies to detect, or confirm the absence of, adverse outcomes.

In this paper, we focus on this first and last step, by outlining methods to identify hazards and subsequently detect, as efficiently as possible, any potentially adverse ecological outcomes associated with the confined and unconfined release of GDMOs. Identifying potentially adverse outcomes, and the causal pathways by which they might manifest, will help analysts quantify risks, implement risk mitigation strategies and design and implement cost-effective monitoring programmes.

Phased test and release pathway

CRISPR/Cas9 gene-drive systems are theoretically reversible, theoretically limitless, systems capable of altering and suppressing (possibly to extinction) populations of target species (Burt 2003; Smidler, Min, and Esvelt 2017). A prudent way to approach a deliberate release of a GDMO into the environment for such purposes is to follow an iterative, phased, test and release pathway (Figure 1) that: (a) gathers risk-relevant data under controlled, contained conditions, and thereby develops confidence that the GDMO can safely progress to the next phase; (b) generates release-relevant data by observing outcomes under increasingly realistic scales and conditions, which will require a deliberate and gradual relaxation of the level of containment and (c) terminates the development of GDMOs that fail to meet pre-specified (in a Target Product Profile), phase-specific, performance- and safety-related criteria (NASEM 2016a), that may be amended in an iterative fashion based on data gathered at subsequent phases.

Figure 1. Example of phased testing and release pathway for Gene Drive Modified Organisms, based on the phased testing pathway for Genetically Modified Mosquitoes (source: World Health Organization 2009). Combined function in Phase 1 refers to the phenotypic effect of the transgene once it is successfully integrated into the genome. Confined field trials in Phase 3 can entail one or more types of physical, ecological or molecular containment strategies. Semi-field conditions in the context of genetically modified mosquitoes refers to enclosed greenhouses that emulate natural environments. More generally, however, semi-field conditions refers to contained release strategies under natural or natural-like conditions.

The need to gather risk- and release-relevant data – that is data that helps quantify the likelihood of hazards at increasingly realistic scales – is well understood in the context of transgenic crops (Rissler and Mellon 2000; Firbank, Lonsdale, and Poppy 2005; Snow et al. 2005; Andow and Zwahlen 2006). Phased test and release strategies that aim to generate such data have since been recommended for genetically modified mosquitoes (Figure 1; World Health Organisation 2009, 2014), transgenic fish (Hayes et al. 2014) and most recently GDMOs (NASEM 2016a).

Phased test and release strategies for GDMOs should incorporate physical, reproductive, ecological and molecular barriers (Akbari et al. 2015). The level of containment provided by physical barriers, such as those described in Arthropod Containment guidelines (ACME/ASTMH 2003), will likely vary as a GDMO progresses through the testing and release pathway. We expect physical containment procedures to be most stringent in Phase 1 or Phase 2, possibly exceeding Biosafety Level 2 or Arthropod Containment Level 2 (Benedict et al. 2017), because relatively little will be known about the phenotypic characteristics of the GDMO or the reliability of molecular containment strategies in these early phases. If implemented correctly, these procedures are expected to reduce the probability of environmental exposure to very low levels; Hunt and Tabachnick (1996), for example, estimate the probability of small insects (<2 mm in length) escaping their Biosafety Level 3 facilities to be between 1 × 10⁻¹⁰ and 2 × 10⁻¹⁴. These procedures will need to be deliberately relaxed to increase the scale and realism of the experimental environment, if a decision is made to proceed to the next phase.

Ecological barriers – i.e. performing experiments outside the habitable range of the GDMO, or in areas without potential wild mates – is a potentially effective containment strategy. It can be gradually relaxed, by releasing the GDMO to a geographically isolated location within the habitable range, or within a location habitable for only part of the year, and it may eventually be removed entirely. Typically this would occur at the confined field trial or open field release phases of the process. The efficacy of this barrier can be overestimated if our knowledge about the organism’s ability to hybridize with closely related species is incomplete. It can also be circumvented by accidental or deliberate transport to new regions, as demonstrated by the escape of the calicivirus agent for Rabbit Hemorrhagic Disease Virus from field trials on Wardang Island (Mutze, Cooke, and Alexander 1995).

Reproductive strategies – i.e. incorporating gene drives in laboratory strains that cannot reproduce with wild organisms – are in principle likely to be highly effective but only suitable for studying drive systems in Phases 1 and 2. Molecular containment methods, and related molecular reversal methods (Esvelt et al. 2014; DiCarlo et al. 2015; Wu, Luo, and Gao 2016), can be theoretically maintained throughout the test and release pathway, but their efficacy in field conditions remains untested.

Identifying hazards

The Royal Society (1983) defines hazard as a situation that in particular circumstances could lead to harm. This definition emphasizes that hazard analysis must identify the circumstances that lead to harm, i.e. the causal pathway, rather than simply identifying potential adverse outcomes. This is useful because the potential adverse ecological outcomes that are being discussed in the context of GDMOs (Table 1) are the same as, or similar to, those that were identified for transgenic crops (Rissler and Mellon 2000; Hilbeck et al. 2005; Snow et al. 2005; Andow and Zwahlen 2006), transgenic fish (Muir and Howard 2002; Devlin et al. 2007), biocontrol agents and invasive species (Simberloff 2012; Simberloff et al. 2013). The risks associated with the next generation of GDMOs could be different, however, because the causal pathways that lead to adverse outcomes – and hence the likelihood and magnitude of these outcomes – may be different from those associated with LMOs, biocontrol agent or invasive species.

Table 1. Example hazardous events identified in the literature (second column) that may be relevant to risk analyses of gene drive modified organisms, because of the potentially adverse ecological outcomes (fourth column) that these events might lead to.

Scale	Hazardous event	Reference	Potentially adverse ecological outcomes
Molecular	Cas9 cleaves loci with similar, but not identical, homology to the target loci	Tsai et al. (2015)	New phenotype with a different (possibly increased) capacity to spread diseases or pathogens.
	Mutated gRNA causes Cas9 cleavage of non-target sequence	Sander and Joung (2014)	New phenotype with a different (possibly increased) capacity to spread diseases or pathogens.
	Cas9 fails to edit or target all alleles	Araki, Nojima, and Ishii (2014)	Mosaicism within individual organisms leading to a gain of function polymorphism that changes the target organism’s ability to survive, reproduce or spread
	Mutations occur during repair of multiple cleavage sites.	Sander and Joung (2014)	Multiple alleles leading to mosaicism in subsequent generations leading to a gain of function polymorphism that changes the target organism’s ability to survive, reproduce or spread
Population	Assortative or non-random mating between new phenotypes	Scott et al. (2002)	Drive is reduced and/or competitive advantage accrues to a more virulent phenotype leading to an increase in the incidence of the disease or pathogen of concern
	Intraspecific (admixture) and interspecific hybridization	David et al. (2013)	Gene drive is acquired by, and spreads within, non-target population or non-target species leading to the suppression or modification of this population or species
	Unpredicted phenotypes from gene by environment interactions	Tabachnick (2003)	Gene drive fails to produce refractory organisms in the wild but increases target organism’s capacity to spread diseases or pathogens.
	Changes in the innate immune response of the GDMO	Scott et al. (2002)	Change in (possible increased) the capacity of the target organism to spread other diseases or pathogens.
Community/Ecosystem	Population/species suppression changes competitive relationships	David et al. (2013)	Release from competition allows a detrimental population or species to increase in abundance
	Population/species suppression causes extinction of (prey) species	David et al. (2013)	Cascading effects on food web caused by decrease in abundance of predators leading to possible loss of ecosystem services
	Eventual increase in drive resistance individuals	Webb (2011);	Incomplete suppression/modification leading to loss of herd immunity and eventually increased incidence of disease or pathogens
	Horizontal (lateral) transfer of gene drive to distant species	Wijayawardena, Minchella, and DeWoody (2013)	Gene drive is acquired by, and spreads within, non-target species, leading to suppression or modification of the non-target species

The lack of familiarity with a new technology like gene drives presents a challenge to risk analysts who want to identify environmental hazards – i.e. the circumstances leading to adverse ecological outcomes – associated with the deliberate or accidental release of GDMOs. There are two approaches to this challenge: (a) seek precedence in the hazards identified for more or less similar situations, hereafter termed the ‘checklist-like approach’, and (b) employ structured hazard identification to postulate what might go wrong. We explore the strengths and weaknesses of both options below.

Checklist-like approach

Compiling lists of the ways in which adverse outcomes have occurred in the past, and how to prevent similar outcomes in the future, is one way to identify hazards and help ensure that mitigation strategies are applied to the risks of an activity. Checklists are most commonly used in the context of activities that have a long history of successful operation – i.e. where the operating history provides assurance that the risks are well understood and have been managed successfully in the past, e.g. greenhouses or cages in ‘semi-field systems’ (Knols et al. 2003; Ferguson et al. 2008; for checklist see http://johnmm.bol.ucla.edu/containment.htm#BreachesOfContainment).

A potential drawback of checklists is that hazard identification is focused on what is known to have occurred in the past. A checklist may therefore lead assessors to miss hazards that are unique to a new technology because it operates in ways that are different from existing technologies. Changes in farming practices associated with herbicide tolerant (HT) crops, for example, had positive and negative effects on biodiversity in the UK large-scale field trials (Hawes et al. 2003), but virtually all prior commentaries on HT crops failed to identify farming practice change as a potential hazard.

Checklists could also mislead risk analysts into believing that all aspects of a system that ought to have been questioned actually were addressed, or conversely lead analysts to address hazards that are irrelevant to the type of gene drive or its intended application. Table 1, for example, provides a list of hazardous events drawn from published studies. This list could serve as a hazard checklist but it is by no means complete, nor necessarily appropriate, because much of the current literature focuses on the use of CRISPR/Cas9 to control mosquito-borne diseases. Thus, analysts seeking to complete a hazard analysis for an RNA-guided Cpf1 endonuclease (Champer, Buchman, and Akbari 2016) to suppress mice populations, for example, might miss hazards if they relied solely on Table 1. As the range of drive mechanisms and applications grows (NASEM 2016a), analysts will need more robust methods to rigorously identify hazards in a comprehensive but case-specific manner.

Structured hazard identification

Structured hazard analysis tools (Table 2) aim to elucidate the causal pathways that link initiating events at a molecular level to harmful outcomes at an individual, population, community or ecosystem level of organization. Thinking of hazards in terms of causal pathways – or pathways to adverse outcomes – has proven useful in ecotoxicological risk assessments (Ankley et al. 2010) but the application of the tools listed here to facilitate this process in ecological risk assessment is still relatively novel.

Table 2. Summary of structured hazard analysis techniques that could be applied to identify hazards associated with GDMOs.

Method	Application to LMOs	References
Hazard and Operability Study (HAZOP)	Modified version of HAZOP, termed GENHAZ recommended as hazard identification tool for LMO’s by a United Kingdom Royal Commission on Environmental Pollution	Watts (1989)
Fault Tree Analysis (FTA)	Applied as heuristic hazard identification tool for ballast water introductions and the release of genetically modified carp, and as a risk quantification tool to male sterile mosquitoes modified with I-PpoI construct	Hayes et al. (2015); Hayes et al. (2014); Hayes (2002a)
Event Tree Analysis (ETA)	Simple examples applied to biological systems – no known examples of application to LMOs	Ericson (2005)
Hierarchical Holographic Modelling (HHM)	Applied to identify hazards associated with breaches of HT Canola license conditions and male sterile mosquitoes modified with I-PpoI construct	Hayes et al. (2004); Hayes et al. (2015)
Failure Modes and Effects Analysis (FMEA)	Modified versions applied to biological systems – hull-fouling introductions and coal mining/coal seam gas developments – no known examples of application to LMOs	Hayes (2002b); Ford et al. (2015)
Qualitative mathematical modelling (QMM)	Numerous examples of the application within ecological systems, together with a demonstration of application to the release of genetically modified carp in Australia	Dambacher, Li, and Rossignol (2003); Hayes et al. (2014)

Most of the tools in Table 2 were developed to identify hazards in industrial/aerospace systems (Kumamoto and Henley 1996). Some have been successfully applied to biological systems (Hayes 2002a, 2002b; Hayes et al. 2004), while others were specifically developed for applications in ecological systems (Dambacher, Li, and Rossignol 2003). The principal advantage of these methods, as compared to a checklist, is that they help analysts to carefully and systematically apply their expertise beyond their own experience when constructing causal pathways. For example, one possible hazard (out of many) identified in Table 1 is that the Cas9 protein complex cleaves loci with similar, but not identical, homology to the target loci, leading to the creation of a new phenotype with enhanced capacity to transmit diseases or pathogens. Structured hazard identification tools assist an analyst to construct models of how this hazard could occur, for example by systematically considering the genes that determine or mediate the underlying parameters of disease transmission, the extent to which these genes contain sequences that are homologous to the target loci, and the processes that determine or may alter the sequence specificity of the Cas9 guide RNA. Carefully deconstructing each of the necessary steps in the causal pathway of a particular hazard helps generate risk hypotheses that can be further analyzed and constructively critiqued before proceeding to the next stages of a risk assessment.

In biological settings, the experts needed to complete these processes are usually volunteers drawn from multiple institutions and this can sometimes make it difficult to secure enough time to complete the processes. These processes also require careful facilitation in order to integrate successfully the competencies of groups with often diverse backgrounds and expertise.

Another challenge is that the techniques listed in Table 2 require a detailed description of the system of interest. At a molecular level this requires knowledge of the biological processes that govern endonuclease promotion, translation, sequence specificity and DNA cleavage and accurate homolog directed repair (e.g. Sander and Joung 2014). At the population/community level, they require knowledge of the target organisms’ life-cycle, phenotype, habitat preferences and trophic interactions at each life-stage (e.g. David et al. 2013; Hayes et al. 2014). The analyst must be able to describe the system, as completely as possible, at an appropriate scale and degree of resolution, in the same way that an engineer would describe an industrial system when implementing these techniques in an industrial/aerospace setting. For biological systems this is a difficult task, which leaves open the possibility that some pathways will be missed.

Identification and detection within the phased testing pathway

Phase 1 and 2: laboratory studies and small-scale experiments

Potential hazards, and their associated adverse outcomes, can be modelled and searched for experimentally (phenotype, population and community outcomes) or via genome sequencing (genetic outcomes). As a GDMO progresses through the phased testing and release pathway, however, the spatial and temporal scales of the concomitant biosafety studies increase, and the suite of tools used to identify hazards and detect adverse outcomes changes (Figure 2).

Figure 2. Spatial and temporal scales of outcomes associated with the phased test and release pathway of Gene Drive Modified Organisms. As a GDMO progresses through the test and release pathway (phases 1 to 4 in yellow) the spatial and temporal scale of experimental outcomes, and potentially adverse outcomes, increases. The techniques used to identify hazards and detect potential adverse ecological outcomes vary as these spatio-temporal scales change.

Experimental studies of GDMOs in phases 1 and 2 (respectively, laboratory studies and laboratory population cages in Figure 1) are anticipated to follow a three-step process: (i) describe the experimental objectives within the context of the GDMO’s planned genotypic and phenotypic and characteristics (Target Product Profile); (ii) select observational targets from a range of potential targets (within available resources) and (iii) develop and execute an adequate (for the objectives) experimental design.

The experimental objectives in Phase 2 would typically include measuring the gene drive’s conversion efficiency (the rate at which heterozygous individuals are converted to a homozygous state) within a wild-type genetic background. The objectives should also include gathering data relevant to the likelihood of potentially adverse outcomes identified by the hazard analysis methods described previously. For example, the hazard analysis might identify the possibility of off-target changes to near homologous sequences in the organism’s genome that are known or thought to govern traits that influence the GDMO’s environmental niche, its susceptibility to existing control methods or its capacity to transmit animal or plant pathogens. This would indicate the need to conduct genetic screening and/or whole-of-organism experiments to test the frequency of these outcomes. The key challenge, however, is that some of the causal pathways identified by the hazard analysis may have a very low expected frequency, or a very high natural variability, thus making experimentation laborious, prohibitively expensive or simply impossible given the size and generation time of the organism concerned.

There are some model organisms that can be raised in very large (10⁹–10¹²) numbers and have very short generation times, such as yeast (Botstein, Chervitz, and Cherry 1997) or the nematode Caenorhabditis elegans. With these organisms, it is theoretically possible to raise the experimental sample size to a sufficiently large number such that ‘rare’, but important events, like spontaneous genetic mutations that compromise molecular containment systems (Moe-Behrens, Davis, and Haynes 2013) can be detected. All such experiments, however, come with the caveats that the evolutionary dynamics of gene drives within model organisms can only approximate those of other species, and cannot account for possible gene by environment (Tabachnick, 2003; Lobo 2008) or epigenetic effects that may occur within real environments.

If the likelihood of a hazard that leads to an undesirable genetic or phenotypic outcome in a target organism is extremely rare, it may be possible to identify influential, intermediate, events on the causal pathway that are expected to occur at a higher frequency, using for example Fault Tree Analysis (FTA), and associated sensitivity analysis techniques (Iman 1987; Rausand and Hoyland 2004; Aven and Nøkland 2010; Hayes et al. 2015). If experimental evidence can determine the rate of one or more intermediate events then, contingent on the structure of the fault tree (including the extent to which it is expanded or pruned) and the assumption that the tree is an adequate model, this may provide enough evidence for decision-making purposes that the fault tree’s top event will be sufficiently unlikely. For example, researchers may not have to complete an exhaustive search for near homology across all possible genes that govern an organism’s capacity to transmit disease (the top event) if the probability of an essential intermediate event on the causal pathway, such as a change in the timing or manner in which the cas gene is expressed (the intermediate event), was quantified and shown to be sufficiently low.

In this context, it is also not necessary for the intermediate event in question (or even a top event) to be actually observed in an experiment. Null event inference methods (for the so-called zero numerator problem) can be used to establish confidence or credible intervals for the probability of events that are not observed in independent experiment trials (Wilson 1927; Winkler, Smith, and Fryback 2002; Gerlach, Mengersen, and Tuyl 2009). So long as the experimental trials are independent these methods can be used to estimate the probability of the event in question, which may be sufficient for decision-making purposes. In this context, however, identifying independent trials may not be straightforward. For example, in an experiment that crosses wild-type organisms with a GDMO to test for undesired phenotypes, it may not be immediately clear what constitutes an independent trial – the number of zygotes, the number of mating events or the number of GDMOs used in the experiment.

Phase 3: confined field trials

The phased testing and release pathway deliberately seeks to increase the realism of experiments by moving outside of the laboratory confines. This is likely to be important for several reasons. Firstly, experience with Bt crops (Lövei and Arpaia 2004; Duan et al. 2009) and biocontrol agents (Barratt et al. 2010) suggests that the relevance of laboratory experiments for some types of hazards, such as non-target effects and host specificity, can be equivocal and therefore contested.

Secondly, the phenotype of a transgenic organism could be influenced by environmental conditions and the organism’s genetic background (Tabachnick, 2003). Genetically modified salmon, for example, can be much larger than wild conspecifics when raised in standard hatchery conditions, but only slightly larger when raised under naturalized stream conditions (Sundström et al. 2007). Phase 3 provides an opportunity to examine these (gene by environment) effects by testing how the GDMO performs relative to the criteria established in its Target Product Profile, under natural or semi-natural field conditions, whilst still maintaining a relatively high degree of containment (compared to Phase 4).

This phase may also enable analysts to design experiments targeting population- and community-scale hazards that arise through causal pathways that involve changes to, or interactions with, wild-type organisms (Pennington et al. 2010) and a realistic suite of predator/prey organisms. Knols et al. (2003), for example, describe semi-natural environments, including breeding sites, feeding sites, indigenous plants and real predators built within contained greenhouses to conduct experiments with the Anopheles gambiae mosquito complex.

Whilst Phase 3 offers a number of advantages over Phase 2, research in Phase 3 will likely face a number of challenges. The probability of inadvertent escape and spread may be higher (depending on the type containment strategy used) than Phase 2. Effect size may be much smaller (as in the case of Sundström et al. 2007), replication could be more expensive, and outcomes may be more variable than those observed under laboratories conditions (Facchinelli et al. 2013). It may therefore be difficult to achieve a satisfactory degree of statistical power. Finally, semi-natural field experiments may fail to adequately capture the conditions that the GDMO experiences during an open field release, and may therefore show outcomes that do not occur in the field (Nguyen et al. 2015).

Phase 4: open field release

As the evaluation of a GDMO moves to Phase 4 (open field release in Figure 1), the task of designing experiments or monitoring programmes to detect potentially adverse outcomes, and attributing them to the GDMO if they are detected, becomes more difficult. The same basic three-step process described under Phases 1 and 2 still applies but some elaboration is required to deal with two additional challenges:

• The increased number of possible confounding effects outside of a controlled environment.

• The increase in scope from mainly temporal variability within the small spatial footprint of contained field trials, to the logistically more challenging and potentially more heterogeneous, spatial and temporal scope of the uncontained field release.

Deliberate release strategies will be able to choose release sites, but will likely have limited control over the subsequent spread of a GDMO within the boundaries set by ecological and reproductive barriers. This means it may be more difficult to identify and/or maintain appropriate control sites.

Data collected in the field are typically more representative of the population of interest than data collected during controlled (e.g. randomized) experimental trials because the latter are often conducted in constrained environments (Rubin 1974). The difficulty with field data, however, is that the analyst may not be able to adequately account for the effect of confounding variables, particularly in the absence of an extensive pre-release monitoring programme that helps establish a comparative baseline. Experience with the yield of Bt crops (NASEM 2016b) and the effect of biocontrol agents (Barratt et al. 2010), for example, suggests that without extensive pre- (and post-) release surveys it can be difficult to disentangle the effect of a transgenic trait, or the deliberate introduction of a new organism, from other variables that influence the desired outcome(s).

Variables that simultaneously influence: (a) which sites or potential sample units are exposed to a GDMO ‘treatment’, as the GDMO spreads through a population and (b) the outcome following a ‘treatment’, must be accounted for either through the monitoring design or in the analysis of the monitoring data or both, otherwise estimates of a treatment effect will be biased and interpretation of the causality of the effect will likely be flawed (Ferraro and Hanauer 2014; Dawid 2015).

A variety of design and analysis methods have been developed in an attempt to either eliminate or adjust for the effects of confounding variables in situations where randomized controlled trials are not possible (Table 3). Most of these methods were developed within medical domains, and some have been applied within environmental domains (Ferraro and Pressey 2015). All of these methods require: (i) at least a conceptual model of how the outcome of interest is likely to be influenced by observable and potentially unobservable variables; (ii) identification of an appropriate sampling frame and survey design and (iii) additional criteria or assumptions in order to assign a causal interpretation.

Table 3. Summary of methods to control for the effect of confounding variables, and help establish causality, with observational data.

Method	Summary
Restriction	Deliberately exclude from the sample frame sample units that are associated with factors that are known to confound the outcome of interest.
Matching	Attempts to match the covariate characteristics of treated and untreated (or case and control) sample units. Matching often based on propensity score analysis.
Stratification	Sample units are grouped by similar covariate characteristics during sampling or analysis phase.
Regression analysis	Use multiple regression models to simultaneously control for the influence of covariates on the outcome of interest.
Synthetic control designs	Uses transparent rules to construct a synthetic single control from multiple control sites.
Discontinuity designs	Exploits similarities in sample units either side of discontinuities in the sample frame created by (for example) eligibility criteria.

Source: (Grimes and Schulz 2002; Ferraro and Hanauer 2014; Madigan et al. 2014).

An appropriate sampling frame and efficient survey design will be essential in any attempt to detect adverse outcomes and attribute these to a GDMO in Phase 4. The sample frame must allow for the spread and (at least temporary) persistence of the GDMO, and ideally extend to areas or regions where the degree of exposure to the GDMO will exhibit spatio-temporal gradients. Moreover, it is reasonable to anticipate that the analyst will know of some factors that will influence exposure to the GDMO, prior to its release, and if so the sampling frame should try to include a reasonable spread of the levels of these factors. For example, if the socio-economic status of neighborhoods or villages influences exposure to the GDMO, then it is important that the monitoring programme’s sampling frame does not exclude neighborhoods or villages from either end of the socio-economic spectrum.

Once the sample frame is established, randomized monitoring designs will help ensure that the sample data enables unbiased estimation of population parameters. Simple random designs and stratified random designs are traditionally used when monitoring ecological outcomes (McDonald 2010), such as those associated with open field release of GM fish (Senanan et al. 2007), but these types of design can often be improved through the use of spatially balanced designs (Stevens and Olsen 2004; Grafström and Lundström 2013). Spatially balanced designs are generally more efficient (provide more information per sample) than the traditional alternatives because they minimize spatial autocorrelation between samples.

The concept and advantages of spatial balance can also be extended into known or suspected covariates (Robertson et al. 2013). If covariates are included in a monitoring design, however, they should also be considered in the analysis (e.g. Gelman et al. 2013), and designs should only include covariates that are already known to strongly affect the observations. Including covariates of dubious explanatory power in the design will make the resulting model larger, and could reduce its ability to detect change. In the extreme case, if there are more covariates than observations in the classical linear model, the model will not have a unique set of estimates and inference becomes impossible.

In phase 4, the spatial and temporal scope of the analysis expands to community and ecosystem level scales. Prior to this stage, it will be useful to compare the efficiency of proposed monitoring designs under a variety of different assumptions regarding the spatio-temporal processes that govern factors such as dispersal and reproduction. For example, comparing a relatively simple process of gradual spread from a source that is slowly growing in size (Shigesda and Kawasaki 1997) against processes that include occasional long-range jumps (Marvier, Meir, and Kareiva 1999; Hallatscheka and Fisher 2014).

Additional considerations

The WHO phased testing and release pathway (Figure 1) can usefully support GDMO risk assessment. We would incorporate two modifications, however. The first is to note the importance of feedback between the Phases. Information and learning gathered during each of the phases should inform the design of experiments and analysis conducted in subsequent iterations of the pathway (NRC 1996). The second modification would be to add two additional Phases to address ‘entry’ and ‘exit’ strategies at the start and end of the pathway.

Entry strategies are deliberate activities that are conducted, prior to, or at least parallel with, Phase 1, and before a formal risk analysis process is initiated (NASEM 2016a calls these ‘Phase 0’ activities). These activities include enabling developers of gene drive technologies to work with interested and affected parties to develop a shared understanding of the problem that the GDMO could solve, setting standards or acceptability criteria for the performance of the GDMO in this context (e.g. through a Target Product Profile) and then comparing the GDMO solution with other options for solving the problem (Lynam et al. 2007; Nelson, Andow, and Banker 2009).

Exit strategies should define conditions under which the apparent absence of adverse ecological outcomes following an open field release is considered sufficient to cease monitoring and reporting. Interested and affected parties should also be involved in determining appropriate triggers for discontinuing monitoring programmes after open field release because this decision may be informed by a mix of social, political, or economic considerations as well as scientific evidence.

Entry strategies developed through a participatory process are expected to support fair and competent decisions to be made throughout the phased testing and release pathway: fair, in that the process helps to provide a role for interested or affected parties, and competent in that the process can facilitate the best decision possible given what was reasonably known under present conditions (Webler and Tuler 2000).

Hazard analysis techniques with a strong graphical basis, such as Qualitative Mathematical Modelling and Fault Tree Analysis (Table 2), are good ways to include a diverse set of interested and affected parties in hazard identification, including those with practical experience with relevant environmental systems. Here we emphasize the need for active participation of interested and affected parties throughout the analysis process, rather than simply communicating the results of the process to these parties. Efficacious stakeholder participation in the phased testing and release process will require a scientifically literate, neutral, and seasoned facilitator (Kaner et al. 2007).

Discussion

Risk analysis recommendations for the current generation of transgenic plants and fish emphasize the importance of adopting a case-by-case approach (NRC 2000; Rissler and Mellon 2000; Snow et al. 2005; Kapuscinski et al. 2007). We expect GDMOs to go through a similar case-by-case risk analysis process, informed by the type of gene-drive, the desired genotypic and phenotypic changes, and the characteristics of the environment into which the GDMO is being tested and released (NASEM 2016a).

The literature to date suggests that the potential adverse ecological outcomes following the deliberate or accidental release of GDMOs are the same as, or at least similar to, those identified for transgenic crops and fish, invasive species and biocontrol agents. The causal pathways (or chain of events) that lead to these potential outcomes, however, may be different because GDMOs contain genetic elements that are designed to spread a transgene across populations or subpopulations. We believe that structured hazard analysis tools will help researchers and risk analysts to systematically and comprehensively identify the causal pathways that lead to adverse environmental outcomes at population, community and ecosystem scales. The period immediately after the proof of concept of a transgenic construct (between phase 1 and 2 in Figure 1) is the appropriate time to conduct these hazard analysis studies and design experiments to detect, and screen for, intermediate events along the causal pathway.

If a decision is taken to move forward with the GDMO, then the process of identifying potential risks of the GDMO, risk management strategies for potential containment failures, and acceptance criteria for progression through each phase of the testing and release pathway should continue to involve relevant stakeholders (Dana et al. 2014). These considerations need to carefully describe the ecological or experimental outcomes that define a ‘successful’ GDMO at each Phase, the conditions under which adverse ecological outcomes would be considered significant enough to terminate development of the GDMO, and the conditions under which the apparent absence of adverse ecological outcomes is sufficient to terminate post-release monitoring.

Long-term monitoring strategies for unexpected adverse ecological outcomes may also (depending on the GDMO application) require innovations in monitoring platforms, such as cheap, automated trapping, and governance and funding arrangements that are secure enough to withstand changing policy and budget priorities. Furthermore, the detection of adverse ecological outcomes may need to be accompanied by additional analysis or experiments to determine the mechanisms by which such an event could be attributable to the GDMO.

Whilst this paper focuses on identifying hazards and detecting potential adverse ecological outcomes associated with GDMOs, it is important to not lose sight of the benefits that gene drive technologies may offer (NASEM 2016a). We recognize that any decision to deliberately test and release GDMOs must balance the benefits and risks of the technology, and compare these with alternative solutions (Nelson, Andow, and Banker 2009). The key issue now is how to integrate transparently developed, scientifically testable, hazard identification and risk assessment methods, within a fair and inclusive decision-making framework that clearly enunciates the risk-based acceptance criteria for progressing this new technology through a phased test and release pathway.

Acknowledgements

The authors would like to thank two anonymous reviewers for their thoughtful and helpful comments, as well as the organizers of the NCSU gene drive workshop for the opportunity to contribute to the ‘Road Map to Gene Drive’ special issue.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Keith R. Hayes is a Senior Research Scientist in CSIRO. Keith leads a small team of statisticians and modellers that focus on developing and applying probabilistic risk assessment methods in the ecological domain.

Geoffrey R. Hosack is a Research Scientist at CSIRO working with the CSIRO’s Environment and Ecological Risk team. Geoff’s research focuses on integrating data analytic methods, monitoring techniques and expert elicitation within risk analyses applied to the domains of ecosystem modelling, genetic engineering and vector-borne disease control.

Genya V. Dana is trained in risk analysis and works on international science and technology policy issues, with a focus on emerging biotechnologies.

Scott D. Foster is a Senior Research Scientist with the CSIRO in Hobart, Australia. The applications are what interests Scott, as well as the (sometimes novel) statistical methods to address them. The applications he works on are quite diverse and include: biodiversity quantification, community modelling, environmental modelling, genetics and fisheries science.

Jessica H. Ford is a Research Scientist working with the CSIRO’s Environment and Ecological Risk team. She is an applied statistician working in areas of environmental risk, fisheries management and the genetic control of vector-borne diseases.

Ron Thresher was the foundation head of the CSIRO Centre for Research on Introduced Marine Pests, and has lead genetic programmes to limit the ecological impacts of escapees from aquaculture facilities (‘sterile feral’ technology) and to manage lower vertebrate pests by means of ‘daughterless’ heritable sex-ratio biasing.

Adrien Ickowicz is a Research Statistician working with the CSIRO’s Environment and Ecological Risk team. His research interest includes spatial–temporal modeling and monitoring, computational and Bayesian statistics.

David Peel is a research scientist with the CSIRO in Hobart, Australia. He is a statistician with a strong application focus specializing in ecological and environmental areas.

Mark Tizard is a team leader at CSIRO Health and Biosecurity. His primary research interest is in genetic control of vertebrate pests.

Paul De Barro is Senior Principal Research Scientist and Research Director of the Risk Evaluation and Preparedness Program at CSIRO Health and Biosecurity.

Tanja Strive is a Principal Research Scientist at CSIRO Health and Biosecurity. Her research interests are in biological control of vertebrates and host–pathogen interactions and co-evolution. Previous research experience ranging from non-lethal, virally vectored immune-contraception approaches for foxes and mice to classical viral biocontrol of rabbits using naturally occurring pathogens.

Jeffrey M. Dambacher is a Senior Research Scientist working with the CSIRO’s Environment and Ecological Risk team. He specializes in the application of Qualitative Mathematical Modelling to problems in the ecological domain.

ORCID

Jessica H. Ford http://orcid.org/0000-0003-4473-8245

Adrien Ickowicz http://orcid.org/0000-0001-7861-7340

Mark Tizard http://orcid.org/0000-0001-8421-2497