Examining the relevance of ‘EVIL DONE’ to the current terrorist threat landscape in the United Kingdom

ABSTRACT

Since the release of the Clarke and Newman’s ‘EVIL DONE’ framework in 2006, there has been limited empirical research on its capacity to predict target attractiveness and vulnerability as intended. This study investigated the utility of the framework in the context of the current United Kingdom (UK) threat landscape, including additions from the [Marchment, Z., & Gill, P. (2022). Spatial decision making of terrorist target selection: Introducing the TRACK framework. Studies in Conflict & Terrorism, 45(10), 862–880. https://doi.org/10.1080/1057610X.2020.1711588] TRACK framework using UK terrorist incidents between 2015 and 2021 (n = 184). For the UK as a whole ‘EVIL DONE’ may not be the best approach to predicting and mitigating the threat. Analysis of cases from Great Britain only demonstrates greater usefulness of the framework in explaining the attractiveness of targets.

KEYWORDS:

• Situational crime prevention
• EVIL DONE
• target desirability

Introduction

Terrorist targets and their vulnerabilities are continuously changing, making it necessary to continually review our understanding of the threat. Situational crime prevention (SCP) has a lengthy pedigree with respect to crime and attempts to reduce specific crimes by affecting the situational determinants (e.g. the immediate environment and opportunity reduction) of that crime and thereby making it less likely to occur (Clarke, 2017). For proponents of SCP, crime is always a choice and therefore ‘creating unfavorable circumstances is the objective of situational crime prevention’ (Clarke, 2017, p. 287). Subsequently, SCP involves five main mechanisms which may affect the potential offender’s decision-making process, namely increasing the effort (e.g. target hardening), increasing the risk (e.g. strengthening surveillance), reducing the rewards (e.g. concealing or removing targets), reducing the provocations (e.g. avoiding disputes) and removing excuses (e.g. posting instructions) – for more details see ASU Center for Problem-Oriented Policing (2021).

Using established SCP techniques, Clarke and Newman (2006) developed a terrorism risk assessment framework for assessing the desirability of targets to terrorists based on eight criteria, known by the acronym ‘EVIL DONE’ (exposed, vital, iconic, legitimate, destructible, occupied, near, and easy). Exposed targets are those that stand out and attract attention (e.g. a high-rise building). Vital targets are necessary to the survival of daily life (e.g. water supplies and power plants). Iconic targets are symbolically significant to society (e.g. the White House and the Statue of Liberty in the United States or Buckingham Palace and the Palace of Westminster in the United Kingdom). Legitimate targets are deemed deserving of an attack (e.g. military personnel or government buildings). Destructible targets are those that are easily destroyed compared to other targets. Occupied targets are those with more potential victims. Near targets are those that are relatively close in proximity to where the terrorists are based. And finally, easy targets are those easily accessed or with little or no security measures in place. Clarke and Newman (2006, p. 4) argued that we ‘must identify vulnerable targets, prioritize them for protection, analyze their specific weaknesses, and provide them with protection appropriate to their risks’.

Clarke and Newman (2006) applied their framework in an illustrative exercise to nine well-known structures in Washington DC in terms of attractiveness to a foreign-based terrorist group, but surprisingly, very few studies have empirically tested ‘EVIL DONE’ and often a limited number of locations have been used for testing (Freilich et al., 2019). For example, Ekici et al. (2008) include analysis of only six sites. Freilich et al. (2019) posit that limited empirical investigation to date might be due to difficulties accessing appropriate and complete data.

Sixteen years since the release of the framework, there remains much uncertainty about its capacity to predict target attractiveness and vulnerability as intended. Since its release in 2006, revisions and additions to the framework have been suggested. However, suggested revisions also lack a sound empirical basis. The current study is designed to test the framework in a contemporary context, accounting for changes to terrorist strategy and suggested revisions to the framework since 2006 in the UK¹ and other western contexts. In the following sections, we discuss previous use and testing of the framework, and identify suggested revisions and additions to the framework from the extant literature.

Previous applications of the framework

Applications of the ‘EVIL DONE’ framework have been limited largely to descriptive analyses. Two exceptions are Paton (2013) and St. George (2017), both theses that apply and analyse the framework using multivariate analysis. Multivariate analysis is an approach called for by Freilich et al. (2019) as a method for revising and updating of the framework. Multivariate analysis is a technique used to determine which combination of variables, from all possible combinations, can best predict something (in this case, predicting the attractiveness/vulnerability of targets). Using this technique, both Paton (2013) and St. George (2017) find a lack of support for the predictive capacity of the framework.

Among the first to utilise the framework was Boba (2008) in an illustrative crime mapping exercise. This was a simulated case study presented in the context of a hypothetical city to illustrate the capacity for ‘EVIL DONE’ to inform crime mapping capabilities for police and practitioners. However, Boba (2008) acknowledged the usefulness is limited without empirical testing of the framework on which the spatial mapping is based. This lack of testing of the foundational elements is a recurring problem with the implementation of the framework.

The first to apply the framework to analyse actual attack sites were Ekici and colleagues (2008). Their examination of terrorist targets (actual and potential) in Istanbul, Turkey in the period 1990–2006 involved the three major terrorist groups operating in the country, namely the Kurdistan Workers’ Party (PKK), the Revolutionary People’s Liberation Party-Front, and Turkish Hezbollah. Their study involved tasking Istanbul-based intelligence service officers from the Turkish National Police with rating identified targets from the perspective of each of the three respective terrorist groups. They found that the attractiveness of targets was similar across the three terrorist groups (Ekici et al., 2008). The study is limited due to the use of only six cases (comparing two sites of previous attack with four other landmarks). However, attempting to compare preferences across different groups ‘supplements the Clarke and Newman framework with an additional dimension’ (p. 131) and is an ongoing consideration in attempts to maximise predictive capacity. Application of the framework in Turkey was extended to 16 sites of previous attack by the PKK in Istanbul between 1998 and 2008. Özer and Akbaş (2011) determined that the variables of most relevance for the PKK were near, destructible, and easy. However, given the limited sample of cases, application of the variables was discussed, but not analysed empirically.

Gruenewald et al. (2015) used data from the American Terrorism Study² to assess the attractiveness and vulnerability of targets selected by environmental and animal rights extremists or eco-terrorists between 1987 and 2012. Their study operationalised the eight criteria of the ‘EVIL DONE’ framework into applicable measures of eco-terrorism targets with eight corresponding hypotheses. The study found that eco-terrorists had a preference to attack targets where access was not restricted and where the general public rarely frequented either during the day or at night. Additionally, eco-terrorists selected ‘easy’ targets that were not protected by security measures. They also suggest ‘that eco-terrorists most commonly attacked or planned to attack legitimate targets, or those targets most directly responsible for engaging in behaviours viewed as harmful to animals and the environment’ (Gruenewald et al., 2015, p. 448). However, this analysis examines each variable in isolation, which does not provide a picture of the predictive capacity of the framework overall.

Select cases have been used to illustrate that attractive targets have high scores on each of the components, suggesting that the component scores are additive (i.e. a more attractive target will have a higher total score). However, analysis of a total ‘EVIL DONE’ score is problematic as the components are not necessarily additive. For example, a location that is exposed, vital, and iconic is less likely to have no security measures in place (easy). Unlike a traditional rating scale with additive components, component scores for ‘EVIL DONE’ are more likely to reveal prioritisation of components (which ones are more important in attracting offenders).

In their illustrative exercise, all the ‘EVIL DONE’ criteria were weighted equally. However, Clarke and Newman (2006, p. 99) acknowledge that terrorists might not equally weight the target selection criteria, and this might explain ‘why the Pentagon is rated overall as a less favorable target than the White House, even though it was successfully attacked, and not the White House, on 9/11’. Further studies highlight the differential importance of the variables. However, findings regarding the most influential predictors differ across studies. Mandala and Freilich (2018) include some ‘EVIL DONE’ related factors amongst other SCP variables (e.g. weapon type) as predictors of the success of global assassination incidents between 2005 and 2014. Of all predictors included in the model, terrorist proximity to a target (related to the near variable) had the greatest predictive capacity. Using a similar approach, incorporating some ‘EVIL DONE’ factors alongside other SCP measures including weaponry and group structure, Klein et al. (2017) also demonstrate that successful incidents are more likely when the far-right offenders analysed live close to their targets. In contrast, the characteristics of occupied and easy were the most influential of the ‘EVIL DONE’ factors for participants in Australia who engaged in a red-team approach to terrorist target selection. Such an approach involved the research participants assuming the role of terrorists (Romyn & Kebbell, 2013, 2018). Both a strength and weakness of this study is the lack of ideological drive inherent in the participants – the results provide insight into what is most important for general target selection, but the impacts of genuine symbolic relevance are more difficult to determine.

Gruenewald et al.’s (2015) analysis of a single ideological group addresses a related critique of the ‘EVIL DONE’ framework – its failure to account for the ideological motivation of an offender (Freilich et al., 2019). The study demonstrates that a characteristic considered to increase vulnerability of a target (occupied) in fact appears to do the opposite when the offender at hand is an eco-terrorist. St. George (2017) also tests differential applications of the framework by offender ideology, demonstrating that certain characteristics of a target are more or less influential dependent on offender group, comparing jihadists to right-wing offenders, and those with a clear ideological motivation to non-ideological attacks (e.g. robbery, gang violence). Freilich et al. (2020) consider each of the ‘EVIL DONE’ components as they relate to public mass violence, concluding that the factors exposed, occupied, near, and easy are likely to be the most useful for identifying risk in this context, and that operationalisation of these factors should be adapted to fit the characteristics of the group.

The destructible criteria findings appear relatively consistent across the literature, though not in the expected direction. The destructibility factor assumes that the less destructible a target, the less attractive it will be. However, Gruenewald et al. (2015) find that environmental extremists tend to target locations that require destruction by an IED. Paton (2013) applied ‘EVIL DONE’ to global terrorism incidents between 1991 and 2011 and demonstrated only weak relationships between ‘EVIL DONE’ factors and vulnerability outcomes. The strongest of these was a negative relationship with destructibility (destructibility of a location is weakly and negatively related to injuries and fatalities). Furthermore, testing all predictors in a multivariate model demonstrated that destructibility alone was the only significant predictor of lethality, and no other combination of factors demonstrated significance. St. George (2017) analysed the capacity for the ‘EVIL DONE’ variables to predict lethality of attacks committed in the US between 1990 and 2014, applying the framework intended for physical targets to the characteristics of human targets. In a series of analyses, destructible was the only consistently significant predictor, regardless of whether an offender was ideologically motivated or not, and across both jihadists and right-wing offenders. Destructibility is one of the factors included in Klein et al.’s (2017) wider analysis of SCP factors predicting outcomes of far-right terrorism incidents in the United States. In the opposite direction to their hypothesis, and consistent with the results from other research, they found that less destructible targets are significantly more associated with successful terrorist incidents. Overall, locations that are more difficult to destroy appear to result in greater lethality when attacked.

The logical explanation for this finding is that offenders exert more force on a target that is more difficult to destroy, resulting in greater lethality when attacks are successful (Klein et al., 2017; St. George, 2017). Though this result is in the opposite direction to initial expectations of Clarke and Newman, the operationalisation of the dependent variables (i.e. the outcome, or what will be predicted) in these studies may explain the discrepancy. In many cases, given difficulties with the prediction of attractiveness of targets (see ‘Operationalising Vulnerability/Attractiveness’ below); lethality outcomes of previous attacks are often used as the dependent variable. In the prediction of attractiveness (i.e. whether a location is targeted in the first place), destructibility may in fact demonstrate predictive capacity in the direction originally expected.

A further consideration is whether an offender intends to have destructive impact on a location (e.g. using IEDs), or whether they target individual(s) within a location (e.g. attacking an individual with a knife). In the latter case, the destructibility of the building in which the person is housed at the time of the attack is unlikely to be a consideration (other components regarding the location or building are also intuitively less likely to be influential in this case). The ‘EVIL DONE’ framework was designed based on considerations of international, large-scale terrorism. In the current UK landscape, many individuals are attacked directly within locations (or more typically, outdoors in residential areas). This is reflected in the adapted coding system for the destructible component – attacks that occur outdoors are rated high on destructibility (i.e. nothing to destroy in order to harm the target).

Though some individual factors of the ‘EVIL DONE’ framework have demonstrated significant capacity to predict vulnerability when included in broader multivariate analyses alongside other SCP factors (e.g. Klein et al., 2017; Mandala & Freilich, 2018), multivariate analyses designed to test ‘EVIL DONE’ specifically do not bode well for the predictive capacity of the framework. In Paton’s (2013) multiple regression analysis³ (note the easy variable was not included due to unavailable information in selected data sources), destructibility was the greatest predictor of injuries and fatalities and demonstrated only a weak negative relationship (consistent with destructible findings discussed above). Like Paton, St. George (2017) does not find support for the predictive capacity of the framework. In a multivariate analysis, the ‘EVIL DONE’ vulnerability score failed to predict the number of fatalities and injured persons, and only the destructible variable was a significant predictor of lethality (consistent with Paton, greater destructibility of a target predicted greater lethality of an attack).

Inconsistencies in results across studies can be attributed to many factors, including the use of different methods of applying and analysing the framework. With multiple rating scales being adapted, the same components measured in different ways may represent different constructs. In some cases, the ‘EVIL DONE’ components are analysed in conjunction with other SCP variables to create fuller models of terrorist opportunity (e.g. Klein et al., 2017; Mandala & Freilich, 2018). Some studies examine global cases whereas others focus on specific ideological groups in specific geographic locations, highlighting the impact of ideological motivation on the findings from any one analysis (e.g. occupied being a predictor generally of vulnerability, but the opposite for eco-terrorists; Gruenewald et al., 2015). The most stable finding across the literature regards destructibility, and this is in the opposite direction to that expected by the original framework.

Alternative predictors and suggested revisions

The ‘EVIL DONE’ framework was comprised with a focus on high-impact attacks by foreign-based terrorists; appropriate for the time it was introduced (Marchment & Gill, 2022). However, terrorist strategies have changed over time in response to increased counter-terrorism capacity (Demir & Guler, 2021). The high-level focus of the current framework, and some of its key factors (vital, iconic, destructible) may be less relevant to recent attacks of low risk and on soft targets. Utilising recent empirical research, Marchment and Gill (2022) suggest an updated framework of terrorists’ spatial decision-making process that aims to account for the operational dynamics of terrorism in the current era: Tolerable, Relevant, Accessible, Close and/or Known (TRACK).

The tolerable factor assesses whether the individual can reach the point of attack without being overcome by fear/anxiety (related to risk of detection pre-attack). The relevant factor assesses whether a target is relevant to the ideology of the offender. Accessibility relates to whether the target is easy for the offender to get from their origin (i.e. via major roads). The final two factors assess whether the target is close to the offender’s home or other activity, and whether the offender knows or had awareness of the target. A location considered tolerable is one that has low situational security measures and a where this is a low risk of detection prior to the attack. A relevant target is symbolic of the ideology of the offender and selected to send a message. An accessible target is in an easily accessible area (near a major road). A target considered close is within 10 miles of the offender’s home, or 10 miles on average from the homes of a group of offenders. A known location is one at which the offender has some history (e.g. previous place of work) or where there is evidence of hostile reconnaissance.

Marchment and Gill (2022) apply the model to several recent examples of terrorism in the UK and find that most attacks studied were consistent with all elements of this framework. Though this simplified and more relevant framework is desirable, the authors’ analysis is largely descriptive. The updated TRACK model is not compared systematically to the original ‘EVIL DONE’ framework to demonstrate an improvement in explanatory capacity, and overlapping factors between models (e.g. near and close/known) are not distinguished thematically or by explanatory capacity. Though the updated TRACK framework could demonstrate improvements to the original ‘EVIL DONE’ framework by accounting for modern terrorist strategies, differences in the explanatory capacity of the variables have not been tested. The five TRACK variables are considered as potential alternatives for revision of the framework in the current study.

As an extremist ideology can impact the view of a potential target as attractive, adding ideology as an element of the ‘EVIL DONE’ framework could improve its usefulness (Freilich et al., 2019). For example, whilst an occupied target is likely to be most attractive to far right and jihadist offenders, eco-terrorists view occupied targets as less attractive (Gruenewald et al., 2015). The inclusion of relevance in the TRACK framework is an attempt to fill this gap by assessing whether a target is relevant to the ideology of the offender (Marchment & Gill, 2022). However, the dimension of relevance does not interact with other dimensions in the framework. In the example above, the dimension of occupation in ‘EVIL DONE’ should be coded in opposite ways based on ideological factors of the offender.

The weighting of variables in the framework based on ideology is a potential approach to addressing this. ‘EVIL DONE’ is limited in that it does not differentiate the importance of each of the eight included elements (Romyn & Kebbell, 2018). In testing what attributes of locations influenced preferences of study participants given the role of terrorists, Romyn and Kebbell (2013), for example, demonstrated that some attributes weigh more heavily than others when selecting a target, and multiple studies point to the significance of destructibility in predicting vulnerability and lethality (e.g. Paton, 2013; St. George, 2017). These results reflect a reasonable notion that some elements will weigh more heavily than others in terrorist decision-making. It may be useful to revise the ‘EVIL DONE’ framework to differentially weigh attributes by their importance/explanatory capacity (Freilich et al., 2019). Weighting variables according to their explanatory capacity may aid in attempts to account for ideological motivation, as weightings could apply distinctly to each of the dimensions based on the ideological grouping of the offender. In the current study, insufficient numbers of offenders outside of NI-related cases preclude comparison of variable weights by offender type (and overall weights cannot be analysed due to the data not meeting the assumptions for analysis; see Results).

Many additional SCP variables are used to predict vulnerability and attractiveness, some of which have been tested alongside some ‘EVIL DONE’ factors to create a fuller model of terrorist opportunity, which includes components of weaponry and structure of a terrorist group (e.g. Klein et al., 2017; Mandala & Freilich, 2018). However, these factors are outside the scope of the current study as they are not potential revisions to the existing framework but rather considerations for future research.

Rating scales

Subjective dimension ratings across individual researchers are a demonstrated problem in the extant literature and a threat to the validity of the research (e.g. whether a target is deemed legitimate; Romyn & Kebbell, 2013). As already noted, Clarke and Newman (2006) provided an example of rating dimensions using potential targets in Washington DC, with each factor rated on a scale of 1–5 (possible range 8–40). However, they provided no guidance for objective rating techniques and called for collaboration to refine the process. Operationalisation of the factors has been approached in different ways, though none of them has undergone a recognised process of validation.

Building on Clarke and Newman’s framework, Boba (2009) produced a set of items for each of the eight ‘EVIL DONE’ criteria that could be used by practitioners to score consistently across potential targets. Each ordinal-level index ranges from 5 (highest vulnerability) to zero (least vulnerable, with a resulting range of 40–0). To illustrate, within the exposed criteria, large high-rise structures in an urban area (e.g. the Washington Monument in the US or The Shard in the UK) would rank as a ‘5’ in Boba’s index, while a cluster of buildings in an urban area (e.g. a university campus) would rank as a ‘1’, namely a low exposed status. While Boba’s work provided a system for individual target assessment of terrorism risk it did not apply the methodology to assess the attractiveness and vulnerability of targets selected by terrorists. However, Boba’s rating scale has been utilised in subsequent studies including Ekici et al. (2008) and Paton (2013) and is the most established of the existing scales. Gruenewald et al. (2015) created new indicators for rating each factor, most of which are binary (e.g. target was occupied or unoccupied) or comprised of three to four categories. These ratings were developed to be specifically relevant to eco-terrorism for the purposes of the study and have since been adopted by others (including Klein et al., 2017) and adapted by St. George (2017) for application to far right and jihadist offenders. In addition to rating the ‘EVIL DONE’ factors, Boba included expected loss in the overall assessment of vulnerability based on Newman and Clarke’s (2008) method of evaluation. For the current study, actual loss and injury from previous incidents functions as the dependent variable.

As discussed, the same ‘EVIL DONE’ components measured in different ways may represent different constructs. For example, Gruenewald et al.’s (2015) measure of exposure used to analyse eco-terrorist decision-making is scored based on accessibility to the public, whereas Boba’s (2009) rating of the same component relies more on type and visibility of the building, and whether it is in an urban or rural area. Rating of the vital component is based on how critical a particular type of target is to a community (Boba, 2009), compared to whether day-to-day operations were significantly affected in the aftermath of an attack (Gruenewald et al., 2015). To rate the occupied component, Boba considers the use of the target space across time on a five-point scale, ranging from only seasonal use to housing many people 7/24. This long-term measure does not reflect the occupancy status at the time of the attack, which is likely to have an impact on decision-making. In contrast, Gruenewald et al. (2015) focuses only on the occupied status at the time of attack with a simple binary measure (occupied or not).

Paton (2013) adapts Boba’s rating of the near variable to account for the unavailability of some relevant data. Unlike most of the factors, which are scored from 0 to 5 based on the selection of the most appropriate option, the near variable is based on a sum of five possible criteria. For many targets, information was lacking regarding two of the criteria (target being close to the location of known/suspected terrorist bases, and close to domestic immigrant communities), resulting in inconsistent scoring for this factor. Paton excluded these two elements, resulting in a maximum score of 3 rather than 5 for this factor. Since scales in circulation, including Boba’s, have no demonstrated validation processes, adapting the scales to suit specific analyses has no demonstrable impact on their validity.

For the current study, it is necessary to adapt the existing ‘EVIL DONE’ rating scales to reflect the current UK threat landscape, particularly given the heavy representation of domestic terrorist incidents that diminish the relevance of factors like close immigrant communities, for example (Boba’s near factor). Table 2 presents the items used to rate each component in the current study. The resulting scale is essentially a set of eight Likert measures, each with three levels, representing the extent to which each of the factors is relevant to a case (significant, somewhat, not at all). As with existing measures, some subjectivity will be necessary to deduce scores for some components. Inter-rater reliability scores are examined to ensure consistency in the conception and implementation of the scale across independent raters.

Marchment and Gill (2022) illustrate the relevance of the five TRACK factors by demonstrating their relevance to six terrorist incidents in the UK between 2013 and 2018. Each factor is operationalised as a binary measure (yes/no). Marchment and Gill provide a descriptive illustration that all or most of the five TRACK factors are present at each of the six attack sites, but they do not provide a scale of measurement or calculation of a total TRACK score.

Given the nature of the UK threat landscape, it is feasible to include two of the TRACK factors as alternative predictors in the current study (tolerable and relevant). Particularly for cases attributed to Northern Ireland (NI)-related terrorism (a large proportion of incidents), we often have an attribution or connection to a ‘group operating in the area’ rather than an offender and offender location. Therefore, the measures of close and known would not be able to be operationalised for most cases and would likely result in issues of multicollinearity with the component of near if they were able to be rated (i.e. a situation where two predictor variables are so highly correlated that one can be used to predict the other, skewing the results of a regression model).

For the current study, the highest score for near includes attribution to a group operating in the area in addition to individual offenders living within a 10-mile radius of the target. Given the heavy proportion of domestic incidents, the accessible component of TRACK (based on accessibility of the target via major roads) is also unable to be operationalised separately from the near component as most attacks are taking place within local areas and we often do not know the initial location of offenders. Though tolerable and relevant are related to the ‘EVIL DONE’ components of easy and iconic respectively, Table 2 highlights the differences in their operationalisation for the current study. Tolerable reflects the risk of detection pre-attack, whereas easy relates to risk of detection or obstruction during an attack. Iconic reflects the iconicity of a target to the targeted group or adversary (and the likelihood its destruction will dishearten the victimised community), whereas relevance refers to explicit symbolism or claims by the offender(s) regarding their ideological motivation.

Operationalising vulnerability/attractiveness

The attractiveness and vulnerability of targets are the dependent variables for the analyses (i.e. the outcomes, or what will be predicted). Multiple approaches have been used in the literature. Options for the representation of attractiveness/vulnerability include whether a location has been attacked vs. not attacked (e.g. Ekici et al., 2008) or number of fatalities and injuries at sites of attacks (e.g. Paton, 2013). The binary of attacked vs. non-attacked highlights attractiveness (useful for thwarting/prevention) and is consistent with the intended use of the frameworks. Analysis of lethality pertains more to harm reduction and mitigation.

The former approach allows examination of which targets are most attractive to terrorists but requires determination of a ‘control’ set of non-attacked locations that are comparable to attack locations, or inclusion of all locations in an area (both attacked and non-attacked) for comparison. This approach has been used with a small number of targets (e.g. two attacked locations compared to four non-attacked locations; Ekici et al., 2008) but is not easily scalable to include hundreds of cases for a multivariate analysis. The selection of control (non-attacked) cases is likely to be based on a predictive factor in the analysis (e.g. other iconic locations in an area, or locations nearby to attack sites), creating sampling bias. However, the selection of controls is necessary, as it is not feasible to include and code near infinite non-attacked locations to examine a large area. An approach taken by St. George (2017) is to compare sites of ideologically motivated attacks to sites of non-ideologically motivated attack (e.g. robbery). Though this approach limits the dataset to a defined number of cases, it is a comparison of terrorist targets and non-terrorist targets, rather than an examination of targets most at risk from terrorism.

The latter approach (utilising lethality to represent vulnerability in the case of past attacks) has been used to analyse attacks in Turkey, the US, and globally. In these cases, lethality has been operationalised as a continuous variable (number of fatalities and injuries; Paton, 2013) or a binary variable, in which only lethal attacks are examined and cases resulting in a single death are compared to those resulting in two or more (St. George, 2017). In the current UK threat landscape, many attacks do not result in injuries or fatalities (see Table 3). An appropriate operationalisation of vulnerability for the current study is an ordinal variable (i.e. made up of categories that occur in a specific order) with three levels: incidents resulting in no injuries or fatalities, incidents resulting in injury but no fatalities, and incidents resulting in fatality.

Some components of the framework, namely iconic and legitimate, are logical factors in the prediction of attractiveness (i.e. whether a location will be targeted by terrorists) but seem intuitively less relevant to the prediction of lethality in the event of an attack. It is possible that these factors may have indirect effects on lethality, for example, by increasing the motivation of the offender, but these indirect relationships are yet to be explored. We retain all components in a multivariate analysis predicting lethality but acknowledge these limitations of the dependent variable. The components of iconic and legitimate are examined most practically via an examination of the frequency with which they are present at sites of attack in the UK (a descriptive measure of attractiveness).

As discussed, accounting for offender ideology in the framework could improve its usefulness (Freilich et al., 2019; Gruenewald et al., 2015). With adequate numbers of offences, St. George (2017) compares attacks by right-wing offenders and jihadist offenders in the United States. However, in the current UK threat landscape, NI-related terrorism features heavily, and no other group is responsible for adequate numbers of attacks to allow for a comparison (see Table 1) in the current study.

Table 1. Distribution of offender ideology.

Offender group	No. of cases	%
Northern Ireland-related terrorists*	162	88
Jihadists (of which Islamic State = 7)	11	6.0
Far Right	8	4.3
Eco-terrorists (Individualistas Tendiendo a lo Salvaje; ITS)	2	1.1
Far Left/Anarchists	1	0.5

*Separatists/Nationalists, IRA, Continuity IRA, New IRA, Real IRA, Óglaigh na hÉireann (ONH), Ulster Defence Association (UDA), Ulster Volunteer Force (UVF)

Aims and research questions

The ‘EVIL DONE’ framework has received limited empirical investigation, and the current UK threat landscape has not been investigated using the framework. The aim of the current study is to examine the relevance of the ‘EVIL DONE’ terrorism target risk assessment framework to reflect the current operational dynamics of terrorism and support protective efforts in publicly accessible locations in the UK.⁴ This study builds on Marchment and Gill’s (2022) descriptive illustration of the TRACK framework using several UK cases of attack by considering the TRACK variables as potential alternatives or additions to the framework. This is believed to be the first study to incorporate alternatives to empirically assess the framework and provide a better understanding of target attractiveness and vulnerability. Two core research questions are addressed:

RQ1. Vulnerability in the event of an attack: Do the ‘EVIL DONE’ (+TR) variables demonstrate strong relationships with lethality?

RQ2. Attractiveness of targets: Which of the components are consistently present or absent across cases (i.e. attractive to, or avoided by, offenders)?

Methods

Cases

The database used for the current study is Dragonfly’s TerrorismTracker,⁵ a database for all global terrorist incidents reported in open sources since 2007. This database was chosen to represent the current threat landscape in the UK as the source is kept up-to-date and case statistics include relevant details of the target, perpetrators, and affected sectors. Cases from 2015 through 2021 are included.⁶ Failed plots are not included due to information often missing regarding specific target locations. Dragonfly uses the following definition of terrorism to make judgments about what incidents and plots are included in the TerrorismTracker (n.d.) database:

Terrorism is the systematic threat or use of violence, whether for or in opposition to established authority, with the intention of communicating a political, religious or ideological message to a group larger than the victim group, by generating fear and so altering (or attempting to alter) the behaviour of the larger group … Either the victim or the perpetrator, or both, will be operating outside a military context; both will never be operating within a military context or in a state of military hostilities at the given instance.

For the current study, cases in which the offender/offending group is unknown (n = 64) are excluded from the analysis as many of the components to be rated depend on knowledge of the offender. Cases of hoaxes (typically fake explosive devices) are also excluded from the analysis due to difficulty coding and incompatibility with the dependent variable of lethality (n = 61). A total of 184 cases are retained.

Note that multivariate analyses were initially planned to determine the predictive capacity of a model with the ‘EVIL DONE’ +TR components as predictors of lethality. However, a lack of diversity in the scoring (due to many similar NI-related cases) resulted in 64.5% of potential unique combinations of scores not existing in the dataset (from 234 possible cell combinations, there are 83 in the dataset). In this case, goodness-of-fit statistics are inappropriate to assess and confidence intervals for the odds ratios are abnormally wide and unreliable (de Irala et al., 1997). Research questions for the current study were adapted to the capacity of the available dataset.

Measures

‘EVIL DONE’ item ratings are adapted for the current UK threat landscape, as previously discussed. The resulting scale is a set of eight Likert type measures, each with three levels, representing the extent to which each of the factors is relevant to a case (significant, somewhat, not at all). The TerrorismTracker database includes news sources for information about each case. These sources are utilised in addition to images, maps, and reports found online to give further information about the location and perpetrator. The range of possible scores for each component is 0 to 2. The total ‘EVIL DONE’ score for any site will range from 0 to 16 and the total score including the additional variables of tolerable and relevant will range from 0 to 20.⁷ Table 2 presents the rating scale and sources from which items are adapted.

Table 2. Rating scale for components.

	Measure	Adapted from
Exposed	2. Victim(s) in a location that ‘stands out’ (one of the larger or more recognisable sites in the town or city it is in – e.g. tourist attraction, highrise building, major rail station) or is the site of a recognisable event at the time of attack (e.g. parade) 1. Victim(s) in a location that is relatively visible and known, but doesn’t stand out as especially recognisable (e.g. shopping centre, restaurant, police station, police car, public park) 0. Victim(s) in a location that is not publicly recognisable and likely inaccessible to public without permission (e.g. private residence, private car), or at a random location on a street/residential area	St. George (2017); Boba (2009)
Vital	2. Victim(s) in a location that if attacked/eliminated would have a great or long-term impact on the day to day functioning of the community (e.g. paralysing transport, destroying power sources) 1. Victim(s) in a location that if attacked/eliminated would have some impact on day to day functioning of the community (e.g. food distribution, police station, church) 0. Victim(s) in a location that if attacked/eliminated would have little to no impact on day to day functioning of the community (e.g. individual supermarket, police vehicle, private home)	Gruenewald et al. (2015); St. George (2017); Boba (2009)
Iconic	2. Location of attack is a major symbol of the target/adversary’s identity and strength – an attack would likely dishearten the target group on a broad scale (e.g. international terrorists target nationally recognised site or monument). In the case of local groups (e.g. IRA), a police station or opposing political party would be examples of iconic targets 1. Location of attack is related to the identity of the target/adversary, but not a major symbol (e.g. international terrorists target local meeting hall of a political group). In the case of local groups (e.g. IRA), individual police officers or police patrols would fit this category. 0. Location of attack is unrelated to the identity of the target/adversary (random choice of location), e.g. residential street or private home	Clarke & Newman’s original (2006) definition of the component
Legitimate	2. Location occupied/owned by only those directly related to or furthering the cause of the target adversary (e.g. targeting military personnel at a military base), or a single victim is specifically targeted in a private space with no risk of harm to others (e.g. shot at close range) 1. Location occupied/owned by those directly related to or furthering the cause of the targeted adversary, as well as those not related to the target adversary (e.g. anti-abortion extremists target staff at an abortion clinic that also houses other clients), or specific victim is in public area/ home where others may be at risk (e.g. from bomb exploding) 0. Location occupied/owned by only those not directly related to the targeted adversary, including private property and citizens (choice appears random)	Gruenewald et al. (2015); St. George (2017); Boba (2009)
Destructible	2. Victim(s) in a location that is easily destructible by weapons such as guns and homemade pipe bombs (attack occurs outdoors, or small structures such as sheds, cars) 1. Victim(s) in a location that is moderately destructible – would require small IEDs (private residences and small businesses/buildings) 0. Victim(s) in a location that is difficult to destroy -requires large weapons or equipment to destroy (concrete buildings, multi-story buildings, and large structures)	St. George (2017)
Occupied	2. Occupied and potentially crowded/busy – frequented by the public or large groups of people (e.g. military base, hospital, shopping centre, restaurant) 1. Occupied, but not crowded (e.g. private offices, police station), or target is a specific individual(s) 0. Unoccupied at the time of attack	Gruenewald et al. (2015); Boba (2009)
Near	2. (At least one) offender lived close (i.e., in the same city as the target, or ∼10 miles or less from the target), or the attack is linked to groups ‘in the region’ 1. (At least one) offender lived within 100-mile radius of target 0. Offender(s) lived over 101 miles from target	Gruenewald et al. (2015)
Easy	2. No security measures 1. Some security (passive security, including CCTV) – level of security measures that would be expected at a public location like a shopping centre or a personal home (e.g. locked doors), or victim(s) travelling in a car 0. High security (more active security, including security teams, screening checks) – includes security measures that would seem atypical/extreme for a public location like a shopping centre	St. George (2017)
Tolerable	2. Offender(s) very likely to reach the point of attack without being detected, and without inducing fear and anxiety related to detection (would not raise suspicions or attention pre-attack). E.g. attack in a park using a concealed knife or gun 1. Offender(s) are less likely to reach the point of attack without being detected (e.g. made a bomb or stored suspicious items at home in advance, had to pass through areas where they might raise suspicion prior to attack, might be seen attaching a device to a vehicle) 0. Offender(s) are not likely to reach the point of attack without being detected, and without inducing fear and anxiety related to detection (e.g. must pass through screening procedures at an airport with concealed weapons)	Marchment & Gill (2022)
Relevant	2. Offender is explicitly motivated by ideological means and provides evidence or makes claims regarding this (e.g. shouts rhetoric or flies flag during an attack; group publicly claims an attack after it occurs) 1. Target appears relevant to the offender or offending group (e.g. police station to suspected IRA attack, specific individual/group/location clearly sought out by offender/s), but there is no explicit evidence of ideological motivation provided by offenders during the attack and no public claim to the attack during or after the event 0. Target (location/victim) unrelated to the ideology of the offender (appears to have been chosen at random), including placing explosives randomly in residential areas where they may hit targets unrelated to offender ideology	Marchment & Gill (2022)

Dependent variable

The dependent variable (DV) was operationalised as an ordinal variable with three levels (i.e. lethality score based on injury and fatality was converted to three categories that reflect increasing levels of lethality). See Table 3 for a breakdown of its distribution.

Table 3. Distribution of the dependent variable.

Lethality (DV)	Number of cases
No injury or fatality	118
Injury, no fatality	52
Incidents involving fatality	14

RQ1

Bivariate correlations (i.e. a statistical technique used to determine the existence and strength of relationships between two different variables) are used to examine the relationships between each of the components and lethality outcomes to determine which variables are significantly related to lethality. Correlations are examined between each of the components and the ordinal DV, as well as with lethality, a continuous DV. The continuous measure of lethality is computed by adding total number of injuries to number of deaths multiplied by 20 to represent the severity of fatal outcomes (from Paton, 2013). This is an imperfect measure that allows for relationships to be examined.

RQ2

The distributions of component scores are analysed to determine which components appear to be attractive to, and avoided by, offenders.

Results

Inter-rater reliability

For the current study, the reliability analysis was calculated based on weighted kappa (κ_w)⁸ scores for a sample of 50 cases, selected to cover a variety of cases. According to Altman’s (1991) classification scale, inter-rater agreement was ‘very good’ for all components, except for the legitimate factor, for which agreement was ‘good’. The κ_w scores for the current study are displayed in Table 4.

Table 4. Weighted kappa score by component.

Component	κw
Exposed	0.855
Vital	0.926
Iconic	0.907
Legitimate	0.756
Destructible	0.895
Occupied	0.921
Near	0.962
Easy	0.949
Tolerable	0.850
Relevant	0.928

Descriptive statistics

Summary statistics for the 184 cases in our dataset reveal that total scores range from 6 to 18, with scores of 8 and 11 featuring heavily (see Figure 1). The bimodal distribution (i.e. two distinct peaks of commonly occurring values in the dataset) likely reflects the similarity of many NI-related cases (rather than a spread of cases with distinct attributes).

Figure 1. Total score (/20) for ‘EVIL DONE’ + TR.

Figure 2 shows the distribution of scores (from 0 to 2) for each of the ten components. Most cases are rated 0 for exposure, vitality, and iconicity. This is consistent with the fact that most NI-related cases happen in local, unremarkable areas (residential areas, roads, private homes). The components of legitimate, occupied, tolerable, and relevant have modal scores of 1. For legitimate and occupied, these scores are due to a large proportion of cases in which individuals at local sites, rather than crowds or busy locations, are the target, i.e. little risk of harm to non-intended (non-legitimate) targets. There are often some risks associated with making and transporting explosives, but given they act mostly in their local areas, there are little external security measures between an offender and their target location and little risk of being caught pre-attack (tolerable). Attacks are mostly attributed to NI-related groups without offenders ever being prosecuted or providing explicit evidence of their motivation (relevant). The components of destructible and easy are the only two that demonstrate modal scores of 2, as would be consistent with the expectations of the ‘EVIL DONE’ framework. Most attacks happen outdoors in unsecured areas or target private homes and cars.

Figure 2. Distribution of scores by component.

RQ1

Two sets of tests are used to examine the relationship between each of the components and lethality outcomes (see Table 5): (1) lethality as a continuous variable (using Kendall’s Tau-b for examining one ordinal and one continuous variable); (2) lethality as an ordinal dependent variable (using Somers’ delta for examining an association between two ordinal variables, distinguishing between a dependent and independent variable).

Table 5. Kendall’s Tau-b (continuous DV) and Somers’ d (ordinal DV) correlations between component scores and ordinal DV.

Component	Kendall's Tau-b	α	Somers’ d	α
Exposed	.060	.384	.042	.630
Vital	.022	.755	-.012	.909
Iconic	-.073	.287	-.105	.185
Legitimate	.299**	<.001	.285**	<.001
Destructible	.091	.186	.113	.116
Occupied	.243**	<.001	.262**	.004
Near	-.155*	.027	-.320	.159
Easy	.232**	.001	.249**	<.001
Tolerable	.546**	<.001	.604**	<.001
Relevant	.216**	.003	.286**	<.001

**. Correlation is significant at the 0.01 level (2-tailed).

*. Correlation is significant at the 0.05 level (2-tailed).

Five of the components are significantly related to both the continuous and ordinal DV (legitimate, occupied, easy, tolerable, and relevant), though only tolerable demonstrates a correlation greater than 0.3. In theory, greater legitimacy scores denote a more targeted attack. For the current dataset, most cases have legitimacy and occupied scores of 1 due to many cases of individuals being targeted in public areas with few others at risk. In cases where an individual is directly targeted by an offender, injury or fatality is likely. This interpretation differs from previous uses of the framework under which targeting individuals might result in less lethality than targeting large and crowded sites (very rare occurrences in the current database). Sites with less security (easy) tend to result in greater lethality, as expected by the original framework.

The relatively strong positive correlation for the tolerable variable at first appears to suggest that offenders with less pre-attack risks attack with greater lethality. However, the raters note that tolerable scores were dependent on type of weapon used, with the use of explosives being rated less tolerable than the use of knives, guns, and other direct physical means, due to the need to gather materials, store, and place explosives in public areas. As direct attacks with guns and knives are more likely to result in lethality than homemade explosives placed in residential areas (typical of this dataset), this relationship between tolerable and lethality is likely confounded by weapon type.

Relevance, as scored in the current study, relates to explicit relevance of the attack to an offender’s ideology. Greater relevance is associated with greater lethality, which may suggest that offenders motivated to express their ideological incentives also have the greatest motivation to attack and the greatest capacity for lethality. However, relevance scores of 0 are associated with different types of attacks. Undirected⁹ attacks that appear to target random sites or populations (typically by leaving explosives) typically involve no ideological claims by individuals or groups and so score 0 for relevance. However, these sorts of undirected attacks are also those that tend to result in less lethal outcomes due to the absence of direct targeting. Direct targeting (the seeking out of individual targets or specific locations) denotes some relevance of the victim to the offender’s ideology, and direct attacks by their nature are also more likely to result in lethality. Interpretation of these relationships must take account of these potential confounds.

The near component demonstrates a negative relationship with the DV (those who travelled further had more lethal outcomes). This result might reflect greater care in attack planning for those who travel greater distances whilst more spontaneity (i.e. limited preparation) is likely for local offenders. However, care must be taken in interpreting this correlation as the near component is rated 2 for almost all cases (targeting individuals and local sites). The non-significance of the destructible component may be explained by the fact that many attacks target individuals via direct means (e.g. guns, knives) rather than aiming to destroy the buildings that house them, and many attacks occur outdoors where there is no site to be destroyed.

RQ2

A lack of diversity in the scoring provided interesting insight into the attractiveness of targets in the current UK terrorist threat landscape. Most of the 83 existing cell patterns (combinations of scores across IVs and the DV) applied to a single case, with limited patterns applying to multiple cases, up to a maximum of five. Demonstrating a clear exception, three patterns each applied to more than 20 cases in the dataset (see Table 6). These three patterns illustrate the factors that are present for most attacks in the UK and span two levels of the DV, providing insight into distinctions between modal patterns that are associated with injury (DV 2) as opposed to none (DV 1). Note that each of these patterns adds to a total score of 8 or 11, consistent with modes demonstrated in Figure 1.

Table 6. Modal cell patterns.

Ex	V	I	L	D	O	N	Ea	T	R	DV	Score	No. of cases
0	0	0	0	2	1	2	2	1	0	1	8	30
0	0	0	1	1	1	2	1	1	1	1	8	22
0	0	0	1	2	1	2	2	2	1	2	11	27

Five of the ten components have common scores across all modal cell patterns – exposed, vital, iconic, occupied, and near. Modal patterns share exposed, vital, and iconic scores of 0, occupied scores of 1 (targeting individuals or smaller sites), and near scores of 2 (perpetrator living in local area), consistent with many local attacks that target individuals and small sites.

The first modal cell pattern in Table 6 is typical of an undirected or random attack in an outdoor/ unsecured area (destructible = 2) using explosives (tolerable  = 1), with no clear target, no security, and no clear relevance of the target to the ideology of the offender (legitimate = 0; easy  = 2; relevant  = 0). Typical of this rating is an IED left randomly in a residential area. The second pattern is similar but reflects cases in which an individual is more clearly targeted (legitimate and relevant  = 1) using explosives (tolerable  = 1) in a more secure area (destructible and easy  = 1). Typical of this rating is an IED targeting an individual in their home or targeting police officers (typically considered less tolerable than random targets).

The third cell pattern is typical of direct targeting of an individual (as in pattern 2) in outdoor/unsecured areas (as in pattern 1). The distinction between this cell pattern and the others is a tolerable score of 2 which reflects the use of concealable weapons, like guns and knives, or the use of direct physical force (tolerable = 2). Unlike the other modal patterns, this one is associated with injury. As discussed, tolerability is related to weapon type, and the use of concealed weapons to target an individual directly is likely to result in injury.

Overall, three typical attack types make up most cases in the current UK landscape: (1) undirected or random attacks with no clearly identified victim, typically involving the random placement of explosives; (2) individuals targeted in private areas using IEDs; (3) individuals targeted directly in outdoor/unsecured areas. The latter is associated with injury.

Post-Hoc analysis of non NI-related cases (Great Britain)

The UK results are heavily skewed by NI-related cases and point to the unsuitability of the framework overall. However, it is possible to look at cases from Great Britain (GB) only (excluding NI) to understand the components and their relationship with lethality for other cases of terrorism (see Table 1 for distribution of offender ideology). The minimum score for this sample is 10 (compared to 6 for the UK), giving an initial indication that the framework is more relevant.

Tables 7 and 8 show the distribution of the DV, and correlations between the DV and each component. Exposed and occupied demonstrate significant positive correlations, and near demonstrates a significant negative correlation, with lethality (measured both ordinally and continuously). These results suggest that locations which are more exposed and more occupied result in greater lethality in the case of an attack, consistent with the ‘EVIL DONE’ framework. The negative result for near suggests that those who travel greater distances to commit attacks are the most lethal. However, as in analysis of the UK statistics, this must be interpreted carefully as almost all cases scored 2 for near (offender/s lived in the area). This result is based on only five offenders who travelled outside of their local area to commit attacks (and only two who travelled more than 10 miles; see Figure 4). Each of the significant correlations is moderate in size.

Table 7. Distribution of the dependent variable (GB).

Lethality (DV)	Number of cases
No injury or fatality	6
Injury, no fatality	7
Incidents involving fatality	9

Table 8. Kendall’s Tau-b (continuous DV) and Somers’ d (ordinal DV) correlations between component scores and ordinal DV (GB).

Component	Kendall's Tau-b	α	Somers’ d	α
Exposed	.484**	.007	.333*	.020
Vital	.296	.097	.140	.395
Iconic	.272	.126	.138	.511
Legitimate	-.203	.259	-.148	.496
Destructible	-.208	.243	-.070	.697
Occupied	.600**	.001	.519**	.005
Near	-.403*	.028	-.560*	.012
Easy	.086	.635	.241	.216
Tolerable	.060	.745	.252	.184
Relevant	.032	.863	-.097	.751

**. Correlation is significant at the 0.01 level (2-tailed).

*. Correlation is significant at the 0.05 level (2-tailed).

The non-significance of the remaining seven components does not exclude them as useful considerations for attractiveness. As discussed, operationalising the dependent variable as lethality of previous attacks is due to limitations of the current dataset. Though these results provide insight into locations that may be most vulnerable in the event of an attack, they do not necessarily account for attractiveness of the targets in the first place.

In the UK context, three modal cell patterns provided insight into attractiveness by illustrating typical targets/attack types. Figures 3 and 4 show the distribution of total scores, and the distribution of scores by component, for GB. With the exception of the near and relevant components (which both demonstrate modal scores of 2), there is meaningful variability in scores for each component, and the distribution of total scores is normal. Examination of the component distributions (Figure 4) reveals which of the components tend to score high or low, providing insight into attractiveness.

Figure 3. Total score (/20) for ‘EVIL DONE’ + TR (GB).

Figure 4. Distribution of scores by component (GB).

Figure 4 reveals that attacks in GB are much more likely (compared to NI) to occur in locations that are exposed, vital, and iconic (consistent with the ‘EVIL DONE’ framework), though for each of these components, approximately one-third of cases score zero. In fact, it is the cases that lack scores of zero that can provide the most insight into target attractiveness. Whilst the distinction between a score of 1 and a score of 2 is relatively subjective (i.e. based on rating scales adapted for the current study), scores of zero are more readily interpretable as they indicate a clear absence of the component being measured. Components lacking scores of zero are those that score consistently in line with expectations of the framework (i.e. attacks do not occur in the absence of these components).

Though many NI-related cases scored zero for relevance (target appears to have been chosen at random), there are no GB cases scoring zero for this component, indicating that offenders consistently seek locations/victims that align to some extent with their ideological affiliation. Almost all cases (18 of 22) scored 2 for relevance, meaning that almost all offenders were not only explicitly motivated by ideology, but also provided evidence or made claims regarding this. Only the single case of eco-terrorism (see Table 1) scored zero for occupied (meaning the location was unoccupied at the time of attack), and this is consistent with previous research that indicates eco-terrorists target unoccupied areas (Gruenewald et al., 2015). Consistent with the framework, all other offenders targeted at least one victim (and in half of cases, targeted busy/crowded areas).

Like NI-related cases, very few cases score zero or 1 on the near component, reflecting that most offences took place in the offender’s local area (consistent with the ‘EVIL DONE’ framework). The easy component also demonstrates a skew towards scores of 2 with very few cases scoring zero, indicating that offenders are attracted to locations with little to no security measures (consistent with the framework). The legitimate and destructible components also demonstrate few cases scoring zero (with a more even spread of cases scoring 1 and 2), indicating that offenders avoid locations that house random populations unrelated to their ideological motivation (consistent with scores for relevance) and that they avoid locations that are relatively indestructible. Both results are relatively consistent with expectations of the framework.

Overall, the ‘EVIL DONE’(+TR) framework appears to fit much better in the analysis of attractiveness for GB cases only. A lack of zeros and a skew towards higher scores indicate that offenders tend to avoid locations that are relatively indestructible, unoccupied, far away, highly secure, likely to expose them pre-attack, or irrelevant to their ideological motivation. A greater spread of scores for the remaining four components (exposed, vital, iconic, and legitimate) indicate that these components may be less helpful in determining typical characteristics of attractiveness across offenders. In addition, correlations between component scores and the DV demonstrated that locations that score higher on exposed, occupied, and near tend to result in greater lethality when attacked.

Discussion

The aim of this study was to examine the relevance of the ‘EVIL DONE’ framework (Clarke & Newman, 2006; 2007) to the current UK threat landscape by analysing the predictive capacity of the components, including additions from the TRACK framework (Marchment & Gill, 2022), to determine suggested revisions to the existing model. A lack of diversity in the scoring (due to many similar NI-related cases) precluded a multivariate analysis but provided three modal cell patterns (unique combinations of scores) that illustrate the three typical attack types comprising most cases in the current UK landscape. One of the three modal cell patterns included injury to victims whilst the other two were associated with no injury. The modal cell pattern involving injury was distinguished by a high tolerability score reflecting the use of direct physical force or concealable weapons in attacks on individuals in unsecured areas.

There are many indications that the framework is not suitable for the current UK threat landscape, which is comprised predominantly of attacks on a local scale. Since the framework and associated rating scales were created to examine terrorism on an international scale, it was necessary to adapt available rating scales to operationalise locally and with individuals included as targets in addition to locations. Even after doing this, many of the components demonstrated little variability across cases (legitimate, occupied, near) with some scoring zero on almost all occasions (exposed, vital, iconic). Some components must be interpreted carefully, given the current rating scale. For example, when applying the destructibility measure to cases in which a victim is targeted in an outdoor area, a high score indicates that there were no obstacles to destroy to reach the victim (not that the victim is in a highly destructible building). These necessary adaptations to the rating scale mean significant relationships between component scores and lethality must be interpreted with caution, as discussed in Results (RQ1). The tolerable component demonstrates the strongest correlation with lethality (RQ1) and is the only variable in the three modal cell patterns to distinguish cases involving injury from cases involving no injury (RQ2). Taken together, these results would appear to suggest that offenders with less pre-attack risks are more likely to harm victims. However, we know that tolerability in the dataset was largely dependent on type of weapon used, with explosives rated less tolerable that other concealable weapons given the need to gather and store materials etc. This confounds the interpretation of the tolerable component as direct attacks with guns and knives are more likely to result in lethality than homemade explosives placed in residential areas. This result suggests that components of the framework may benefit from integration with wider SCP measures including weapon type, an approach taken by Mandala and Freilich (2018) and Klein et al. (2017).

The dependent variable of target attractiveness/vulnerability was operationalised using number of injuries and fatalities at sites of previous attacks (e.g. Paton, 2013) rather than sites of attack compared to no attack (e.g. Ekici et al., 2008) as the latter approach is not easily scalable. Though a comparison of attack and non-attack sites appeared to be the gold standard in determining attractiveness, the results of the current study highlight that it may not be helpful in examining the current UK threat landscape. Many of the attacks are based on individuals attacking other individuals outdoors in residential areas or targeting private residences. In these cases, offenders do not appear to be logically weighing up the costs and benefits of attacking particular buildings as assumed by SCP theory and the ‘EVIL DONE’ framework. In reality, local characteristics including history of the surrounding area and differences in religiosity of the residents are likely to be impactful, but difficult to account for using the framework.

Given the local and seemingly undirected or random nature of terrorist activity in the UK (mostly driven by NI-related terrorism), ‘EVIL DONE’ may not be the best approach to predicting and mitigating the threat. Analysis of terrorist activity on a local scale may require examination of more local details including characteristics of the victims and perpetrators, as well as the types of residential areas and homes most at risk. However, this is a unique case. The untested efficacy of the framework remains an issue for prioritisation in other contexts where operationalisation of attractiveness as sites of attack versus non-attack on a large scale would still be considered the gold standard. Though SCP and ‘EVIL DONE’ have not demonstrated utility for UK terrorism incidents generally, they may be useful in predicting rare cases of more traditional (large-scale and/or international) terrorism in the UK as in other contexts.

Analysing GB cases only (excluding NI) provided a small sample to examine this possibility using descriptive statistics and correlations between each component and lethality. For GB cases only, the framework appears to fit much better in explaining the attractiveness of targets. Six of the ten variables tested (DONE + TR) skew towards higher scores and demonstrate few scores of zero, indicating that offenders tend to avoid locations that are relatively indestructible, unoccupied, far away, highly secure, likely to expose them pre-attack, or irrelevant to their ideological motivation. The remaining four components (EVIL) demonstrate a greater spread of scores, suggesting less usefulness in understanding the characteristics of locations that are avoided by, or attract, offenders. Correlations between component scores and lethality outcomes demonstrated that, in the event of an attack, locations that are more exposed and occupied, and attacks that require the offender to travel further, are associated with greater lethality.

Limitations and directions for future research

Whilst the current framework is not suitable for NI-related cases, modification of some components could improve their suitability in this context. Given the local nature of offences, and the tendency for individuals to be targeted (rather than locations), greater consideration must be given to intended impacts on people. For example, the vital and destructible components could be reconstrued as two impact components covering both location and victim – just as operationalisation of vulnerability for victims relies on loss of life and injury, physical location-specific impact would consider degree of loss of functionality. Adaptations such as this would allow separation of location and victim, helpful in the unique context of NI. If the individual is the primary target, their location will influence their vulnerability and offender opportunity. If the location is the primary target, the individual is secondary and influenced instead by their proximity to the location. However, limitations of the available data make it difficult to deduce whether a specific individual was targeted as opposed to a random individual within the proximity of the attacker (attacks are often attributed to an offending group without knowledge of the specific motivations of the offender).

Retrospective data analysis also limits the interpretation and inclusion of components. Data regarding the near component, for example, is limited to residence of the offender or location of the offending group. However, near may also be considered in terms of the virtual space. The virtual space is a consideration of the ‘known’ component of the TRACK framework (a known location is one at which the offender has some history, including where there is evidence of hostile reconnaissance). The tactic of virtual hostile reconnaissance enables the enactment of activity as though it is ‘near’. Such virtual attack planning is a modern phenomenon that, when used effectively, is likely to overcome traditional security measures (Holbrook, 2015; Schuurman et al., 2018). Particularly in the absence of a known offender, this pre-attack information cannot be considered (we could not include the known component in the current study). It is not possible to consider whether the intended target is different from the attacked target, or whether other targets may have been considered.

Analysis of GB cases, for which the framework was more relevant, was limited by the sample size. Placing these cases in a wider context (including international cases) would increase the sample and allow for predictive analysis. A wider sample would also allow for additional variables to be included, for example, the threat level at the time of an attack could provide additional context regarding security posture and vulnerability (there was not enough variability to consider threat level for the period analysed in the current study). However, limitations of the current data would remain even with a wider sample. These include issues with operationalising the dependent variable of attractiveness and utilising retrospective rating techniques to make determinations, including deducing motivation and intent of an offender, without all the relevant information.

Experimental methods may be used to overcome these limitations by providing the capacity to control for and systematically modify components to measure their impact on attractiveness. As discussed, a red-team approach to testing the framework has begun to be explored in an Australian context (Romyn & Kebbell, 2013, 2018). Technology will be an asset in this approach, and methods of gamification and simulation can be exploited. As results of the current study point to the importance of additional variables in predicting attractiveness and lethality, wider SCP variables including weapon type, as well as pre-attack considerations (such as reconnaissance), can also be manipulated and measured experimentally to determine their impact.

Acknowledgements

We would like to thank CTPSR colleagues for their helpful comments on an earlier draft of this article. This work was commissioned by the National Counter Terrorism Security Office (NaCTSO), UK.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

This data for this study was accessed via a paid subscription to the TerrorismTracker database (operated by Dragonfly, a security intelligence and analysis service). Please contact the corresponding author for details of the data we used and we thank Dragonfly for permitting its sharing.

Additional information

Funding

This work was commissioned by the National Counter Terrorism Security Office (NaCTSO), UK.

Notes on contributors

Rachel Monaghan

Rachel Monaghan is a Professor of Peace and Conflict at the Centre for Trust, Peace and Social Relations, Coventry University. She has been researching political violence for over 25 years and has examined single-issue terrorism, informal justice, and paramilitary violence. More recently, she has been researching criminal justice approaches with respect to terrorism. She is the current President of the Society for Terrorism Research.

Bianca Slocombe

Bianca Slocombe is an Assistant Professor at the Centre for Trust, Peace and Social Relations. She has expertise in experimental and statistical methods and is interested in decision-making and bounded rationality. Her doctoral research focused on the moderation of beliefs and values in an actor’s willingness to act violently for a sacred cause. More recently, she has been utilising her expertise in psychology and behavioural science in the field of protective security.

David McIlhatton

David McIlhatton is a Professor of Protective Security and Resilience at the Centre for Trust, Peace and Social Relations, Coventry University. He has published widely on counter-terrorism and is on a number of editorial boards of international peer-review journals and reviews for multiple terrorism journals. His research portfolio includes projects with a value of over £17 million in security and business-related research areas internationally.

John Cuddihy

John Cuddihy is a Visiting Professor in Urban and Maritime Security at the Centre for Trust, Peace and Social Relations, Coventry University. As a former Head of Organised Crime and Counter Terrorism, Police Scotland, he has an extensive background and experience of policing, investigating and countering terrorism with significant expertise in strategic threat and risk assessments.

Notes

1 The United Kingdom is comprised of Great Britain (England, Scotland and Wales) and Northern Ireland.

2 The American Terrorism Study (ATS) is an open-source database housed at the Terrorism Research Center at the University of Arkansas. It consists of data on federal terrorism-related court cases, persons indicted in these court cases, and related officially designated terrorism incidents and uses the FBI’s definition of terrorism: ‘The unlawful use of force or violence against persons or property to intimidate or coerce a Government, the civilian population, or any segment thereof, in furtherance of political or social objectives.’ For more information see https://terrorismresearch.uark.edu/

3 Multiple regression is used to predict the value of a variable based on the value of two or more other variables. For example, you might want to know how much of the variation in exam performance can be explained by revision time, test anxiety, lecture attendance and gender (Laerd Statistics, 2022).

4 This research was commissioned to specifically assess the validity of Clarke and Newman’s ‘EVIL DONE’ framework in the context of the current UK terrorist threat landscape, to identify attributes of target vulnerability and attractiveness and to develop a revised terrorism target risk assessment framework.

5 For more details on the database, see https://www.dragonflyintelligence.com/intelligence/terrorismtracker/.

6 2015 was chosen due to the UK Government’s 2015 National Security Strategy and Strategic Defence and Security Review identifying terrorism as one of the highest priority risks to the UK. See https://www.gov.uk/government/publications/national-security-strategy-and-strategic-defence-and-security-review-2015 for more details.

7 Note that in the case of multiple targets within one incident, the highest relevant scores are applied. Multiple perpetrators (rather than individuals) were typically deemed ‘local groups’ with no individual identification. Components were based on group characteristics without the need for rating and comparing individual scores (e.g., near rating based on locality of the group, tolerable rating based on types of weapons concealed by the group).

8 Weighted kappa (κw) is a measure of inter-rater agreement for two raters using ordinal scales. It measures the proportion of agreement over and above chance agreement (scores range from -1 to +1, with a score of 0 indicating that agreement is no better than chance). Altman’s (1991) classification scale (adapted from Landis & Koch, 1977) indicates strength of agreement is poor at <0.20, fair up to 0.40, moderate up to 0.60, good up to 0.80, and very good up to a maximum score of 1 (perfect agreement).

9 By undirected, we mean that the target is not clearly identified. Examples of undirected attacks include an IED (pipe bomb) left outside a residential property, or a paramilitary punishment attack where link between the offender and the victim is not known (or not included in the open-source data). However, it should be noted that for the communities in which these attacks occur, it is likely that the identification of the target is known by the offender.