Strengths and Weaknesses of Open Source Data for Studying Terrorism and Political Radicalization

Abstract

Compared to studying other forms of illegal violence, terrorism research faces unique data challenges. Notably, consistent police data are generally lacking, victims often have no direct knowledge of attacks and are sometimes killed, and perpetrators are difficult to interview and are often unreliable sources. In the face of these data challenges, the increasing availability of open source data on terrorist attacks and perpetrators has profoundly affected the scientific study of terrorism. Over time, a growing proportion of published academic research on terrorism—especially more heavily quantitative studies—relies on open source data. Moreover, the growing availability of individual-level terrorism data has allowed empirical tests of psychological and social-psychological theories of radicalization. In this paper, we assess the impact of open source data on the study of terrorism. We begin with an overview, providing a brief history of terrorist attack and perpetrator open source data bases. We then consider the major strengths and weaknesses of both types of open source data and suggest ways of countering the weaknesses. We close with observations about the impact of open source data on terrorism research and future opportunities for improvement.

Keywords:

• Open source data
• terrorism
• political radicalization
• databases

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1 Although far less common, a third type of open source terrorism database examines terrorist organizations. See for example, Victor Asal, H. H. Park, R. K. Rethemeyer, R. K., and G. Ackerman, “With Friends Like These… Why Terrorist Organizations Ally,” International Public Management Journal 19 (2016): 1–30.

2 Gary LaFree, Toward a Criminology of Terrorism (Cambridge: Cambridge University Press, 2023); Gary LaFree and Joshua Freilich, The Handbook of the Criminology of Terrorism (Chichester: Wiley Blackwell, 2016).

3 Andrew Silke and Jennifer Schmidt-Petersen, “The Golden Age? What the 100 Most Cited Articles in Tterrorism Studies Tell us,” Terrorism and Political Violence 29, no. 4 (2017): 692–712.

4 Bart Schuurman, “Research on Terrorism, 2007–2016: A Review of Data, Methods, and Authorship,” Terrorism and Political Violence 32, no. 5 (2020): 1011–26.

5 Peter Neumann and Scott Kleinmann, “How Rigorous is Radicalization Research?,” Democracy and Security 9, no. 4 (2013): 360–82.

6 Neil G. Bowie, “Terrorism Events Data: An Inventory of Databases and Data Sets, 1968–2017,” Perspectives on Terrorism 11 (2017): 50–72.

7 Edward F. Mickolus, “How Do We Know We’re Winning the War Against Terrorists? Issues in Measurement,” Studies in Conflict & Terrorism 25 (2002): 151–60; Edward F. Mickolus, Todd Sandler, Jean M. Murdock, and Peter Flemming, International Terrorism: Attributes of Terrorist Events (ITERATE) (Dunn Loring, VA: Vinyard Software, 2010).

8 We define international attacks as those in which the nationality of the perpetrator, the target or the location are different (Gary LaFree, Laura Dugan, and Erin Miller, Putting Terrorism in Context: Lessons from the Global Terrorism Database (London: Routledge, 2014), 150.

9 John Wigle, “Introducing the Worldwide Incidents Tracking System (WITS),” Perspectives on Terrorism 4 (2010): 21.

10 Ibid, 5.

11 Gary LaFree and Laura Dugan, “Introducing the Global Terrorism Database,” Terrorism and Political Violence 19 (2007): 181–204.

12 LaFree, Dugan, and Miller, “Putting Terrorism in Context”.

13 Michael Jensen, “Discussion Point: The Benefits and Drawbacks of Methodological Advancements in Data Collection and Coding: Insights from the Global Terrorism Database” (Unpublished study, University of Maryland, 2013).

14 The FBI defines terrorism as “The unlawful use of force or violence against persons or property to intimidate or coerce a government, the civilian population, or any segment thereof, in furtherance of political or social objectives.”

15 Brent L. Smith and Kelly R. Damphousse, “American Terrorism Study, 1980–2002” (Unpublished study, University of Arkansas, 2007).

16 Ibid.

17 Ibid.

18 Joshua D. Freilich, Steven M. Chermak, Roberta Belli, Jeff Gruenewald, and William S. Parkin, “Introducing the United States Extremis Crime Database (ECDB),” Terrorism and Political Violence 26, no. 2 (2014): 372–84.

19 Ibid., Appendix 4.

20 Paul Gill, Lone-actor Terrorists: A Behavioural Analysis (London/New York: Routledge, 2015).

21 Ibid.

22 Ibid.

23 Paul Gill, James Silver, John Horgan, Emily Corner, and Noemie Bouhana, “Similar Crimes, Similar Behaviors? Comparing Lone-Actor Terrorists and Public Mass Murderers,” Journal of Forensic Sciences 66, no. 5 (2021): 1797–804.

24 Robert A. Fein and Bryan Vossekuil. “Assassination in the United States: An Operational Study of Recent Assassins, Attackers, and Near-Lethal Approachers,” Journal of Forensic Sciences 44 (1999): 321–33.

25 LaFree, Dugan, and Miller, Putting Terrorism in Context, ch. 2.

26 Michael A. Jensen and Gary LaFree, Empirical Assessment of Domestic Radicalization (EADR): Report to the National Institute of Justice, Office of Justice Programs, US Department of Justice (START, Unpublished study, University of Maryland, 2016).

27 David Pyrooz, Gary LaFree, Scott Decker, and Patrick James, “Cut From the Same Cloth? Comparing Gangs and Violent Political Extremists,” Justice Quarterly 35 (2017): 1–32.

28 Gary LaFree, “Terrorism and the Internet,” Criminology and Public Policy 16 (2017): 93–8.

29 Michael Jensen, Neil Ferguson, Elizabeth Sheehan, and Gary LaFree, “Do Social Networks Distinguish American ISIS Foreign Fighters from ISIS-Inspired Terrorists?” Journal of Conflict Resolution 68, no. 1 (2024): 3–29.

30 Michael Wolfowicz, Yael Litmanovitz, David Weisburd, and Badi Hasisi,”A Field-wide Systematic Review and Meta-analysis of Putative Risk and Protective Factors for Radicalization Outcomes,” Journal of Quantitative Criminology 36 (2020): 407–47.

31 LaFree, Dugan and Miller, Putting Terrorism in Context, 13.

32 LaFree, Toward a Criminology of Terrorism.

33 Gill, Lone-Actor Terrorists.

34 See also Sophia Moskalenko, Joshua D. Freilich, Steven Chermak, Jeff Gruenewald, and Clark McCauley, “Growth Opportunities in American and British Terrorism Research,” Dynamics of Asymmetric Conflict 16, no. 1 (2023): 44–69.

35 Jenkins, Brian, “Will Terrorists Go Nuclear?” (January 1975). https://www.rand.org/pubs/papers/P5541.html.

36 Steven M. Chermak, Joshua D. Freilich, William S. Parkin, and James P. Lynch, “American Terrorism and Extremist Crime Data Sources and Selectivity Bias: An Investigation Focusing on Homicide Events Committed by Far-Right Extremists,” Journal of Quantitative Criminology 28, no. 1 (2012): 191–218.

37 Paul Gill and John Horgan, “Who were the Volunteers? The Shifting Sociological and Operational Profile of 1240 Provisional Irish Republican Army Members,” Terrorism and Political Violence 25, no. 3 (2013): 435–56.

38 Ibid.

39 Paul Gill, Emily Corner, Amy McKee, Paul Hitchen, and Paul Betley, “What do Closed Source Data Tell us about Lone Actor Terrorist Behavior? A Research Note,” Terrorism and Political Violence 34, no. 1 (2022): 113–30.

40 Alex P. Schmid and Albert J. Jongman, Political Terrorism: A New Guide to Actors, Authors, Concepts, Data Bases, Theories, and Literature, (Amsterdam/New Brunswick: Transaction Publishers, 1988).

41 Ivan S. Sheehan, “Assessing and Comparing Data Sources for Terrorism Research,” in Evidence-Based Counterterrorism Policy, ed. Cynthia Lum and Leslie W. Kennedy (New York, NY: Springer, 2012), 13–40.

42 The fifth database reviewed was the Terrorism in Western Europe: Event Data (TWEED). We excluded TWEED from the earlier discussion because it is limited to domestic terrorism attacks in 18 West European countries (Jan Oskar Engene, “Five Decades of Terrorism in Europe: The TWEED Dataset,” Journal of Peace Research 44 (2007): 109–21).

43 See Moskalenko et al., “Growth Opportunities in American and British Terrorism Research”, note 31.

44 Konstantinos Drakos and Andreas Gofas, “The Devil you Know but are Afraid to Face: Underreporting Bias and its Distorting Effects on the Study of Terrorism,” Journal of Conflict Resolution 50 (2006): 714–35.

45 Gary LaFree, “Terrorism Open-source Databases,” in Contemporary Terrorism Studies, ed. Diego Muro and Tim Wilson (New York: Oxford University Press, 2022), 113–134.

46 Jamie Grierson, “Islamist Extremism Remains Dominant UK Terror Threat, Say Experts,” The Guardian, December 10, 2019. http://www.theguardian.com/politics/2019/dec/10/islamist-extremism-remains-dominant-uk-terror-threat-say-experts.

47 Gary King and Will Lowe, “An Automated Information Extraction Tool for International Conflict Data with Performance as Good as Human Coders: A Rare Events Evaluation Design,” International Organization 57 (2003): 617–42; Philp A. Schrodt and David Van Brackle, “Automated Coding of Political Event Data,” in Handbook of Computational Approaches to Counterterrorism, ed. V. S. Subrahmanian (New York, NY: Springer, 2013), 23–49.

48 Mark Fishman, Manufacturing the News (New York: University of Texas Press, 1998).

49 Emily Corner and Paul Gill, “Is There a Nexus Between Terrorist Involvement and Mental Health in the Age of the Islamic State?,” The CTC Sentinel 10, no. 1 (2017): 1–10.

50 Erin M. Kearns, Allison E. Betus, and Anthony F. Lemieux, “Why Do Some Terrorist Attacks Receive More Media Attention Than Others?” Justice Quarterly 36 (2019): 985–1022.

51 Micahel G. Findley and Joseph K. Young, “More Combatant Groups, More Terror? Empirical Tests of an Outbidding Logic, Terrorism and Political Violence, 24 (2012): 706–21.

52 Erin Miller, Global Terrorism Overview: Terrorism in 2019 (College Park: START, July 2020). https://www.start.umd.edu/pubs/START_GTD_GlobalTerrorismOverview2019_July2020.pdf.

53 Ibid., 8.

54 See for example: Lasse Lindekilde, Francis O’Connor, and Bart Schuurman, “Radicalization Patterns and Modes of Attack Planning and Preparation among Lone-actor Terrorists: An Exploratory Analysis,” Behavioral Sciences of Terrorism and Political Aggression 11, no. 2 (2019): 113–33; P. C. Gattinara, F. O’Connor, and L. Lindekilde, “Italy, no Country for Acting Alone? Lone Actor Radicalisation in the Neo-fascist Milieu, Perspectives on Terrorism, 12, no. 6 (2018): 136–49.

55 Paul Gill, The Data Collection Challenge: Experiences Studying Lone-actor Terrorism (Washington, DC: Resolve Network, 2020).

56 Gary LaFree and Yesenia Yanez, “Criminological Perspectives on Extremist Radicalization and Mobilization,” in Handbook on Radicalization and Countering Radicalization, ed. J. Busher, L. Malkki, and S. Marsden (London: Routledge, 2023); Daren Fisher and Erin M. Kearns, “The Theorizing of Terrorism within Criminology,” Journal of Research in Crime and Delinquency 61, no, 4 (2024): 487–520.

57 Caitlin Clemmow, Sandy Schumann, Nadine L. Salman, and Paul Gill, “The Base Rate Study: Developing Base Rates for Risk Factors and Indicators for Engagement in Violent Extremism,” Journal of Forensic Cciences 65, no. 3 (2020): 865–81.

58 Global Terrorism Database, Codebook: Methodology, Inclusion Criteria and Variables. (Unpublished manuscript, University of Maryland, 2021), 10. https://www.start.umd.edu/publication/global-terrorism-overview-terrorism-2019.

59 Fred Burton and Scott Stewart, “The ‘Lone Wolf’ Disconnect,” STRATFOR, January 30, 2008; Edwin Bakker and Beatrice A. de Graaf, “Preventing Lone Wolf Terrorism: Some CT Approaches Addressed,” Perspectives 5, no. 5–6 (2011): 8.

60 Ramon Spaaij, Understanding Lone Wolf Terrorism: Global Patterns, Motivations and Prevention (Dordrecht: Springer, 2011); Petter Nesser, “Single Actor Terrorism: Scope, Characteristics and Explanations,” Perspectives on Terrorism 6, no. 6 (2012): 61–73; Matthew Feldman, “Comparative Lone Wolf Terrorism: Toward a Heuristic Definition,” Democracy and Security 9, no. 3 (2013): 270–86.

61 LaFree, Dugan, and Miller, “Putting Terrorism in Context,” 17–18.

62 Gary LaFree, Recent Developments in the Quantitative Study of Terrorism (Recent Developments in the Quantitative Study of Terrorism (Vienna: European Institute for Counter Terrorism and Conflict Prevention, 2024).