There Is No Such Thing as Open Source Intelligence

Abstract

This article argues that “open source intelligence (OSINT)” is a fundamentally incoherent concept that should be abandoned. It does so in two steps. First, by challenging the underlying criteria used to demarcate it as a separate “INT” among its more traditional peers. Second, through a historical critique that argues that “OSINT” as a conceptual category served a transitionary stage that has long passed. That is, it helped intelligence practitioners and scholars appreciate the influx of valuable unclassified information made newly available by the World Wide Web in the 1990s, but the advantages gained from this notion have now declined, and the concept is now a liability. By discarding the term altogether, and recategorizing openly derived sources of information back into their traditional homes, significant conceptual and analytical benefits can be attained.

ACKNOWLEDGMENTS

The author thanks Hamilton Bean, Justin Chock, Jules Gaspard, David V. Gioe, Michael S. Goodman, Jacob Hatfield, Michael Warner, and two anonymous reviewers for their instructive comments on an earlier draft of this article. Special thanks also to Tarak Barkawi, with whom I had a discussion years ago that initially led me down the road traveled in this article.

Notes

1 Vanessa Smith-Boyle, “How OSINT Has Shaped the War in Ukraine,” https://www.americansecurityproject.org/osint-in-ukraine/#:∼:text=Open%2Dsource%20intelligence%2C%20or%20OSINT,synthesized%2C%20and%20analyzed%20into%20intelligence (accessed 15 August 2022); Huw Dylan and Thomas J. Maguire, “Secret Intelligence and Public Diplomacy in the Ukraine War,” Survival, Vol. 64, No. 4 (2022), pp. 33–74. doi:10.1080/00396338.2022.2103257

2 Amy B. Zegart, Spies, Lies, and Algorithms (Princeton, NJ: Princeton University Press, 2022), p. 82.

3 Arthur S. Hulnick, “The Downside of Open Source Intelligence,” International Journal of Intelligence and CounterIntelligence, Vol. 15, No. 4 (2002), pp. 565–579. doi:10.1080/08850600290101767

4 Bowman H. Miller, “Open Source Intelligence (OSINT): An Oxymoron?,” International Journal of Intelligence and CounterIntelligence, Vol. 31, No. 4 (2018), pp. 702–719. doi:10.1080/08850607.2018.1492826

5 David V. Gioe, Joseph M. Hatfield, and Mark Stout, “Can United States Intelligence Community Analysts Telework?” Intelligence and National Security, Vol. 35, No. 6 (2020), pp. 885–901.

6 Chelsea Manning, “I’m Still Bound to Secrecy,” New York Times. URL=https://www.nytimes.com/2022/10/08/opinion/chelsea-manning.html (accessed 10 October 2022).

7 Mark M. Lownthal, Intelligence: From Secrets to Policy (9th ed.) (Thousand Oaks, CA: CQ Press, 2023), p. 147.

8 Zegart, Spies, Lies, and Algorithms, p. 5.

9 Jeffrey T. Richelson, The US Intelligence Community (6th ed.) (Boulder, CO: Westview Press, 2011), p. 2.

10 A certain set of assumptions regarding the trustworthiness of openly available sources of information underlies this definitional criterion. Such assumptions are common in Western democracies, where a free press and a tradition of citizen-journalism has been established. OSINT may therefore be a purely Western conceptual category, a point made by Michael Hitchings, to whom I am indebted for this insight. In connection with this, see Joseph M. Hatfield, “Intelligence under Democracy and Authoritarianism: A Philosophical Analysis,” Intelligence and National Security, Vol. 37, No. 6 (2022), pp. 903–919.

11 “John Anthony Walker Jr. Spy Case,” Federal Bureau of Investigation, https://www.fbi.gov/history/artifacts/john-anthony-walker-jr-spy-case (accessed 13 August 2022).

12 “Venona: Soviet Espionage and The American Response 1939-1957,” Central Intelligence Agency, https://www.cia.gov/static/fc3235f14ff505b6f839321755cfe72d/Venona-Soviet-Espionage-and-The-American-Response-1939-1957.pdf (accessed 13 August 2022).

13 The author vouchsafes such usages through his many years of intelligence analysis work.

14 Stephen C. Mercado, “Open-Source Intelligence,” in Intelligence: The Secret World of Spies. edited by Loch K. Johnson and James J. Wirtz (Oxford: Oxford University Press, 2019), pp. 78–79.

15 David Omand, “The Cycle of Intelligence,” in Routledge Companion to Intelligence Studies, edited by Robert Dover, Michael S. Goodman, and Claudia Hillebrand (London: Routledge, 2014), pp. 59–70, at p. 69.

16 Michael S. Goodman, Spying on the Nuclear Bear: Anglo-American Intelligence and the Soviet Bomb (Redwood City, CA: Stanford University Press, 2007).

17 Personal correspondence with Michael S. Goodman, 5 September 2022.

18 Again, the author can attest to these from personal experience working for years within the U.S. IC.

19 Central Intelligence Agency, https://www.cia.gov/careers/jobs/open-source-collection-specialist/ (accessed 13 August 2022).

20 USAJobs, https://www.usajobs.gov/job/670505500 (accessed 13 August 2022).

21 National Academy of Sciences, “1964 Highway Research Board Report” (Washington, DC: National Research Council, 1964), p. 20.

22 American Society for Information Science, Annual Review of Information Science and Technology, Vol. 36 (1966), p. 704.

23 Google Ngram Viewer (2022). https://books.google.com/ngrams/graph?content=open-source+intelligence%2Copen+source+intelligence%2Cosint&year_start=1965&year_end=2019&corpus=26&smoothing=3&direct_url=t1%3B%2Copen%20-%20source%20intelligence%3B%2Cc0%3B.t1%3B%2Copen%20source%20intelligence%3B%2Cc0%3B.t1%3B%2Cosint%3B%2Cc0#t1%3B%2Copen%20-%20source%20intelligence%3B%2Cc0%3B.t1%3B%2Copen%20source%20intelligence%3B%2Cc0%3B.t1%3B%2Cosint%3B%2Cc0 (accessed 13 August 2022).

24 Peter H. Lewis, “The New York Times Introduces a Web Site,” TimesMachine. New York Times, 22 January 1996, https://timesmachine.nytimes.com/timesmachine/1996/01/22/687588.html?pageNumber=49

25 Thomas R. Pedtke, “Open Source Subcommittee,” Scientific & Technical Intelligence Committee Report (1995), https://irp.fas.org/dni/osc/stic1995.pdf (accessed 13 August 2022).

26 Mercado, “Open-Source Intelligence,” pp. 78–79.

27 Loch K. Johnson, in chapter one, “The Development of Intelligence Studies,” in the Routledge Companion to Intelligence Studies, edited by Robert Dover, Michael S. Goodman, and Claudia Hillebrand (London: Routledge, 2014), offers a similar analysis for the period between 1986 and 2011, although only using INS but adding some helpful comparisons with the New York Times.

28 Don E. Gordon, “Winners and Losers,” International Journal of Intelligence and CounterIntelligence, Vol. 1, No. 3 (1986), pp. 1–24; Arthur S. Hulnick, “The Intelligence Producer–Policy Consumer Linkage: A Theoretical Approach,” Intelligence and National Security, Vol. 1, No. 2 (1986), pp. 212–233.

29 Hulnick, “The Intelligence Producer–Policy Consumer Linkage”; Arthur S. Hulnick, “What’s Wrong with the Intelligence Cycle,” Intelligence and National Security, Vol. 21, No. 6 (2006), pp. 959–979.

30 G. M. McGill, “OSCINT and the Private Information Sector,” International Journal of Intelligence and CounterIntelligence, Vol. 7, No. 4 (1994), pp. 435–443.

31 Pew Research Center, “Americans Going Online…Explosive Growth, Uncertain Destinations” (October 1995), https://www.pewresearch.org/politics/1995/10/16/americans-going-online-explosive-growth-uncertain-destinations/ (accessed 13 August 2022).

32 Michael J. Hogan, The End of the Cold War: Its Meaning and Implications (Cambridge: Cambridge University Press, 1992); John Lewis Gaddis, We Now Know: Rethinking Cold War History (Oxford: Oxford University Press, 1997); Melvyn P. Leffler, For the Soul of Mankind: The United States, the Soviet Union, and the Cold War (New York: Hill and Wang, 2007).

33 John Keegan, Intelligence in War: The Value—and Limitations—of What the Military can Learn about the Enemy (New York: Vintage Books, 2002).

34 Christopher Andrew and Vasili Mitrokhin, The Sword and the Shield: The Mitrohkin Archive and the Secret History of the KGB (New York: Basic Books, 1999); Christopher Andrew, Defend the Realm: The Authorized History of MI5 (New York: Alfred A. Knopf, 2009).

35 This unease did not disappear with the discursive successes of OSINT-related terms, but simply became relegated within Intelligence Studies to a minority position, one that remains as such as evidenced by Miller’s respectably cited“Open Source Intelligence (OSINT).”

36 Abram N. Shulsky, Silent Warfare: Understanding the World of Intelligence (1st ed.) (New York: Brassey’s); Abram N. Shulsky and Gary J. Schmitt. 2002. Silent Warfare: Understanding the World of Intelligence (3rd ed.) (Sterling, VA: Potomac Books, 2002).

37 Christopher Andrew, The Secret World: A History of Intelligence (New Haven, CT: Yale University Press, 2018). Andrew’s usage of OSINT-related terms is semantically mixed. That is, although ten references to pamphlets and 38 references to the role and importance of newspapers appear in The Secret World, either as important sources of information or as targets of covert influence, Andrew nevertheless sometimes includes open sources within the definition of intelligence and in other contexts makes a distinction between the two. See ibid., pp. 122, 147, and 412 for the former and pp. 2 and 19 for the latter.

38 Robert David Steele, “The Importance of Open Source Intelligence to the Military,” International Journal of Intelligence and CounterIntelligence, Vol. 8, No. 4 (1995), pp. 457–470. doi:10.1080/08850609508435298.

39 Hulnick, “The Downside of Open Source Intelligence”; Robert W. Pringle, “The Limits of OSINT: Diagnosing the Soviet Media, 1985–1989,” International Journal of Intelligence and CounterIntelligence, Vol. 16, No. 2 (2003), pp. 280–289. doi:10.1080/08850600390198706

40 Hamilton Bean, “The DNI’s Open Source Center: An Organizational Communication Perspective,” International Journal of Intelligence and CounterIntelligence, Vol. 20, No. 2 (2007), pp. 240–257. doi:10.1080/08850600600889100

41 Ibid.

42 Laura M. Calkins, “Patrolling the Ether: US–UK Open Source Intelligence Cooperation and the BBC’s Emergence as an Intelligence Agency, 1939–1948,” Intelligence and National Security, Vol. 26, No. 1 (2011), pp. 1–22. doi:10.1080/02684527.2011.556355; Eric Setzekorn, “Open Source Information and the Office of Naval Intelligence in Japan, 1905–1920,” International Journal of Intelligence and CounterIntelligence, Vol. 27, No. 2 (2014), pp. 368–386. doi:10.1080/08850607.2014.842812; Ben Wheatley, “British Open Source Intelligence (OSINT) and the Holocaust in the Soviet Union: Persecution, Extermination and Partisan Warfare,” Intelligence and National Security, Vol. 33, No. 3 (2018), pp. 422–438. doi:10.1080/02684527.2017.1410516

43 Melonie K. Richey and Mathias Binz, “Open Source Collection Methods for Identifying Radical Extremists Using Social Media,” International Journal of Intelligence and CounterIntelligence, Vol. 28, No. 2 (2015), pp. 347–364. doi:10.1080/08850607.2014.962374; Aviva Guttmann, “The Rise of ISIS as a Partial Surprise: An OpenSource Analysis on the Threat Evolution and Early Warnings in the United Kingdom,” International Journal of Intelligence and CounterIntelligence (2022). doi:10.1080/08850607.2022.2095543

44 Christopher Eldridge, Christopher Hobbs, and Matthew Moran, “Fusing Algorithms and Analysts: Open-Source Intelligence in the Age of ‘Big Data,’” Intelligence and National Security, Vol. 33, No. 3 (2018), pp. 391–406. doi:10.1080/02684527.2017.1406677; Hüseyin Akarslan, “A Model Proposal for Analyzing Open-Source Information in End-User Computers,” International Journal of Intelligence and CounterIntelligence, Vol. 35, No. 3 (2022), pp. 482–503. doi:10.1080/08850607.2021.1899514.

45 Rune Saugmann, “The Civilian’s Visual Security Paradox: How Open Source Intelligence Practices Create Insecurity for Civilians in Warzones,” Intelligence and National Security, Vol. 34, No. 3 (2019), pp. 344–361. doi:10.1080/02684527.2018.1553700

46 David V. Gioe, Joseph M. Hatfield, and Mark Stout, “Can United States Intelligence Community Analysts Telework?”

47 Steele, “The Importance of Open Source Intelligence to the Military.”

48 Robert David Steele, “The Evolving Craft of Intelligence,” in Routledge Companion to Intelligence Studies, edited by Robert Dover, Michael S. Goodman, and Claudia Hillebrand (London: Routledge, 2014), pp. 59–83, at pp. 79–80.

49 Microsoft’s first serious effort to build security into Windows came in 2004, when Microsoft released Service Pack 2, which introduced Windows Security Center, for its Windows XP. Microsoft, “Microsoft Releases Windows XP Service Pack 2 with Advanced Security Technologies to Computer Manufacturers” (2004), https://news.microsoft.com/2004/08/06/microsoft-releases-windows-xp-service-pack-2-with-advanced-security-technologies-to-computer-manufacturers/ (accessed 9 October 2022). The success of Linux, which was first released in 1991, and other open source projects, were driven partly as a response to these security concerns.

50 Personal correspondence with Michael Warner, 23 August 2022.

51 For a more detailed discussion of Foucault’s ideas within the context of Intelligence Studies, see Hatfield, “Intelligence under Democracy and Authoritarianism.”

52 Robert W. Cox, “Social Forces, States and World Orders: Beyond International Relations Theory,” Millennium, Vol. 10, No. 2 (1981), pp. 126–155.

53 James Bohman, “Critical Theory,” The Stanford Encyclopedia of Philosophy (Winter 2019 ed.), edited by Edward N. Zalta, https://plato.stanford.edu/archives/win2019/entries/critical-theory/

54 Gioe, Hatfield, and Stout, “Can United States Intelligence Community Analysts Telework?”

55 “The Inner Ring” was given by C. S. Lewis as the Memorial Lecture at King’s College, University of London, in 1944.

56 One is reminded of Kim Philby’s explanation of his agreement to become a Soviet agent, saying, in the Forward to his 1968 memoir, My Silent War, “One does not look twice at an offer of enrolment in an elite force.” Kim Philby, My Silent War (New York: Modern Library, 1968).

57 https://www.understandingwar.org/ (accessed 10 October 2022).

58 https://www.bellingcat.com/ (accessed 10 October 2022).

59 https://rusi.org/ (accessed 10 October 2022).

60 Sandra Erwin, “NRO Signs Agreements with Six Commercial Providers of Space-Based RF Data” (2022) https://spacenews.com/nro-signs-agreements-with-six-commercial-providers-of-space-based-rf-data/ (accessed 10 October 2022).

61 Robert David Steele. “The Importance of Open Source Intelligence to the Military,” International Journal of Intelligence and CounterIntelligence 8, No. 4 (1995): 457-470, 10.1080/08850609508435298.

62 I am not suggesting that analysts explicitly think about this in these terms explicitly, but these are the implicit assumptions that result.

63 David Omand, Jamie Bartlett, and Carl Miller, “Introducing Social Media Intelligence (SOCMINT),” Intelligence and National Security, Vol. 27, No. 6 (2012), pp. 801–823.

64 “Number of Monthly Active Facebook Users Worldwide as of 2nd Quarter 2022,” Statista.com, https://www.statista.com/statistics/264810/number-of-monthly-active-facebook-users-worldwide/#:∼:text=How%20many%20users%20does%20Facebook,years%20to%20reach%20this%20milestone (accessed 21 August 2022).

65 Acts 17:22–23, Berean Standard Bible.

66 “Tunisia Suicide Protester Mohammed Bouazizi Dies,” BBC News (2011), https://www.bbc.com/news/world-africa-12120228 (accessed 21 August 2022).

67 Phil Corbett, “Why Does the New York Times use Anonymous Sources?,” New York Times (2022), https://www.nytimes.com/article/why-new-york-times-anonymous-sources.html (accessed 8 October 2022). The New York Times offers four criterial questions asked in the evaluation of their sources: (1) How do they know the information? (2) What is their motivation for telling us? (3) Have they proved reliable in the past? (4) Can we corroborate the information they provide?

68 Anecdotally, a prominent BBC reporter with over a decade’s experience working in China confirmed to me that this scenario almost never happens.

69 Marc Santora, Ivan Nechepurenko, and Haley Willis, “Ukrainian Forces Claim to Destroy a Russian Landing Ship,” New York Times (2022), https://www.nytimes.com/2022/03/24/world/europe/russian-landing-ship-berdyansk-ukraine.html (accessed 21 August 2022).

70 David V. Gioe and Joseph M. Hatfield, “A Damage Assessment Framework for Insider Threats to National Security Information: Edward Snowden and the Cambridge Five in Comparative Historical Perspective,” Cambridge Review of International Affairs, Vol. 34, No. 5 (2021), pp. 704–738.

71 Quoted in Hamilton Bean, “The Paradox of Open Source: An Interview with Douglas J. Naquin,” International Journal of Intelligence and CounterIntelligence, Vol. 27, No. 1 (2014), pp. 42–57, at p. 52. doi:10.1080/08850607.2014.842797

Additional information

Notes on contributors

Joseph M. Hatfield

Joseph M. Hatfield is a Naval Intelligence Officer and an Assistant Professor at the U.S. Naval Academy. He holds a Ph.D. from Cambridge University. He can be contacted at [email protected].