Order and chaos: the CIA’s HYDRA database and the dawn of the information age

ABSTRACT

In the summer of 1967, with antiwar and civil rights protests dominating the news, U.S. president Lyndon B. Johnson tasked law enforcement and intelligence agencies with investigating links between domestic civil unrest and foreign governments, particularly the Soviet Union. One of the outcomes was HYDRA, a CIA-led counter-intelligence database intended to leverage novel digital information technology to uncover previously unseen links to foreign threats. The paper argues that conceptualizing HYDRA as technological system which mobilized resources from across the federal government as well as from foreign partner agencies, allows us to raise larger questions about the impact of information technology on intelligence work: How did computer technology change everyday practices within intelligence and security services? Did public opposition to computerization efforts contribute to a critical discourse within Western societies associating security databases with attacks on freedom and democracy? Did the use of computers contribute to a new culture of security that shifted attention from great power rivalries to technological, networked or transnational threats which became characteristic of the Information Age?

KEYWORDS:

• Central Intelligence Agency
• Computers
• Intelligence History
• History of Technology

Notes

¹ Cf. Rhodri Jeffreys-Jones, We Know All About You: The Story of Surveillance in Britain and America (Oxford University Press, 2017), 131–39; Frank Rafalko, MH/CHAOS: The CIA’S Campaign Against the Radical New Left and the Black Panthers (Annapolis, MD: Naval Institute Press, 2011); and Joan M. Jensen, Army Surveillance in America, 1775–1980 (New Haven, CT: Yale University Press, 1991).

² For an expanded version of this argument in a comparative, transatlantic perspective see also Constantin Goschler, Christopher Kirchberg, and Jens Wegener, “Sicherheit, Demokratie und Transparenz. Elektronische Datenverbundsysteme in der Bundesrepublik und den USA in den 1970er und 1980er Jahren,“ in Wege in die digitale Gesellschaft: Computernutzung in der Bundesrepublik 1955–1990, ed. Frank Bösch, 1st ed. (Göttingen: Wallstein, 2018), 64–86.

³ Cf. Halkin v. Helms, 690 F.2d 977 (1982).

⁴ Cf. Rhodri Jeffreys-Jones, The CIA and American Democracy, 3rd ed. (New Haven, CT: Yale University Press, 2003), 168; Michael Howard Holzman, James Jesus Angleton, the CIA, and the Craft of Counterintelligence (Amherst, MA: University of Massachusetts Press, 2008), 214f.

⁵ Cf. Angus Mackenzie, Secrets: The CIA’s War at Home (Berkeley, CA: University of California Press, 1999), chap. 2; Tim Weiner, Legacy of Ashes: The History of the CIA (New York: Anchor Books, 2007), 329–31; for a critique of related FBI operations see William W. Keller, The Liberals and J. Edgar Hoover: Rise and Fall of a Domestic Intelligence State (Princeton, NJ: Princeton University Press, 1989).

⁶ Recent contributions in this direction include Constantin Goschler and Michael Wala, ‘Keine neue Gestapo’ das Bundesamt für Verfassungsschutz und die NS-Vergangenheit (Reinbek bei Hamburg: Rowohlt, 2016); Jeffreys-Jones, We Know All About You; Rüdiger Bergien, “‘Big Data‘ Als Vision. Computereinführung Und Organisationswandel in BKA Und Staatssicherheit (1967–1989),“ Zeithistorische Forschungen/Studies in Contemporary History Online 14, no. 2 (2017), http://www.zeithistorische-forschungen.de/2-2017/id=5488.

⁷ Cf. the theoretical argument about mediating technologies in Hans Krause Hansen and Mikkel Flyverbom, “The Politics of Transparency and the Calibration of Knowledge in the Digital Age,” Organization 22, no. 6 (1 November 2015): 2.

⁸ On this general dynamic see Leon Hempel, Susanne Krassmann, and Brockling, Ulrich, “Sichtbarkeitsregime: Eine Einleitung,“ in Sichtbarkeitsregime : Überwachung, Sicherheit und Privatheit im 21. Jahrhundert (Wiesbaden: VS Verlag für Sozialwissenschaften, 2011), 8.

⁹ See the interdisciplinary field of visibility/invisibility studies: Henriette Steiner and Kristin Veel, Invisibility Studies: Surveillance, Transparency and the Hidden in Contemporary Culture (Oxford: Peter Lang, 2015); Andrea Brighenti, “Visibility: A Category for the Social Sciences,” Current Sociology 55, no. 3 (1 May 2007): 323–42; John B. Thompson, “The New Visibility,” Theory, Culture & Society 22, no. 6 (1 December 2005): 31–51.

¹⁰ Cf. Cornel Zwierlein, “Sicherheitsgeschichte: Ein neues Feld der Geschichtswissenschaften,“ Geschichte und Gesellschaft 38, no. 3 (2010): 365–86; Christopher Daase, “‘National, Societal‘, and ‘Human Security’: On the Transformation of Political Language,“ Historical Social Research/Historische Sozialforschung 35, no. 4 (134) (2010): 22–37.

¹¹ ‘Final Report of the Select Committee to Study Government Operations With Respect to Intelligence Activities’, Book III, ‘Supplementary Detailed Staff Reports on Intelligence Activities and the Rights of Americans’ (Washington, DC: GPO, 1976) [cited as ‘Church Committee Report’]; Rockefeller Commission, Report by the President by the Commission on CIA Activities Within the United States (Washington, DC: GPO, 1975) [cited as ‘Rockefeller Commission Report’].

¹² The present article utilizes digital MHCHAOS records released by the CIA in CD-ROM format (cited as ‘MHCHAOS Files’), for a brief overview of the CIA’s handling of MHCHAOS files see Robert J. Eatinger to Chief, Historical Review Group, 8 May 1993, JFK Assassination Records (https://www.archives.gov/files/research/jfk/releases/docid-32404173.pdf).

¹³ Records of the Inter-Divisional Information Unit, RG 60, National Archives II [cited as ‘IDIU Records’].

¹⁴ Cf. Church Committee Report, 688–689.

¹⁵ Cf. Jeffreys-Jones, We Know All About You, 131f; Jensen, Army Surveillance in America, 245f.

¹⁶ Martin Klimke, The Other Alliance: Student Protest in West Germany and the United States in the Global Sixties (Princeton: Princeton University Press, 2011); Jeremi Suri, Power and Protest: Global Revolution and the Rise of Detente (Cambridge, MA: Harvard University Press, 2005).

¹⁷ Cf. Klimke, The Other Alliance, 77.

¹⁸ Cable, Richard Ober, “MHCHAOS,” 19(?) July 1968, MHCHAOS Files, Doc ID C00016381.

¹⁹ Briefing Papers. Special Operations Group, Counter Intelligence Staff, 1 June 1972, 6–9, MHCHAOS Files, Doc ID C00016380.

²⁰ Cf. Rafalko, MH/CHAOS, 34f.

²¹ Cf. Poul Villaume, Rasmus Mariager, and Helle Porsdam, “Introduction: The Long 1970s,” in The ‘Long 1970s’: Human Rights, East-West Détente and Transnational Relations, ed. Poul Villaume, Rasmus Mariager, and Porsdam (New York: Routledge, 2016), 1.

²² Cf. Jürgen Danyel and Annette Schuhmann, “Wege in die digitale Moderne. Computerisierung als gesellschaftlicher Wandel,“ in Geteilte Geschichte: Ost- und Westdeutschland 1970–2000, ed. Frank Bösch (Vandenhoeck & Ruprecht, 2015), 295.

²³ Charles A. Briggs, Memo ‘Replacement for the IBM 360/50 Time Sharing Computer System,’ 17 July 1968, General Records of the Central Intelligence Agency, CIA CREST Database, Doc. ID CIA-RDP71R00510A000300180009-5.

²⁴ Cf. Manuel Castells, The Rise of the Network Society: Second Edition, with a New Preface, 2nd ed. (Malden, MA: Wiley-Blackwell, 2012).

²⁵ Cf. Kiran Klaus Patel, The New Deal: A Global History (Princeton, NJ: Princeton University Press, 2016), 91–96.

²⁶ ‘The Long Range Plan of the Central Intelligence Agency,’ 31 August 1965, General Records of the Central Intelligence Agency, CIA CREST Database, Doc. ID CIA-RDP82M00311R000100350001-3.

²⁷ Richard Helms, “Address to the Information Services Division,” 17 November 1971, CIA CREST Database, Doc. ID CIARDP84-00161R000400210059-3.

²⁸ Cf. Jeffrey T. Richelson, The Wizards Of Langley: Inside The Cia’s Directorate Of Science And Technology (Basic Books, 2008), 79: Memo “Briefing for the DCI on Automatic Data Processing in the Agency by [],” 20 August 1969, CIA CREST Database, Doc. ID CIA-RDP78-04723A000100100032-0.

²⁹ Briefing Papers. Special Operations Group, Counter Intelligence Staff, 1 June 1972, 11f.

³⁰ Richard Helms to Henry Kissinger, 18 February 1969, CIA CREST Database, Doc. ID LOC-HAK-1-2-21-4.

³¹ Rafalko, MH/CHAOS, 31f.

³² J. Edgar Hoover, “Now: Instant Crime Control in Your Town,” Popular Science, January 1967, 67f.

³³ Memo, “Strengthening the Information Processing Structure of the Agency,” 15 September 1969, CIA CREST Database, Doc. ID CIA-RDP78-04723A000100100029-4.

³⁴ Memo “Briefing for the DCI,” 20 August 1969, CIA CREST Database, Doc. ID CIA-RDP78-04723A000100100032-0; Richard Helms, Memo to DDP, DDI, etc., 6? September 1969, MHCHAOS Files, Doc ID C00016391.

³⁵ Rafalko, MH/CHAOS, 30.

³⁶ “Congress to Seek Evidence of a Campus Conspiracy,” New York Times, 10 May 1969.

³⁷ Lawrence K. White, Memorandum for the Record, 12 March 1969, CIA CREST Database, Doc. ID CIA-RDP80R01284A001800090118-9.

³⁸ Kessler, The Bureau: The Secret History of the FBI (New York: St. Martin’s Press, 2002), 151; and Church Committee Report, 688.

³⁹ Lawrence K. White, Memorandum for the Record, 12 March 1969, Records of the Central Intelligence Agency, CIA CREST Database, Doc. ID CIA-RDP80R01284A001800090118-9.

⁴⁰ [Minutes of meeting between Assistant Attorney General Jerris Leonard and CIA representatives], 19 May 1969, CIA CREST Database, Doc. ID CIA-RDP08-02430R000100030005-8.

⁴¹ Department of Justice Press Release, 14 January 1975, IDIU Records, Box 1, Folder ‘IDIU 1972–1974’, RG 60, NARA II.

⁴² Cf. Jeffreys-Jones, We Know All About You, 132f; Paul R. Clancy, Just a Country Lawyer: A Biography of Senator Sam Ervin (Bloomington, IN: Indiana University Press, 1974), 209–12.

⁴³ Federal Data Banks, Computers and the Bill of Rights: Hearings before the Subcommittee on Constitutional Rights of the Committee on the Judiciary United States Senate, 92nd Congress, 1st Session (Washington, DC: GPO, 1971), 148.

⁴⁴ Ibid., 57, 618.

⁴⁵ Ibid., 2.

⁴⁶ “Huge C.I.A. Operation Reported in U.S. Against Antiwar Forces, Other Dissidents in Nixon Years,” New York Times, 22 December 1974.

⁴⁷ “Computer/Operations Research Skill Development Program,” September 1969, CIA CREST Database, Doc. ID CIA-RDP84-00780R003100110025-1.

⁴⁸ Richard Ober was born in 1921, was a veteran of the OSS and had joined the CIA in the late 1940s; his deputy Jason Horn was born in 1920, had served in the U.S. Army in post-war Japan and had entered the Agency in 1952.

⁴⁹ Rafalko, MH/CHAOS, 31; Michael Abbell to Henry E. Petersen, 4 May 1973, 3.

⁵⁰ Abbell to Petersen, 4 May 1973.

⁵¹ Cf. Testimony of Ralph Stein, in: Federal Data Banks, “Computers and the Bill of Rights: Hearings before the Subcommittee on Constitutional Rights of the Committee on the Judiciary United States Senate,” 92nd Congress, 1st Session (Washington, DC: GPO, 1971), 248.

⁵² ”Briefing by Department of Justice on Organized Crime Computer System,” 17 April 1970, CIA CREST Database, Doc. ID CIA-RDP72-00310R000200120022-9.

⁵³ Cf. David Gugerli and Hannes Mangold, “Betriebssystem und Computerfahndung. Zur Genese einer digitalen Überwachungsstruktur,“ Geschichte und Gesellschaft, no. 1 (2016): 150.

⁵⁴ Cf. Jeffreys-Jones, The CIA and American Democracy, 168.

⁵⁵ “IDIU Coding Manual, April 1971,” IDIU Records, Box 1, Folder “IDIU 1972–1974,” RG 60, NARA II.

⁵⁶ “Black Radicalism in the Caribbean,” 6 July 1970, p. 13, CIA CREST Database, Doc. ID CIA-RDP76M00527R000700200009-3.

⁵⁷ Cf. CIA “Restless Youth,” June 1970, p. 20, CIA CREST Database, Doc. ID DOC_0002987248; similarly “Black Radicalism in the Caribbean,” 14.

⁵⁸ Henry Kissinger, “Summary of CIA Survey,” 7 March 1969.

⁵⁹ Richard Ober, Memorandum for the Record, “Meeting with the Director on MHCHAOS,” 5 December 1972, MHCHAOS Files, Doc ID C00016396; and William V. Bros to William E. Colby, “MHCHAOS,” 30 May 1973, MHCHAOS Files, Doc ID C00016731.

⁶⁰ Abbell to Petersen, 4 May 1973.

⁶¹ Church Committee Report, 707.

⁶² Cf. Tity de Vries, “The 1967 Central Intelligence Agency Scandal: Catalyst in a Transforming Relationship between State and People,” Journal of American History 98, no. 4 (1 March 2012): 1075–92; Jason Ross Arnold, Secrecy in the Sunshine Era: The Promise and Failures of US Open Government Laws (Lawrence, KS: University Press of Kansas, 2014).

⁶³ Cf. Larry Frohman, “‘Only Sheep Let Themselves Be Counted’. Privacy, Political Culture, and the 1983/87 West German Census Boycotts,” Archiv für Sozialgeschichte 52 (2012): 335–78; in the United Kingdom, the ‘data banks’ hearings also appear to have influence privacy debates, cf. Jon Agar, The Government Machine: A Revolutionary History of the Computer (Cambridge, MA: MIT Press, 2003), 353.

⁶⁴ FBI Oversight: Hearings before the Subcommittee on Civil and Constitutional Rights of the Committee on the Judiciary, House of Representatives, 96^th Congress, 1^st and 2^nd session (Washington, DC: GPO, 1980), p. 124.

⁶⁵ “Aide Says Pentagon Computers Aren’t Used to Check Citizens,” New York Times, 24 June 1975.

⁶⁶ Rafalko, MH/CHAOS, 31.

⁶⁷ Ibid.

⁶⁸ Memorandum, “Status on MHCHAOS Activities,” 20 August 1973, MHCHAOS Files, Doc ID C00016301.

⁶⁹ Richard Ober, “Memorandum for the Record, ‘Meeting with the Director’,” 5 December 1972, MHCHAOS Files, Doc ID C00016396.

⁷⁰ Cf. Rafalko, MH/CHAOS, 12; on the CCCT see Timothy J. Naftali, Blind Spot: The Secret History of American Counterterrorism (New York: Basic Books, 2005), 55–59.

⁷¹ Cf. Lisa Stampnitzky, Disciplining Terror: How Experts Invented ‘Terrorism’ (Cambridge: Cambridge University Press, 2013), 100.

⁷² Cf. Christina Prell, Social Network Analysis: History, Theory and Methodology (London: SAGE, 2012), 19f.

⁷³ Cf. Rockefeller Commission Report, 143f.

Additional information

Notes on contributors

Jens Wegener

Jens Wegener is a postdoctoral researcher at the Department of History, Ruhr-Universität Bochum. He is currently part of a research team investigating the impact of computerization of intelligence services in Germany and the United States from the late 1960s to 1980s. He received his Ph.D. from the Department of History and Civilization at the European University Institute in Florence, Italy.