Intelligence Failures: What Are They Really and What Do We Do about Them?

Abstract

Intelligence failures occur for more reasons than just sloppy tradecraft and are often attributable to decision-makers as well as to the intelligence community. Before exploring the subjective nature of intelligence failures, this article first discusses three foundational concepts underlying them: process vs. product, fact vs. judgment, and prediction. It then outlines major components of intelligence failures: accuracy, surprise, and the role of decision-makers, particularly unrealistic expectations and the use or non-use of intelligence. The article concludes with a discussion of what the intelligence community and decision-makers can do to deal with these three components.

Acknowledgments

All statements of fact, opinion, or analysis expressed in this article are those of the author. Nothing in this article should be construed as asserting or implying US government endorsement of its factual statements and interpretations. This article has gone through the pre-publication review process of the Office of the Director of National Intelligence (ODNI).

Notes

Email: [email protected]¹Some examples include Operation Zapata in 1961 (Bay of Pigs invasion), Operation Eagle Claw in 1980 (Iranian hostage rescue attempt), and Operation Continue Hope in 1993 (attempt to subdue Aidid-led guerrilla attacks in Somalia).

²Woodrow J. Kuhns, ‘Intelligence Failures: Forecasting and the Lessons of Epistemology’ in Richard K. Betts and Thomas G. Mahnken (eds.) Paradoxes of Strategic Intelligence: Essays in Honor of Michael I. Handel (London: Frank Cass 2003) p.80; see also John A. Gentry, ‘Intelligence Failure Reframed’, Political Science Quarterly 123/2 (2008) pp.247–70.

³For example, ‘Military Intelligence Failures’, University of California, Davis, 9–11 June 2005 and ‘Intelligence Failures and Cultural Misperceptions: Asia, 1945 till the present’, Netherlands Intelligence Studies Association, 27–28 September 2008. An example of a panel on the subject in a larger conference is ‘American Intelligence Failures and Successes: The Lessons for the Future?’, part of the conference ‘Breaking Down the Walls: Increasing the Discourse in the American Policy Making Community’, Arizona State University, 31 March–2 April 2010.

⁴Center for the Study of Intelligence, ‘Our First Line of Defense’: Presidential Reflections on US Intelligence (Washington, DC: Center for the Study of Intelligence 1996); one rare publicly known intelligence success lead to the demise of Osama Bin Laden on 2 May 2011. See Ken Dilanian, ‘In Finding Osama bin Laden, CIA Soars from Distress to Success’, Los Angeles Times, 8 May 2011, <www.latimes.com/news/nationworld/nation/la-na-bin-laden-cia-20110508,0,7184857.story> (accessed 9 May 2011).

⁵Other terms often used to designate users of intelligence who are outside of the IC include policymaker, consumer, customer, client, warfighter, or law enforcement officer. Users inside the IC are intermediate consumers, who use intelligence information provided to them to craft other products for the ultimate consumers – those outside the IC. For simplicity purposes, the term ‘decision-maker’ will be used throughout this article to refer to individuals outside the IC who use intelligence products.

⁶For example: Bill Gertz, Breakdown: How America's Intelligence Failures Led to September 11 (Washington, DC: Regnery Publishing, Inc. 2002); Senate Select Committee on Intelligence, Report on the US. Intelligence Community's Prewar Intelligence Assessments on Iraq, 7 July 2004; Angelo M. Codevilla, Why U.S. Intelligence is Inadequate, and How to Fix It, Occasional Papers Series, No. 2 (Washington, DC: The Center for Security Policy, July 2004).

⁷For example, the IC was severely criticized because it did not discover the ‘Christmas bomber’ until after he had boarded a transatlantic flight on 25 December 2009 and attempted to ignite explosives hidden in his underwear. The IC has also been denounced for numerous National Intelligence Estimates (NIE), although often for perceived politicization (e.g. the 1995 NIE on the ballistic missile threat or the 2007 NIE on Iranian weapons of mass destruction (WMD)).

⁸Kristan J. Wheaton, ‘Evaluating Intelligence: Answering Questions Asked and Not’, International Journal of Intelligence and Counterintelligence 22/4 (2009–2010), p.618.

⁹Numerous publications have been written on process failures and how to fix them, such as: Mary Sandow-Quirk, ‘A Failure of Intelligence’, Prometheus 20/2 (2002), pp.131–42; Uri Bar-Joseph and Jack S. Levy, ‘Conscious Action and Intelligence Failure’, Political Science Quarterly 124/3 (2009) pp.461–88; ‘Intelligence Failures: Some Historical Lessons, Parts 1 and 2’, The Estimate 15/12 (2002); Michael A. Turner, Why Secret Intelligence Fails, Rev. ed. (Washington, DC: Potomac Books 2006).

¹⁰Richard K. Betts, ‘Analysis, War, and Decisions: Why Intelligence Failures Are Inevitable’, World Politics 31/1 (1978) pp.35–54; see also a relevant article with a similar theme: Russ Travers, ‘The Coming Intelligence Failure’, Studies in Intelligence 40/2 (1996) pp.27–34.

¹¹Intelligence Reform and Terrorism Prevention Act of 2004, Public Law 108–458, December 17, 2004, Section 1019 (a).

¹²Similar words are often used interchangeably to express the same thought about analytical conclusions, e.g. estimate, inference, assessment, opinion, or judgment. Critics would also words such as guess, conjecture, speculation, or assumption. ‘Estimate’ is often used in connection with products about the future, e.g. National Intelligence Estimate. For purposes of this article, the word ‘judgment’ will generally be used to describe analytical conclusions about the past and present; ‘estimate’ will be used for judgments about the future.

¹³Martin Petersen, ‘What I Learned in 40 Years of Doing Intelligence Analysis for US Foreign Policymakers’, Studies in Intelligence 55/1 (2011) p.17.

¹⁴For a discussion of how the White House did not appreciate DNI James Clapper's view of Gadhafi's anticipated endurance, see Michael V. Hayden, ‘Is it OK for Spy Agency Chiefs to Tell the Truth?’ CNN, 22 April 2011, <http://www.cnn.com/2011/OPINION/04/11/hayden.intelligence.truth/index.html?hpt=C2> (accessed 11 April 2011).

¹⁵Three excellent articles on politicization are: Robert M. Gates, ‘Guarding Against Politicization’, Studies in Intelligence 36/5 (1992) pp.5–13; Michael Handel, ‘The Politics of Intelligence’, Intelligence and National Security 2/4 (1987) pp.5–46; Richard K. Betts, ‘Politicization of Intelligence: Costs and Benefits’ in Richard K. Betts and Thomas G. Mahnken (eds.) Paradoxes of Strategic Intelligence: Essays in Honor of Michael I. Handel (London: Frank Cass 2003) pp.59–79.

¹⁶Wheaton, ‘Evaluating Intelligence’, p.617.

¹⁷For products dealing with the past and present, certainty is theoretically possible. However, truth may not be knowable in a practical sense. Insufficient funds, personnel, time, or access may preclude the IC from obtaining the answer in sufficient time to impact a decision. In addition, unwillingly adversaries may employ denial and deception techniques to prevent collection.

¹⁸Robert Mandel, ‘On Estimating Post-Cold War Enemy Intentions’, Intelligence and National Security 24/2 (2009) pp.194–215.

¹⁹For a useful discussion on probabilities, see Joab Rosenberg, ‘The Interpretation of Probability in Intelligence Estimation and Strategic Assessment’, Intelligence and National Security 23/2 (2008) pp.139–52.

²⁰Yogi Berra Quotes, <http://www.digitaldreamdoor.com/pages/quotes/yogiberra.html> (accessed 7 April 2011).

²¹Dusko Doder, ‘Did the CIA Blow the Call?’ review of: Breakdown: How America's Intelligence Failures Led to September 11, by Bill Gertz, The Nation, 4 November 2002.

²²Marcus Baram, ‘CIA's Mideast Surprise Recalls History of Intelligence Failures’, The Huffington Post, 14 February 2011, <http://www.globalsecurity.org/org/news/2011/110214-egypt-intelligence.htm> (accessed 20 May 2011).

²³See for example: Richard K. Herrmann and Jong Kun Choi, ‘From Prediction to Learning’, International Security 31/4 (2007) pp.132–61; Puong Fei Yeh, ‘Using Prediction Markets to Enhance US Intelligence Capabilities’, Studies in Intelligence 50/4 (2006) pp.37–49; Captain Paulo Shakarian, ‘The Future of Analytical Tools: Prediction in a Counterinsurgency Fight’, Military Intelligence Professional Bulletin 34/4 (2008) pp.80–7.

²⁴Lisa Daniel, ‘Panetta: Intelligence Community Needs to Predict Uprisings’, Defense.gov, 11 February 2011, <http://www.defense.gov/news/newsarticle.aspx?id=62790> (accessed 26 April 2011).

²⁵Greg Miller, ‘Faulty Comment on Egypt by Panetta Leads to Confusion’, The Washington Post, 11 February 2011, <http://www.washingtonpost.com/wp-dyn/content/article/2011/02/10/AR2011021007642.html> (accessed 7 April 2011).

²⁷Daniel, ‘Panetta’. Earthquakes, like weather, are caused almost exclusively by the forces of nature and the principles of physics. They should be theoretically easier to predict than human action, which is subject to the whims of people who may change their minds at any time. Even though physical phenomena are more predictable, weather forecasters are often wrong, even about events only several days into the future.

²⁶Kimberly Dozier, ‘Spy Chiefs Defend Mideast Work but Miss Egypt Call,’ Associated Press, 11 February 2011, <http://www.msnbc.msn.com/id/41529505/ns/politics-more_politics/> (accessed 26 April 2011); see also James Clapper, ‘Remarks as delivered by James R. Clapper Director of National Intelligence, Open Hearing on the Worldwide Threat Assessment House Permanent Select Committee on Intelligence, February 10, 2011’, <http://www.dni.gov/testimonies/20110210_testimony_hpsci_clapper.pdf> (accessed 27 April 2011).

²⁸Marc Ambinder, ‘An Intelligence Failure in Egypt?’ The Atlantic, 5 February 2011, <http://www.theatlantic.com/politics/archive/2011/02/an-intelligence-failure-in-egypt/70820/> (accessed 26 May 2011).

²⁹James Clapper, ‘Statement for the Record by James R. Clapper, Jr., Nominee for the Position of Director of National Intelligence Before the Senate Select Committee on Intelligence United States Senate, 20 July 2010’, <http://www.dni.gov/testimonies/20100720_testimony.pdf> (accessed 27 April 2011) .

³⁰Dan De Luce, ‘CIA Chief's Egypt Remark Causes Confusion’, The Sydney Morning Herald, <http://news.smh.com.au/breaking-news-world/cia-chiefs-egypt-remark-causes-confusion-20110211-1apfo.html> (accessed 26 April 2011).

³¹D. McMorrow, Rare Events (McLean, VA: JASON, The MITRE Corporation 2009) p.7.

³²Philip Zelikow, ‘20^th Century and the Onset of the Cold War’, speech sponsored by Brookings Institution, Keswick, VA, 16 March 2011.

³³Jim Steinberg, ‘Deputy Secretary of State Steinberg on Intelligence Support to Policymakers’, speech, Central Intelligence Agency auditorium, 18 February 2011.

³⁴See also Paul R. Pillar, ‘Predictive Intelligence: Policy Support or Spectator Sport?’ SAIS Review 28/1 (2008) pp.25–35.

³⁵This is consistent with the definition of intelligence failures provided by Ehud Eiran, ‘Preventing the “Next Intelligence Failure”? The Three Tensions of Investigating Intelligence Failures’, paper presented at the 46th Annual Convention of the International Studies Association, Honolulu, Hawaii, 1–5 March 2005, p.4. Parenthetically, the three tensions are time, purpose, and process.

³⁶Philip Shenon, The Commission: The Uncensored History of the 9/11 Investigation (New York: Twelve 2008) pp.151–2.

³⁷In this specific case, the execution of intelligence processes could have been better and some authors have concluded the attacks could have been prevented had the IC exercised more due diligence in following up leads and sharing information more widely; see for example Bob Graham and Jeff Nussbaum, Intelligence Matters: The CIA, the FBI, Saudi Arabia, and the Failure of America's War on Terror, Reprint ed. (Lawrence, KS: University Press of Kansas 2008).

³⁸For examples of ‘accurate’ and ‘inaccurate’ NIEs, see Loch Johnson, ‘Weighing the Value of Estimates’ in The Threat on the Horizon: An Inside Account of America's Search for Security after the Cold War (Oxford: Oxford University Press 2011) pp.173–4.

³⁹Robert Callum, ‘The Case for Cultural Diversity in the Intelligence Community’, International Journal of Intelligence and Counterintelligence 14/1 (2001) p.25.

⁴⁰Robert Jervis, Why Intelligence Fails: Lessons from the Iranian Revolution and the Iraq War (Ithaca, NY: Cornell University Press 2010) p.178.

⁴¹Mike Hayden, ‘Transcript of Director Hayden's Interview with Fox News’, 21 January 2009, <https://www.cia.gov/news-information/press-releases-statements/cia-director-interview-with-fox-news.html> (accessed 3 June 2011).

⁴²An opponent's mistake here means more than just what is scored as an official error. These mistakes can include pitching too close to the center of the plate, positioning oneself defensively without considering a batter's hitting tendencies, overrunning a base, or turning the wrong way to pursue a fly ball, etc.

⁴³Wheaton, ‘Evaluating Intelligence’, p.618.

⁴⁴James P. Finley, ‘Nobody Likes to be Surprised: Intelligence Failures’, Military Intelligence 20/1 (1994) pp.15–21, 40.

⁴⁵Richard K. Betts, ‘Intelligence for Policymaking’, The Washington Quarterly 3/3 (1980) p.128.

⁴⁶Ambinder, ‘An Intelligence Failure in Egypt?’ The article's author goes on to muse: ‘If the CIA thought that Ben Ali [former Tunisian president] would be deposed in, say, a week instead of 48 hours, does that count as a botched call?’

⁴⁷Josh Gerstein, ‘Feinstein: U.S. Intelligence Offered “No Real Warning” on Egypt’, Politico, 8 February 2011, <http://www.politico.com/blogs/joshgerstein/0211/Feinstein_US_intelligence_offered_no_real_warning_on_Egypt.html> (accessed 27 April 2011).

⁴⁸Paul R. Pillar, ‘Don't Blame the Spies’, Foreign Policy, 16 March 2011, <http://www.foreignpolicy.com/articles/2011/03/16/dont_blame_the_spies> (accessed 26 May 2011).

⁴⁹Gentry, ‘Intelligence Failure Reframed’, pp.255–61. Gentry devotes an entire section of the article to ‘Policymaking and Leadership Failures’ and specifically addresses expectations.

⁵⁰Betts, ‘Analysis, War, and Decisions’, p.39.

⁵¹The Institute of World Politics, ‘Institute Event: Experts Identify Anti-Terror Problems & Solutions’, 5 December 2003, <http://www.iwp.edu/news_publications/detail/institute-event-experts-identify-anti-terror-problems-solutions> (accessed 8 April 2011).

⁵²William C. Prillaman and Michael P. Dempsey, ‘Mything the Point: What's Wrong with the Conventional Wisdom about the C.I.A’, Intelligence and National Security 19/1 (2004) p.5.

⁵⁴David Kahn, ‘The Rise of Intelligence’, Foreign Affairs 85/5 (2006) p.134.

⁵⁵John McLaughlin, ‘Serving the Policymaker’ in Roger Z. George and James B. Bruce (eds.) Analyzing Intelligence: Origins, Obstacles, and Innovations (Washington, DC: Georgetown University 2008) p.72.

⁵³Pillar, ‘Don't Blame the Spies’.

⁵⁶See an excellent discussion of how intelligence does or does not influence decisions in Ohad Leslau, ‘The Effect of Intelligence on the Decisionmaking Process’, International Journal of Intelligence and Counterintelligence 23/3 (2010) pp.426–48. See also Amanda J. Gookins, ‘The Role of Intelligence in Policy Making’, SAIS Review 28/1 (2008) pp.65–73.

⁵⁷ Intelligence Community Directive 208, Write for Maximum Utility, Director of National Intelligence, 17 December 2008.

⁵⁸Doug MacEachin and Janne E. Nolan, co-chairs, The Soviet Invasion of Afghanistan in 1979: Failure of Intelligence or of the Policy Process? Working Group Report No. 111 (Washington, DC: Georgetown University 2005) pp.12–13.

⁵⁹Stephen Marrin, ‘The 9/11 Terrorist Attacks: A Failure of Policy not Strategic Intelligence’, Intelligence and National Security 26/2–3 (2011) pp.182–202.

⁶⁰See Erik J. Dahl, ‘Missing the Wake-up Call: Why Intelligence Failures Rarely Inspire Improved Performance’, Intelligence and National Security 25/6 (2010) pp.778–99 on the importance of decision-maker receptivity to intelligence; see also W.R. Baker, ‘The Easter Offensive of 1972: A Failure to Use Intelligence’, Military Intelligence Professional Bulletin 24/1 (1998) pp.40–2, 60.

⁶¹The United States viewed its overthrow of Iranian Prime Minister Mossadegh in 1953 at the time as a success. In light of the Iranian revolution in 1979 and seizure of the US embassy, one could question the long-term success of the policy.

⁶²John Hollister Hedley, ‘Learning from Intelligence Failures’, International Journal of Intelligence and Counterintelligence 18/3 (2005) p.446.

⁶³Petersen, ‘What I Learned in 40 Years of Doing Intelligence Analysis for US Foreign Policymakers’, p.19.

⁶⁴Two excellent works outlining ways to improve intelligence analysis are: Richards J. Heuer, Jr., Psychology of Intelligence Analysis (Washington, DC: Center for the Study of Intelligence 1999) and Rob Johnston, Analytic Culture in the US Intelligence Community (Washington, DC: Center for the Study of Intelligence 2005).

⁶⁵Glenn Hastedt, ‘Intelligence and U.S. Foreign Policy: How to Measure Success?’ International Journal of Intelligence and Counterintelligence 5/1 (1991) pp.49–62; Mark M. Lowenthal, ‘Towards a Reasonable Standard for Analysis: How Right, How Often on Which Issues?’ Intelligence and National Security 23/3 (2008) pp.303–15.

⁶⁶Arthur S. Hulnick, review of Estimative Intelligence: The Purposes of Problems of National Intelligence Estimating by Harold Ford, Conflict Quarterly 14/1 (1994) p.74.

⁶⁷Betts, ‘Intelligence for Policymaking’, p.121.

⁶⁸See for example, Chester A. Crocker, ‘Reflections on Strategic Surprise’ in Patrick Cronin (ed.) The Impenetrable Fog of War: Reflections on Modern Warfare and Strategic Surprise (Westport, CT: Greenwood Publishing Inc. 2008); Steve Chan, ‘Intelligence of Stupidity: Understanding Failures in Strategic Warning’, American Political Science Review 73/1 (1979) pp.171–80; Stephen Marrin, ‘Preventing Intelligence Failures by Learning from the Past’, International Journal of Intelligence and Counterintelligence 17/4 (2004–2005) pp.655–72.

⁶⁹Fareed Zakaria agrees when he states: ‘The goal should instead [of trying to predict events] be preparedness. Government agencies should be readying policymakers and bureaucrats for sharp changes in international, regional and national patterns. They should be imaginative about the possibilities of sudden shifts and new circumstances and force policymakers to confront the scenarios in advance. That is what has distinguished the most successful private-sector firms in managing crises.’ Fareed Zakaria, ‘Risk Management’, Washington Post, 28 April 2011, p.17.

⁷⁰Zelikow, ‘20^th Century and the Onset of the Cold War’.

⁷¹For further discussion on service to decision-makers see Josh Kerbel and Anthony Olcott, ‘Synthesizing with Clients, Not Analyzing for Customers’, Studies in Intelligence 54/4 (2010) pp.11–27.

⁷²Dennis C. Wilder, ‘An Educated Consumer is Our Best Customer’, Office of the Director of National Intelligence, 2010 Galileo Awards Winner; see also Martin Petersen, ‘What We Should Demand from Intelligence’, National Security Studies Quarterly 5/2 (1999) pp.107–13.

⁷³Mark M. Lowenthal, ‘The Real Intelligence Failure? Spineless Spies’, Washington Post, 25 May 2008, <http://www.washingtonpost.com/wp-dyn/content/article/2008/05/22/AR2008052202961.html> (accessed 24 May 2011).

⁷⁴Gregory L. Schulte, ‘From the Balkans to Iran – Coupling Policy and Intelligence to Address the World's Complex Problems’, Lessons learned presentation, McLean, VA, 2 December 2009. Ambassador Schulte is the former executive secretary of the National Security Council.

Additional information

Notes on contributors

Mark A. Jensen

This work was authored as part of the Contributor's official duties as an Employee of the United States Government and is therefore a work of the United States Government. In accordance with 17 U.S.C. 105, no copyright protection is available for such works under U.S. Law.