The Perils of Shallow Theory: Intelligence Reform and the 9/11 Commission

REFERENCES

• The National Commission on Terrorist Attacks upon the United States , The 9/11 Commission Report ( New York : W.W. Norton , 2004 ), pp. 339 – 348 .
   Google Scholar
• 9/11 Commission Report , p. 344 . [CSA]
   Google Scholar
• 9/11 Commission Report , p. 346 . [CSA]
   Google Scholar
• 9/11 Commission Report , p. 344 . [CSA]
   Google Scholar
• Making a similar argument are Stan A. Taylor and David Goldman , “ Intelligence Reform: Will More Agencies, Money, and Personnel Help? ” Intelligence and National Security , Vol. 19 , No. 3 , Autumn 2004 , pp. 416 – 435 . [CSA] [CROSSREF]
   Google Scholar
• Roger Z. George , “ Fixing the Problem of Analytical Mind-Sets: Alternative Analysis ,” International Journal of Intelligence and CounterIntelligence , Vol. 17 , No. 3 , Autumn 2004 , pp. 385 – 404 , at p. 391. [CSA] [CROSSREF]
   Google Scholar
• Roger Z. George , “ Fixing the Problem”; Stephen Marrin, “CIA's Kent School: Improving Training for New Analysts ,” International Journal of Intelligence and CounterIntelligence , Vol. 16 , No. 4 , Winter 2003 , pp. 609 – 637 . [CSA]
   Google Scholar
• 9/11 Commission Report , pp. 408 – 409 . [CSA]
   Google Scholar
• Intelligence Reform and Terrorism Prevention Act of 2004, S.2845, sec. 102A ; http://thomas.loc.gov/cgi-bin/query/C?c108:./temp/~c108Lpf8R0 .
   Google Scholar
• Aaron Wildavsky , “T he Self-Evaluating Organization ,” Public Administration Review , Vol. 32 , No. 5 , September/October 1972 , pp. 509 – 520 . [CSA]
   Web of Science ®Google Scholar
• 9/11 Commission Report , p. 411 . [CSA]
   Google Scholar
• Robert Jervis , Perception and Misperception in International Politics ( Princeton , NJ : Princeton University Press , 1976 ), pp. 117 – 202 .
   Google Scholar
• For an excellent history of intelligence-policy relations, see Christopher Andrew , for the President's Eyes Only: Secret Intelligence and the American Presidency from Washington to Bush ( New York : HarperCollins , 1995 ). For discussions of why policymakers ignore intelligence, see Richard Betts, “Analysis, War, and Decision: Why Intelligence Failures Are Inevitable,” World Politics, Vol. 32, No. 1, 1978, pp. 61–89; Michael Handel, “The Politics of Intelligence,” Intelligence and National Security, Vol. 2, No. 4, October 1987, pp. 5–46; Thomas L. Hughes , The Fate of Facts in a World of Men ( New York : Foreign Policy Association , Headline Series No. 233 , 1976 ); and Hans Heymann , “ Intelligence/Policy Relationships ,” in Alfred C. Maurer , Marion D. Tunstall , and James M. Keagle , eds., Intelligence: Policy and Process ( Boulder , CO : Westview Press , 1985 ), pp. 57 – 66 .
   Google Scholar
• The Commission called for a National Intelligence Director (NID), while the final legislation designated a Director of National Intelligence (DNI). For the sake of clarity we use only the latter title .
   Google Scholar
• 9/11 Commission Report , p. 415 .
   Google Scholar
• Diversity is a laudable goal because the intelligence community badly needs an injection of professional analysts with linguistic and area-specific expertise. Broader outreach efforts will also increase the available talent pool. But we must be very clear about the goals of diversity and the trade-offs involved. Past discussions have argued for diversity on legal and moral grounds, while remaining quite vague on the substantive impact of diversification. See, for example, Alton Dunham , “ Leading a Diverse Workforce into the 21st Century ,” Defense Intelligence Journal , Vol. 7 , No. 1 ( 1998 ), pp. 89 – 105 . [CSA]
   Google Scholar
• The DNI “ shall establish a process and assign an individual or entity the responsibility for ensuring that, as appropriate, elements of the intelligence community conduct alternative analysis … of the information and conclusions in intelligence products .” Intelligence Reform Act , sec. 1017 .
   Google Scholar
• Intelligence Reform Act , secs. 1019 – 1020 .
   Google Scholar
• Intelligence Reform Act , sec. 1018 .
   Google Scholar
• 9/11 Commission Report , p. 416 .
   Google Scholar
• Gregory F. Treverton , Reshaping National Intelligence for an Age of Information ( Cambridge , MA : Cambridge University Press , 2001 ), pp. 167 – 176 .
   Google Scholar
• Intelligence Reform Act , sec. 2002 .
   Google Scholar
• 9/11 Commission Report , pp. 427 – 428 .
   Google Scholar
• Intelligence Reform Act , secs. 7306 – 7307 .
   Google Scholar
• 9/11 Commission Report , p. 353 .
   Google Scholar
• 9/11 Commission Report , p. 353 – 354 . The NSA had some information on the two terrorists in late 1999, but did little to follow up, as it was not asked to by any of its “customers” in the intelligence community. The CIA failed to appreciate the significance of the two names and did not launch a major effort to track them prior to their entry into the U.S., though some efforts were made. This failure was compounded when the FBI was not informed that the two might attempt to enter the U.S. and therefore failed to place the names on a watchlist .
   Google Scholar
• 9/11 Commission Report , p. 357 .
   Google Scholar
• Tenet specifically designated Osama bin Laden a “Tier 0” priority, the highest tier in the prioritization matrix then in use. See Steve Coll , Ghost Wars: The Secret History of the CIA, Afghanistan, and Bin Laden, From the Soviet Invasion of Afghanistan to September 10, 2001 ( New York : The Penguin Press , 2004 ), p. 435 .
   Google Scholar
• 9/11 Commission Report , p. 357 – 358 .
   Google Scholar
• 9/11 Commission Report , p. 419 .
   Google Scholar
• See Bill Owens , with Ed Offley , Lifting the Fog of War ( New York : Farrar, Strauss, and Giroux , 2000 ).
   Google Scholar
• See David Talbot , “How Technology Failed in Iraq,” MIT Technology Review , November 2004 .
   Google Scholar
• On the Myers-Briggs personality test administered by the CIA to new personnel, many analysts score as highly introverted, making the formation of these interagency relationships difficult apart from any bureaucratic restrictions . [CSA]
   Google Scholar
• The best example of the failure of centralization to solve infighting is provided by the history of the Secretary of Defense (SecDef). The SecDef was originally intended to coordinate the budget and actions of the Army, Air Force, and Navy Departments. Yet after nearly four decades and several strong-willed SecDefs such as Robert McNamara, Congress concluded that the services were still just as fractious as ever. This led to the oft-heralded Goldwater–Nichols reform of 1986, promoting “jointness” among the services. Nearly twenty years after its passage, Goldwater–Nichols has not ended tension among services over priorities. At best, it has mutated fierce competition among services into an exercise in log-rolling, so that each service gets its piece of the budgetary pie to spend as it wishes. The Air Force's continued pursuit of the expensive F/A-22 airplane despite its unclear role in supporting near-term joint operations such as counterinsurgency is one example of this continuing problem . [CSA]
   Google Scholar
• Representative examples include Michael Handel , “ The Yom Kippur War and the Inevitability of Surprise ,” International Studies Quarterly , Vol. 21 , No. 3 ( September 1977 ), pp. 461 – 502 ; Richard K. Betts , “ Analysis, War, and Decision”; Steve Chan, “The Intelligence of Stupidity: Understanding Failures in Strategic Warning ,” American Political Science Review , Vol. 73 , No. 1 , March 1979 , pp. 171 – 180 ; Richard K. Betts, Surprise Attack: Lessons for Defense Planning (Washington, DC: Brookings, 1982); Ephraim Kam, Surprise Attack: The Victim's Perspective (Cambridge, MA: Harvard University Press, 1988); and Jack Davis, “Strategic Warning: If Surprise Is Inevitable, What Role for Analysis?” Sherman Kent Center for Intelligence Analysis Occasional Paper, Vol. 2, No. 1, January 2003; http://www.cia.gov/cia/publications/Kent_Papers/vol2no1.htm . [CSA]
   Web of Science ®Google Scholar
• “I can look at a knot in a piece of wood,” said poet William Blake, “until it frightens me.” [CSA]
   Google Scholar
• Winnefeld is quoted in Carl Connetta and Charles Knight, “Dealing with Uncertainty: The New Logic of American Military Planning,” Project on Defense Alternatives, February 1998 ; http://www.comw.org/pda/bullyweb.html . Futuristic weapons systems are also up against generous military pay and benefit policies. See James Flanigan, “Troop Costs Take Bigger Bite out of Defense-Contract Pie,” The Los Angeles Times, 2 January 2005, p. C1; and Cindy Williams, “Making the Cuts, Keeping the Benefits,” The New York Times, 11 January 2005, p. 19 .
   Google Scholar
• Post-hoc reviews of Pearl Harbor, the Yom Kippur War, and the 1998 Indian nuclear test all emphasize this problem. On Congress's 1946 review of Pearl Harbor, see Roberta Wohlstetter , Pearl Harbor: Warning and Decision ( Palo Alto : Stanford University Press , 1962 ). On Israel's post-mortem, see Ephraim Kahana, “Early Warning Versus Concept: The Case of the Yom Kippur War,” Intelligence and National Security, Vol. 17, No. 2, Summer 2002, pp. 81–104 , and Eliot A. Cohen and John Gooch , Military Misfortunes: The Anatomy of Failure in War ( New York : The Free Press , 1990 ), pp. 95 – 132 . On the Jeremiah Commission report that examined the U.S. failure to anticipate India's nuclear test, see Walter Pincus, “Spy Agencies Faulted for Missing Indian Tests,” The Washington Post, 3 June 1998, p. A18. For a more general description of analytical tunnel vision, see Richards Heuer , Jr. , Psychology of Intelligence Analysis ( Washington , DC : CIA Center for the Study of Intelligence , 1999 ); www.cia.gov/csi/books/19104/art3.html .
   Google Scholar
• The basic response to 9/11 has been to increase the size and funding of the Intelligence Community. Its budget has risen by an estimated $10 billion since the attacks, and President George W. Bush has called for a substantial increase in the number of analysts. Along with calls for more imagination, this heady expansion may undermine efforts to prioritize among threats. Stephen Daggett, “The U.S. Intelligence Budget: A Basic Overview,” Congressional Research Service, 24 September 2004; http://www.fas.org/irp/crs/RS21945.pdf .
   Google Scholar
• Terms taken from conversations with intelligence community personnel. For additional examples of historic DIA-CIA tension , see Thomas Powers , The Man Who Kept the Secrets: Richard Helms and the CIA ( New York : Alfred A. Knopf, 1979 ), pp. 160 – 161 , 173 – 176 , and 212 – 213 .
   Google Scholar
• On the differences between the operational and analytic components of the CIA , see Thomas Powers , The Man Who Kept the Secrets , pp. 35 – 37 .
   Google Scholar
• 9/11 Commission Report , p. 354 .
   Google Scholar
• 9/11 Commission Report , p. 353 .
   Google Scholar
• Executive Order 12333, United States Intelligence Activities, 4 December 1981, section 1.12(b). Text available online at : http://www.reagan.utexas.edu/resource/speeches/1981/120481d.htm .
   Google Scholar
• For more detailed discussion of Tenet's memo and the lack of response in the community, see Steve Coll , Ghost Wars , pp. 435 – 436 . Coll argues that resource allocation could have been changed by the President and/or Congress and that Tenet's attempts to get more money for the “war on terror” were denied by both the White House and Congress .
   Google Scholar
• This point is not a new one. The 1996 Commission on the Roles and Capabilities of the U.S. Intelligence Community chaired by Harold Brown, notes: “Intelligence agencies cannot operate in a vacuum. Like any other service organization, intelligence agencies must have guidance from the people they serve. They exist as a tool of government to gather and assess information, and if they do not receive direction, chances are greater that resources will be misdirected and wasted. Intelligence agencies need to know what information to collect and when it is needed. They need to know if their products are useful and how they might be improved to better serve policymakers. Guidance must come from the top. Policymaker direction should be both the foundation and the catalyst for the work of the Intelligence Community.” Preparing for the 21st Century: An Appraisal of U.S. Intelligence ( Washington , DC : Government Printing Office , 1996 ), p. 29 .
   Google Scholar
• Ray S. Cline , “ Is Intelligence Over-Coordinated? ”, Studies in Intelligence , Vol. 1 , No. 4 , Fall 1957 . [CSA]
   Google Scholar
• Gregory F. Treverton and Peter A. Wilson , “True Intelligence Reform Is Cultural, Not Just Organizational Chart Shift,” The Christian Science Monitor , 13 January 2005 ; http://search.csmonitor.com/search_content/0113/p09s02-coop.html .
   Google Scholar
• Jonathan Kirshner , “Rationalist Explanations for War?”, Security Studies , Vol. 10, No. 1, Autumn 2000, pp. 143–150. [CSA]
   Web of Science ®Google Scholar
• Available as Pearl Harbor Attack, Hearings before the Joint Committee on the Investigation of the Pearl Harbor Attack. The 39-volume set of hearings includes the proceedings of the Roberts Commission, the Army Pearl Harbor Board, the Navy Court of Inquiry, the Hart inquiry, the Hewitt investigation and the Clausen investigation . Roberta Wohlstetter , Pearl Harbor: Warning and Decision ( Stanford , CA : Stanford University Press , 1962 ) remains the seminal work on Pearl Harbor .
   Google Scholar
• See Michael Warner , ed., Central Intelligence: Origin and Evolution ( Washington , DC : Center for the Study of Intelligence , 2001 ), pp. 1 – 2 , for comment on the direct impact of the Pearl Harbor investigation on the National Security Act of 1947. This impact was certainly aided by the strong relationship between Chairman Barkley and President Truman. The two had worked together closely when Barkley was Senate majority leader and Truman Vice President. Barkley became Truman's running mate in 1948 .
   Google Scholar
• Available in Hebrew as Duach Va'adat Agranat . [CSA]
   Google Scholar
• See Nina Gilbert, “Law to Halt Agranat Commission Report,” Jerusalem Post, 26 January 2005. The Agranat Report was to be declassified in January 2005, but changes to the law delayed and will probably prevent the declassification for several more decades . [CSA]
   Google Scholar
• This summary of the Agranat Commissions findings draws on Ephraim Kahana , “ Early Warning Versus Concept ,” and Eliot A. Cohen and John Gooch , Military Misfortunes , pp. 112 – 117 . [CSA]
   Google Scholar
• See Ephraim Kahana , “ Reorganizing Israel's Intelligence Community ,” International Journal of Intelligence and CounterIntelligence , Vol. 15 , No. 3 , Fall 2002 , pp. 415 – 428 . [CSA] [CROSSREF]
   Google Scholar
• The findings of Jervis's report are drawn from Bob Woodward , Veil: The Secret Wars of the CIA ( New York : Simon and Schuster , 1987 ), pp. 108 – 111 . Jervis has publicly admitted his link to the CIA, but has not commented on the Iran postmortem. Many of the details of Jervis's report are supported by William J. Daugherty, “Behind the Intelligence Failure in Iran,” International Journal of Intelligence and CounterIntelligence, Vol. 14, No. 4, Winter 2001–2002, pp. 449–484. Daugherty was serving in Iran as an officer in the CIA's Directorate of Operations at the time of the fall of the Shah and became one of the hostages held by Iran from 1979 to 1981. In addition to the Jervis Postmortem, the House Subcommittee on Evaluation of the Permanent Select Committee on Intelligence produced a short (8-page) staff report entitled Iran: Evaluation of US Intelligence Performance Prior to November 1978 ( Washington , DC : Government Printing Office , 1979 ). The Iran failure also prompted the CIA's Center for the Study of Intelligence to produce guidelines for warning of revolution. See Robert Hopkins , Warnings of Revolution ( Washington , DC : Center for the Study of Intelligence , 1980 ).
   Google Scholar
• Bob Woodward , Veil: The Secret Wars of the CIA , p. 110 .
   Google Scholar
• See Ibid., p. 111 and Executive Order 12333, United States Intelligence Activities, and Executive Order 12334, President's Intelligence Oversight Board, both from 4 December 1981 . [CSA]
   Google Scholar
• The press conference is available online : www.odci.gov/cia/public_affairs/press_release/1998/jeremiah.html . Other sources which comment on the Jeremiah Report and/or the failure to predict the Indian test are Walter Pincus, “Spy Agencies Faulted for Missing Indian Tests”; Mary O. McCarthy, “The Mission to Warn: Disaster Looms,” Defense Intelligence Journal, Vol. 7, No. 2, Fall 1998; Richard A. Best, Jr., Intelligence Issues for Congress (Washington, DC: Congressional Research Service, 2004); and Richard A. Best, Jr., U.S. Intelligence and India's Nuclear Tests: Lessons Learned (Washington, DC: Congressional Research Service, 1998) .
   Google Scholar
• See David Jeremiah press conference .
   Google Scholar
• Available as The Need for Change: The Legacy of TMI, Report of the President's Commission on the Accident at Three Mile Island ( Washington , DC : Government Printing Office , 1979 ), with four volumes of Reports from the Technical Assessment Task Force . Charles Perrow , Normal Accidents: Living with High Risk Technologies, , 2nd ed. ( Princeton , NJ : Princeton University Press , 1999 ), pp. 15 – 31 and 334 – 339 provide a reasonable summary of the Commission's findings on the course of the accident .
   Google Scholar
• See The Need for Change: The Legacy of TMI , pp. 8 – 10 .
   Google Scholar
• Ibid. , pp. 61 – 71 .
   Google Scholar
• Douglas Porch and James J. Wirtz , “Surprise and Intelligence Failure,” Strategic Insights , Vol. 1 , No. 7 , September 2002 ; http://www.ccc.nps.navy.mil/si/sept02/homeland.asp . [CSA]
   Google Scholar
• Robert Jervis , Perception and Misperception , pp. 117 – 202 . See also Robert Jervis , “ Perceiving and Coping with Threat ,” in Robert Jervis , Richard Ned Lebow , and Janice Gross Stein , eds., Psychology and Deterrence ( Baltimore , MD : Johns Hopkins University Press , 1989 ), pp. 13 – 33 .
   Google Scholar
• Yuen Foong Khong , Analogies at War: Korea, Munich, Dien Bien Phu, and the Vietnam Decisions of 1965 ( Princeton , NJ : Princeton University Press , 1992 ). On varieties of cognitive biases, see Deborah Welch Larson , Origins of Containment: A Psychological Explanation ( Princeton , NJ : Princeton University Press , 1989 ), pp. 24 – 65 .
   Google Scholar
• Prospect theory covers this and related psychological tendencies. The theory was developed in Daniel Kahneman and Amos Tversky , “ Prospect Theory: An Analysis of Decision under Risk ,” Econometrica , Vol. 47 , No. 2 , March 1979 , pp. 263 – 292 . See also the special issue of Political Psychology, Rose McDermott, ed., Vol. 25, No. 2, April 2004 . [CSA]
   Web of Science ®Google Scholar
• Sherman Kent , “A Crucial Estimate Relived,” Studies in Intelligence , Spring 1964 . Reprinted in Sherman Kent and the Board of National Estimate: Collected Essays ( Washington , DC : Center for the Study of Intelligence , 1994 ); http://www.cia.gov/csi/books/shermankent/toc.html .
   Google Scholar
• See especially Richards J. Heuer , Jr. , Psychology of Intelligence Analysis , www.cia.gov/csi/books/19104/art3.html . [CSA]
   Google Scholar
• See the excellent discussion of these tradeoffs in Stephen Marrin , “ Preventing Intelligence Failures by Learning from the Past ,” International Journal of Intelligence and CounterIntelligence , Vo. 17 , No. 4 , Winter 2004–2005 , pp. 655 – 672 , at 664 – 665 . [CSA] [CROSSREF]
   Web of Science ®Google Scholar
• On Ball's “ritual” objections, see Leslie H. Gelb and Richard K. Betts , The Irony of Vietnam: The System Worked ( Washington , DC : Brookings , 1979 ), pp. 125 and 146 .
   Google Scholar
• Anne Hessing Cahn , “ Team B: The Trillion Dollar Experiment, Part I ,” Bulletin of the Atomic Scientists , April 1993 ; www.thebulletin.org/issues/1993/a93/a93Teamb.html. For a more general condemnation of competitive analysis, see Kevin P. Stack , “ A Negative View of Competitive Analysis ,” International Journal of Intelligence and CounterIntelligence , Vol. 10 , No. 4 , Winter 1997–1998 , pp. 456 – 464 . [CSA]
   Google Scholar
• Sherman Kent , “ The Law and Custom of the National Intelligence, Estimate ,” part II, DCI Miscellaneous Study No. 12 , 1976 , reprinted in Sherman Kent and the Board of National Estimates . [CSA]
   Google Scholar
• Robert M. Gates , From the Shadows: The Ultimate Insider's Story of Five Presidents and How They Won the Cold War ( New York : Simon and Schuster , 1996 ), pp. 223 – 224 . Italics in original .
   Google Scholar
• John A. Gentry , “ Intelligence Analyst/Manager Relations at the CIA ,” Intelligence and National Security , Vol. 10 , No. 4 , October 1995 , pp. 133 – 146 . [CSA]
   Google Scholar
• Roger Z. George , “ Fixing the Problem ,” pp. 390 – 402 , at 402. See also Stephen Marrin, “Preventing Intelligence Failures,” p. 668. Training for new analysts is conducted in the Career Analyst Program. There is some indication that the curriculum has been shortened . [CSA]
   Google Scholar
• Comments at a panel discussion, “After Intelligence Reform,” Harvard University Institute of Politics, 9 February 2005 . [CSA]
   Google Scholar
• See Charles Perrow , Normal Accidents , pp. 62 – 100 for a detailed discussion of normal accidents theory. Eliot A. Cohen and John Gooch apply the concept of normal accidents to military failure. See Military Misfortunes , pp. 21 – 23 . [CSA]
   Google Scholar
• Charles Perrow , Normal Accidents , pp. 282 – 293 , describes specific early warning systems of nuclear attack such as those operated by the NORAD as moderately complex and coupled, but notes that, unlike nuclear power, the system is “intentional and self-activating. The enemy can intervene in the operation… through decoys, deception, disruption of communications, theft of designs, even perhaps the corruption of operators … This adds another dimension of complexity to something already complex enough, and greatly limits the possibility of recovering from failures before the system itself fails. There does not seem to be any end to this increase in complexity and coupling.” Perrow, p. 292. This logic applies at least as much to less technical forms of warning intelligence . Scott Sagan , The Limits of Safety: Organizations, Accidents and Nuclear Weapons ( Princeton , NJ : Princeton University Press , 1993 ) applies normal accidents theory to a number of cases involving early warning and finds the evidence in support of the theory compelling .
   Google Scholar
• Charles Perrow Normal Accidents , pp. 329 – 334 , discusses optimum organization for each type of system .
   Google Scholar
• Ibid. , p. 335 .
   Google Scholar
• Richard K. Betts , Surprise Attack , pp. 87 – 111 ; Michael Handel , “ The Yom Kippur War ,” pp. 464 – 468 ; and Jack Davis , “ Strategic Warning: If Surprise is Inevitable, What Role for Analysis? ”
   Google Scholar
• Richard K. Betts , “ Analysis, War, and Decision ”; and Steve Chan , “ The Intelligence of Stupidity .” Ephraim Kam is most pessimistic about the ability to prevent surprise attack because of the complexity of events that precede the outbreak of war . See Ephraim Kam , Surprise Attack . For a more general review of policymakers’ reliance on preconceptions , see Robert Jervis , Perception and Misperception , pp. 117 – 202 .
   Google Scholar
• Richard K. Betts , Surprise Attack , p. 17 .
   Google Scholar
• Ephraim Kam argues that focusing on analysis will provide only marginal improvement in warning intelligence. We agree, but marginal improvement is better than costly stagnation through organizational reform . Surprise Attack , pp. 216 – 222 .
   Google Scholar