Analytical innovation in intelligence systems: the US national security establishment and the craft of ‘net assessment’

ABSTRACT

This article develops a theory of analytical innovation in intelligence systems using the craft of the ‘net assessment’ methodology in the US as a case study. Employing congruence method and process-tracing and drawing on multi-archival sources, the article demonstrates that at the roots of analytical innovation are three variables: the setting of a requirement, the conduct of methodological experiments and the synthesis of analytical knowledge. The study also reveals that the nature of analytical intelligence innovation is dyadic, consisting of an organizational and an ideational component.

Acknowledgements

The author would like to thank Dr. John Prados and Prof. Leopoldo Nuti for their comments on previous versions of this paper, as well as the journal’s anonymous reviewers for their constructive criticism and suggestions.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. ODNI, “Meet Lateral Innovation,” June 26, 2019; Zabierek, “The Case for an Intelligence Community Innovation Unit”; AAVV, “The Future of Intelligence Analysis”; and EU Counter-Terrorism Coordinator, “Note from EU Counter-Terrorism Coordinator to Justice Home Affairs Delegations”.

2. Drucker, “The Discipline of Innovation”; Tushman, Anderson and O’Reilly, “Technology Cycles, Innovation Streams,” 3–23; and Gray, The Strategy Bridge, 8.

3. Fagerberg, “Innovation,” 1–26.

4. Christensen, Innovator’s Dilemma.

5. For a broad survey of military innovation studies, see: Grissom, “The Future of Military Innovation Studies”; and Griffin, “Military Innovation Studies”.

6. Rosen, “New Ways of War”.

7. Ross, “On Military Innovation”.

8. Ross, “The Dynamics of Military Technology,” 106–140.

9. Berkowitz & Goodman, Best Truth; Townsend, Cyber Intelligence Tradecraft; and Gioe, Goodman, Stevens, “Intelligence in the Cyber Era”.

10. Herman, Intelligence Services in the Information Age, 33; Omand, “The Cycle of Intelligence”; Clark, Intelligence Analysis; Davies, “The Intelligence Cycle is Dead, Long Live the Intelligence Cycle”; and Johnston, Analytic Culture in the US Intelligence Community.

11. Winterbotham, The Ultra Secret; Jones, The Wizard War; Richelson, The Wizards of Langley; Kosal, Technology and the Intelligence Community; and Warner, “Reflections on Technology and Intelligence Systems”.

12. Sharfman, “Intelligence Analysis”; Zegart, Flawed by Design; Davies, MI6 and the Machinery of Spying; Hammond, “Intelligence Organizations and the Organization of Intelligence”; and Kamarck, Transforming the Intelligence Community.

13. George & Bennett, Case Studies and Theory Development, 87–88, 183–188.

14. Mohr, “The Reliability of the Case Study as a Source of Information,” 65–94.

15. Levy, “Qualitative Methods in International Relations,” 432–454.

16. Steury, “Introduction”.

17. Fischer, “’We May Not Always Be Right, but We’re Never Wrong’,” 98.

18. Gentry, ‘Has the ODNI Improved U.S. Intelligence Analysis?’; Immerman, “Transforming Analysis”; and Nicander, “Understanding Intelligence Community Innovation”.

19. O’Brien, The South African Intelligence Services, 66–68.

20. Samuels, Special Duty, 223; Kotani, “Japanese Intelligence in WWII: Successes and Failures”.

21. Garthoff, “Soviet Leaders, Soviet Intelligence,” 34–36; and Haslam, Near and Distant Neighbours, 215–218.

22. Mandel, “The Occasional Maverick of Analytic Tradecraft”; and Coulthart, “From Laboratory to the WMD Commission”.

23. Jervis, System Effects, 35; and Zegart, Spying Blind, 1–11.

24. Cohen, “Net Assessment”; Rosen, “Net Assessment as an Analytical Concept,” 283–301; Pickett, Roche and Watts, “Net Assessment: A Historical Review,” 158–185.

25. Watts, “Net Assessment in the Era of Superpower Competition,” 27–29; Krepinevich and Watts, The Last Warrior; Karber, Net Assessment for SecDef; Adamsky, “The Art of Net Assessment”; Augier, “Thinking about War and Peace”; and Gentry, “Andrew Marshall-Intelligence Analyst”.

26. FRUS, 1950–1955, “The “Net Estimates” Problem,” August, 25, 1954, 523–530.

27. FRUS, 1950–1955, “Memorandum from DCI to NSC ExSec,” October 14, 1952, 342; “Memorandum from Acting DDDI (Amory) to DCI Smith,” November 25, 1952, 366–368.

28. Leighton, Strategy, Money and the New Look, 117.

29. FRUS, 1950–1955, “CIA Paper: Preparation of Coordinated Evaluations of the Net Capabilities of the USSR to Inflict Direct Injury on the United States,” March 25, 1954, 485–486.

30. The President’s Committee on International Information Activities, Report to the President, June 30, 1953, 3–4, CIA CREST, https://www.cia.gov/library/readingroom/docs/DOC_0000476939.pdf.

31. Leighton, Strategy, Money and the New Look, 277.

32. FRUS, 1950–1955, “NSC Directive 5423 for a Net Capabilities Evaluation Subcommitteev” June 23, 1954, 505–507; “NSC Directive 5511 on A Net Evaluation Subcommittee,” February 14, 1955, 599–601.

33. Hoover Institution Archives (HIA), Seymour Weiss Papers (SWP), Box 6, no folder, “Memorandum from Seymour Weiss to Ambassador Johnson,” July, 20, 1964.

34. The ‘estimate of the situation’ is a military analytical methodology designed and employed to consider the circumstances affecting a military situation and reach a logical decision on the best course of action to accomplish the mission, see: Michel, Historical Development of the Estimate of the Situation.

35. “Report of the Net Capabilities Evaluation Subcommittee,” November 3, 1954; “Summary and Conclusions of 1958 Report of the NSC Net Evaluation Subcommittee,” November 10, 1958; “1959 Report of The NSC Net Evaluation Subcommittee”; “1961 Report of The NSC Net Evaluation Subcommittee”; “1963 Report of The Net Evaluation Subcommittee,” https://nsarchive2.gwu.edu/nukevault/ebb480.

36. FRUS 1964–1968, Vol X, doc. 72, “Draft Memorandum from Secretary of Defense McNamara to President Johnson,” undated.

37. Karber, Net Assessment for SecDef, 31–41.

38. FRUS 1961–1963 Vol. IX, doc. 115, “Draft Memorandum from SecDef McNamara to President Kennedy,’ December 3, 1962; FRUS 1964–1968, Volume X, doc. 213, “Memorandum from the JCS to SecDef Clifford,” August 26, 1968.

39. “Memorandum from CIA ONE, IAP-64: Intelligence Assumptions for Planning,” CIA CREST, https://www.cia.gov/library/readingroom/docs/DOC_0000794608.pdf.

40. “Memorandum for Director of National Estimates,” CIA CREST, https://www.cia.gov/library/readingroom/docs/CIA-RDP79R00967A001000030022-4.pdf; and Seidel, “Intelligence for Defense Planning”.

41. HIA, Albert and Roberta Wohlstetter Papers (ARWP), Box 94, Folder 21, “RAND Oral History Project: Albert Wohlstetter,” 19–20, 40, 58; HIA, Richard B. Foster Papers (RFP), Box 2, 4, 6.

42. “Memorandum: National Intelligence Projections for Planning,” cit., 18.

43. HIA, SWP, Box 8, Folder April–May 1966, “Memorandum from Kitchen to Johnson: State-Defense Long-Range Study Group,” May 17, 1966.

44. “Memorandum from CIA ONE to BNE,” December 17, 1968, CIA CREST, 1 https://www.cia.gov/library/readingroom/docs/CIA-RDP79R00967A001200030005-1.pdf.

45. FRUS 1964–1968, Volume X, doc. 211, “Memorandum from PFIAB Chairman to President Johnson,” August 9, 1968.

46. FRUS 1964–1968, Volume X, doc. 214, “Telegram from Rostow to President Johnson,’ September 3, 1968; Ibid. doc. 215 “Letter from SecDef Clifford to PFIAB Chairman,” September 20, 1968.

47. FRUS 1964–1968, Volume X, doc. 218 and doc. 222, “Letter from PFIAB Chairman to SecDef Clifford,” October 8, 1968; “Report by PFIAB Chairman to President Johnson,” November 25, 1968. For the PFIAB report, see: Richard Nixon Presidential Library (RNPL), NSC Files, HAK Office Files, Administration and Staff Files, Box 1, Folder: PFIAB, “PFIAB Report to the President,’ November 25, 1968.

48. “Memorandum for DDI: Projections for the Military Planner,” November 18, 1968, CIA CREST, https://www.cia.gov/library/readingroom/docs/CIA-RDP85G00105R000100060032-3.pdf.

49. Author’s personal interview with Hon. Henry Rowen, Center for International Security and Cooperation (CISAC), Stanford, November 20, 2014.

50. RNPL, NSC Files, HAK Office Files HAK Administrative and Staff Files, Transition, Box 2, folder NS Planning Material [1 of 4], R.B. Foster, “The Need for a National Net Evaluation Group,” December 9, 1968ʹ. For an example of the SRI long-range net studies see: HIA, R.B. Foster Papers, Box 2, “Mark Earle Jr. & Francis W. Dresch, Research by SRI/SSC in support of the Net Technical Assessment Program,” February, 1973.

51. Gerald Ford Presidential Library (GFPL), Melvin Laird Papers (MLP), Box A58, Folder: 1972 Planning (I), “Memorandum from Buzhardt to SecDef Laird ‘Establishment of Defense Net Assessment Group,” 2; FRUS 1969–1976, Vol. II, Organization and Management of US Foreign Policy 1969–1972, “Memorandum from the Assistant SecDef Intelligence to SecDef Laird,” February 23, 1972, 601.

52. GFPL, MLP, Box A82, Folder Net Assessment 1969–1972 (I), “Memorandum from Harris to Baroody, Proposed Net Assessment: Net Assessment of Continuation of Strategic Bombers in US Strategic Forces in the 1980s,” December 5, 1969; “Memorandum from Harris to Baroody. Proposed Net Assessment: Net Assessment of the Effects of a Reduction in US Strength in NATO,’ December 5, 1969; “Memorandum from Harris to Baroody, ‘Proposed Mission, Organization and Operation of the Net Assessment Office,” January 15, 1970.

53. Hunt, Melvin Laird and the Foundation of a Post-Vietnam Military, 21–22.

54. Schutte, “Casting Net Assessment,” 67.

55. Watts, “Net Assessment in the Era of Superpower Competition,” 34–36.

56. RAND Library and Archives (RLA), A.W. Marshall, Comparisons, R&D Strategy, and Policy Issues, RAND Working Note WN-7630-DDRE (1971), 4–5.

57. ‘RNPL, NSC Files, H-Files, Box H–022, NSC Meeting 6/18/69 SALT (NSSM 28) [1 of 2], “Minute of NSC Meeting”; FRUS 1969–1976, Vol. II, “Memorandum from President Nixon to the Chairman of the DPRC,” April 2, 1970, 224.

58. “Memorandum from Cline to Kissinger, Intelligence Assessments in the NSC System,” September 15,1970, 1.

59. Cline, Secrets, Spies and Scholars, 141.

60. RNPL, H-files, Box H-301, Folder: Staff Committees (2), “Memorandum from Ray Cline to Henry Kissinger, Intelligence Assessments in the NSC System,” September, 15, 1970, 2.

61. RNPL, H-files, Box H-301, Folder: Staff Committees (2), “Memorandum from Kissinger to Cline: Intelligence Assessments in the NSC System,” October 12, 1970 and “Memorandum from Latimer to Kissinger: Intelligence and the NSC,” September 21, 1970.

62. Bexfield, Pournelle and Sheldon, “Interview of Andrew W. Marshall,” 72; Garfinkle, “A Conversation with Andrew Marshall,” https://www.the-american-interest.com/2015/04/09/the-exit-interview-a-conversation-with-andrew-marshall.

63. Library of Congress (LOC), James R. Schlesinger Papers (JSP), Box 39, Folder 13 Special Defense Panel, “Andrew Marshall – Net Assessment of US and Soviet Force Posture Draft,” October, 1970, 18; RLA, Andrew Marshall, “Net Assessment of US and Soviet Force Posture’ Report to NSC SDP,” 1970.

64. LOC, JSP, Box 4, Folder 5, “C.M. Herzfeld – Net Assessment,” September 23, 1970, 1–2.

65. LOC, JSP, Box 4, Folder 5, “Study by OMB and NSC Staffs, A Review of the Intelligence Community,” March, 10, 1971, 21, 46 https://nsarchive2.gwu.edu//NSAEBB/NSAEBB144/document%204.pdf; “Memorandum from Marshall to Schlesinger: Agency Problems,” September 10, 1969’; and Durbin, The CIA and the Politics of US Intelligence Reform, 115–116.

66. INR was at the time working on its own ‘Euro-net assessment’. See FRUS 1969–1976, Vol. II, “Memorandum from the Director of the Net Assessment Group to Assistant for National Security Affairs,” March 15, 1972ʹ, 606.

67. FRUS 1969–1976, Vol. II, “Memorandum from the INR Director to the Under Secretary of State: Factors in Making a Net Assessment of US and Soviet Strategic Forces,” March 8, 1971, 487–490.

68. RNPL, NSC Files, Subject Files, Intelligence Reorganization, 1970–1972, Box 332, Folder 8, “Memorandum from President Nixon, Organization and Management of the US Foreign Intelligence Community,” November 5, 1971.

69. GFPL, MLP, Box A58, Folder: 1972 Planning (I), DoD Directive 5105.39 “Director of Net Assessment,” December 6, 1971.

70. Krepinevich and Watts, The Last Warrior, 87–88.

71. FRUS 1969–1976, Vol. II, ‘‘Memorandum from the Director of the Net Assessment Group to Assistant for National Security Affairs,” March 15, 1972ʹ, 603–606.

72. GFPL, MLP, Box A58, Folder: 1972 Planning (I) “Memorandum from Laird to Baroody: Long-Range Planning and Net Assessment Functions of the Secretary of Defense”.

73. GFPL, MLP, Box B1-B3, Folder: Net Assessment Study Effort 1972, “Memorandum from D.A. SecDef (Technical Evaluation) to Net Threat Assessment Group,” April 12, 1972.

74. GFPL, MLP, Box B1-B3, Folder Net Assessment Study Effort 1972, “Memorandum from JCS Director Joint Staff to JCS Director Plans and Policy,’ April 28, 1972; “Memorandum from M. S. Malkin to Baroody,” April 12, 1972.

75. GFPL, MLP, Box B1-B3, Folder: Net Assessment Study Effort 1972, “Memorandum from (Acting) Director of Net Assessment Wikner to Baroody,’ March 12, 1972; “Memorandum to Deputy SecDef Baroody Annex B ‘Definition of Long-Range Planning Concepts,” B-5.

76. GFPL, MLP, Box A58, Folder: 1972 Planning (I), “Memorandum from Buzhardt to Laird ‘Establishment of Defense Net Assessment Group,” 4.

77. GFPL, MLP, Box A82, Folder: Net Assessment White House, 1972, “Memorandum from Arbesman to Baroody, Review of The Cost of Weapon Systems,” August 10, 1972, 2.

78. GFPL, MLP, Box A82, Folder: Net Assessment White House, 1972, “Memorandum from Arbesman to Baroody, “Follow-on to The Cost of Weapon Systems,” August 25, 1972, 2; “Memorandum from Laird to Kissinger, Net Assessments at the National Level,” August 22, 1972, 4.

79. GFPL, MLP, Box A82, Folder: Net Assessment White House, 1972, “Memorandum by Col. Donald S. Marshall: ‘Luncheon Discussion with Dr. Marshall,” May 23, 1972, 2, 3.

80. CIA Memorandum, “Net Assessment of North Vietnamese and South Vietnamese Military Forces,” April 10, 1972, CIA CREST, https://www.cia.gov/library/readingroom/docs/LOC-HAK-537-7-19-8.pdf.

81. RNPL, NSC H-Files, NSS Memorandums 1969–1974, Box H-199, Folder NSSM 178 [1 of 2], “Memorandum from Kissinger to the SecState, SecDef, DCI, National Net Assessment of the Comparative Efficiency and Effectiveness of the US and Soviet Military Establishments,” undated.

82. GFPL, MLP, Box A82, Folder: Net Assessment White House, 1972, “Memorandum from N.F. Wikner to Baroody, Draft National Security Study Memorandum on the Net Assessment of the Comparative Efficiency and Effectiveness of U.S. and Soviet Military Establishments,” August 17, 1972; and Memorandum from Col. Marshall to Baroody, “White House Request for National Net Assessment,” August, 17, 1972.

83. RNPL, NSC H-Files, NSS Memorandums 1969–1974, Box H-199, Folder NSSM 178 [1 of 2], “Memorandum from Marshall Kissinger, Secretary Laird’s Comments on Our Initial Net Assessment NSSM,” October 13, 1972, 2–4; RNPL, NSC Files, Box 825, Folder: Marshall, Andrew, Vol. I, “Memo from Marshall to Kissinger, Intelligence Inputs for Major Issues,” May 1, 1970.

84. RNPL, NSC H-Files, NSS Memorandums 1969–1974, Box H-199, Folder NSSM 178 [1 of 2], “A.W. Marshall Memorandum for the Record: Definition of the National Net Assessment Process,’ October 14, 1972.

85. Almost concurrently CIA director James Schlesinger established an apparently a short-lived analytical cell called Strategic Evaluation Center, tasked with conducting net assessments. Not much is known about this organization or about the analytical methodologies it employed.“Memorandum from National Intelligence Officer for Strategic Programs to Turner: What Role Should Intelligence Play in US-Soviet Net Assessments?” March 7, 1977, CIA CREST, https://www.cia.gov/library/readingroom/docs/CIA-RDP83M00171R001200160009-9.pdf.

86. RNPL NSC H-Files, NSS Memorandums 1969–1974, Box H-199, Folder NSSM 178 [1 of 2], “Memorandum from Marshall to Kissinger: Status of Net Assessment Studies,” January 9, 1973, 2; and “MeMo from Marshall & Odeen to Kissinger,” January 17, 1973, TAB G.

87. A.W. Marshall, “Nature and Objectives of a National Net Assessment Program”, attachment to “Memorandum from A.W. Marshall to Ad Hoc Net Assessment Group,” April 9, 1973, 1, 2. This version of the paper is in fact revised.

88. RNPL NSC H-Files, NSS Memorandums 1969–1974, Box H-199, Folder NSSM 178 [1 of 2], “Attachment A to “Memorandum from Weiss to Marshall,” April 19, 1973, 2–3.

89. HIA, SWP, Box 19, folder: Jan–April 1973, “Annex B to Memorandum from Seymour Weiss to Mr. A.W. Marshall: Jeffrey S. Milstein, Methodological Elements of Net Assessment in Priority of Importance,” April 19, 1973, 2–3; RNPL, NSC H-Files, NSS Memorandums 1969–1974, Box H-199, Folder NSSM 178 [1 of 2], “Memorandum from Thro to Marshall,” May 3, 1973.

90. RNPL, NSC H-Files, NSS Memorandums 1969–1974, Box H-199, Folder NSSM 178 [1 of 2], “Attachment to Memorandum from Ad Hoc NAG Chairman to NSCIC Chairman: Report on the Discussions of Methodology Appropriate for Use in Preparing Net Assessments,” May 18, 1973, 3–4, 6.

91. Mahnken and Roche, “What is Net Assessment,” 11–26; and Cohen, “Net Assessment”.

92. Krepinevich and Watts, The Last Warrior, 59–62; Augier, “Thinking about War and Peace”.

93. Rosen, “Net Assessment as an Analytical Concept,” 283–286.

94. A note of caution should be added on this point. Andrew W. Marshall is a figure that has commanded enormous respect and consideration within the US national security establishment, as well as within the broader international defense experts’ community and the security studies academic community. In the last years nonetheless, the scope of his contribution to national security and strategic thought and practice has been the object of occasional controversies and debates. The conclusions of this study should not be seen as belittling his role and contribution in the genesis of NA. When properly contextualized, NA cannot be considered his brainchild, as his contribution to the methodology itself was far from crucial. His role was nonetheless pivotal in meshing different approaches and paradigms into one overarching framework consistent with the requirement set by policymakers. See Lewis, “Yoda Has Left the Building”; Mahnken, “An Uncivil Treatment of a Civil Servant”; and Desch, “Don’t Worship at the Altar of Andrew Marshall”.

95. For such an instance, see Richelson, The Wizards of Langley, 42.

96. Omand, How Spies Think; Gartin, “The Future of Analysis,” 2.

97. Laipson, “Think Tanks,” 298.

98. Marrin, “Understanding and Improving Intelligence Analysis”; Mark Phythian, “Intelligence Analysis and Social Science Methods”.

99. Kozlowski, “Workforce Effectiveness,” 271–308.

Additional information

Notes on contributors

Niccolò Petrelli

Niccolò Petrelli is Adjunct Professor, Department of Political Science, Università Roma Tre, in Italy. He was previously postdoctoral fellow at Stanford’s CISAC, Lecturer at the University of Leeds, Eisenhower Defense Fellow at the NATO Defense College, and Research Fellow at the Military Center for Strategic Studies (Ce.Mi.S.S.) in Roma.