An empirical test of the performance of intelligence and the prospects for organizational reforms

ABSTRACT

The occurrence of strategic surprises has led to the development of organizational reforms intended to prevent future intelligence failures. The limited effectiveness of such reforms conforms to a perspective that holds intelligence to be beset by inherent difficulties, such as ambiguous information, enemy deception, and human psychology, which make recurring intelligence failures inevitable. If this perspective is correct, then all functions of intelligence should be distinctly unreliable regardless of implemented reforms. I test this perspective through a comparison of the performance of military capabilities and tactical intelligence capabilities, in terms of their influence on battlefield outcomes. While tactical intelligence is susceptible to the inherent difficulties common to all functions of intelligence, it is less susceptible to the bureaucratic difficulties that most reforms, aimed at preventing strategic intelligence failures, are intended to address. I find that relatively strong tactical intelligence capabilities significantly improve battlefield effectiveness. However, the improvement associated with tactical intelligence capabilities is also significantly more inconsistent when compared to the corresponding improvement associated with military capabilities. Even when evaluating a function of intelligence less susceptible to bureaucratic difficulties, intelligence is still distinctly inconsistent in performance, suggesting that organizational reforms have only a limited ability to prevent future intelligence failures.

Resumen

La ocurrencia de sorpresas estratégicas ha llevado al desarrollo de reformas organizativas destinadas a prevenir futuros fallos de inteligencia. La eficacia limitada de tales reformas se ajusta a una perspectiva que sostiene que la inteligencia se ve asediada por dificultades inherentes, como son la información ambigua, el engaño del enemigo y la psicología humana, que hacen que los errores de inteligencia sean inevitables. Si esta perspectiva es correcta, entonces todas las funciones de la inteligencia deben ser característicamente no confiables independientemente de la implementación de reformas. Pongo una prueba esta perspectiva mediante una comparación del rendimiento de capacidades materiales e la inteligencia operacional, en términos de su influencia en los resultados militares. Mientras que la inteligencia operacional es susceptible a las dificultades inherentes común a todas las funciones de la inteligencia, esta es menos susceptible a las dificultades burocráticas que la mayoría de las reformas, destinadas a prevenir fallas estratégicas, tratan de abordar. Encuentro que las capacidades de inteligencia operacional relativamente sólidas mejoran significativamente la eficacia militar. Sin embargo, la mejora asociada con

las capacidades de inteligencia operacional es también significativamente más inconsistente cuando se compara con la mejora correspondiente asociada a las capacidades materiales.

Incluso cuando se evalúa una función de inteligencia menos susceptible a las dificultades burocráticas, la inteligencia sigue siendo característicamente inconsistente en el rendimiento, lo que sugiere que las reformas organizacionales tienen solo una capacidad limitada para prevenir los fallos futuros de inteligencia estratégica.

KEYWORDS:

• International relations
• intelligence failures
• intelligence reforms
• strategic studies

Palabras clave:

• inteligencia
• ataques sorpresa
• reformas de inteligencia
• seguridad
• fallos de inteligencia

Notes

^1. The concept of strategic intelligence is used here to indicate the gathering of information relevant to long-range trend analysis, policy planning, and attack warning (Office of the Director of National Intelligence, 2009, p. 7).

^2. The term tactical intelligence can also indicate more local unit level, intelligence (Handel, 1989, pp. 361–454). As it is used in this paper, the term is meant to indicate theater of operations-level intelligence with the information being acquired having a useable “half-life” measured in days (Ferris, 2005, p. 283–7).

^3. The intention of the concept of surprise as used in the hypothesis tests is that of a misperception of enemy intentions associated with a major military operation. This concept of surprise is meant to convey a failure to appreciate the intentions of a major enemy offensive as opposed to a type of strategic surprise in which there is a failure to anticipate the onset of a conflict (Handel, 1989, pp. 229–49).

^4. The number of troops deployed does not include support troops, except those that are organic to a combat division or independent brigade.

^5. As with the number of troops deployed, only combat ship tonnage is totaled. Support ships, such as supply vessels and mine sweepers, are not incorporated when calculating combat ship tonnage.

^6. As naval battles are generally not contesting territory, outcome in naval battles is determined by whether either combatant achieved their objective. For example, the British are coded as having won the battle of Jutland, as the Royal Navy compelled the German fleet to return to port achieving their objective of maintaining the blockade of the Central Powers.

^7. All of the models presented in Tables 3 and 4 did not violate the proportional odds assumption as indicated by an insignificant χ² test result.

^8. For both Graphs 1 and 2, predicted probabilities are generated for the minimum, tenth, thirtieth, median, seventieth, ninetieth, and maximum values of log war duration.

^9. The distances are based upon air distances between the relevant cities. Information was obtained from http://www.distancesfrom.com.

^10. With respect to naval battles, the ratio of the combat tonnage of the fleets in the battle is used as a substitute for the ratio of combat troops deployed in calculating the value of troop ratio.

^11. If there was a coalition of states fighting, the values used to determine per-soldier spending is based upon which member of a coalition composes the greatest part of the forces fighting in the battle.

^12. The corresponding changes in predicted probability for military capabilities are battle win (Δ = .550 (95% CI: .433 to .668)) and loss (Δ = –.544 (95% CI: –.661 to –.426)).

^13. Ellis and Cox (1993); Bennett (2005); Cruttwell (2007); Ferris (1988); Greiss (2011).

^14. Gilbert (1994); Ellis and Cox (1993); Cruttwell (2007); Greiss (2011); Schindler (2000); Kahn (1980), p. 619.

^15. Ellis and Cox (1993); Cruttwell (2007); Ferris (1988); Greiss (2011).

^16. Kukiel (1929), pp. 48–65; Davies (1972); Clodfelter (2002), pp. 370–1.

^17. Edwards (2008); Ellis (1993).

^18. Coox (1973); Coox (1977); Clodfelter (2002), p. 395.

^19. Drea (1981); Clodfelter (2002), p. 395.

^20. Jackson (1967); Mitcham, (1983a, 1983b); Bennett (1990); Ellis (1993); Botjer (1996); Coates (2000); Darman (2009); Zaloga (2009).

^21. Ellis (1993); Dildy (2007); Greiss (2011).

^22. Warner (1990); Ellis (1993); Greiss (2011).

^23. Zaloga (2002); Ellis (1993); Greiss (2011).

^24. Esposito (1959); Ellis (1993).

^25. Esposito, (1959); Bateson (1968); Toland (1970); Alexander (2005); Marston (2005); O’Hara (2007).

^26. Erickson (1975a, 1975b)

^27. Bekes, Byrne, and Rainer (2002); Sebestyen (2006); Congdon, Kiraly, and Nagy (2006).

^28. History Division (1992a); Clodfelter (2002), pp. 639–41; Shrivastava (2013), ch. 2, 4.

^29. History Division (1992b); Clodfelter (2002), pp. 642–46; Shrivastava (2013), ch. 2 and 4.

^30. Dupuy (1978), pp. 3–121; Pollack (2004), ch. 1–3, 6; Gazit and Herzog (2005), ch. 1–4; Tal (2014), ch. 8–10.

^31. Dupuy (1978), pp. 129–215; Sheffy (1990), pp. 7–56; Pollack (2004), ch. 1; Gazit and Herzog (2005), pp. 109–40.

^32. Dupuy (1978), pp. 221–333; Pollack (2004), ch. 1–3, 6; Gazit and Herzog (2005), pp. 145–92.

^33. Dupuy (1978), pp. 387–584; Pollack (2004), ch. 1, 6; Gazit and Herzog (2005), pp. 231–306.

^34. Grant (2002), pp. 58–62; Pollack (2004), ch. 6; Gazit and Herzog (2005), pp. 347–76.; Clarke et al. (1983)

^35. Aid (1999); Aid (2000); Korean Institute of Military History, (2001a, 2001b, 2001c).

^36. Maxwell (1970); Calvin (1984); Palit (1991), p. 639.

^37. Ngo (1980); Lung (1981); Clodfelter (1995).

^38. Cordesman and Wagner (1990); Clodfelter (2002), pp. 627–9.

^39. Hopple (1984); Anderson (2002).

^40. Atkinson (1993); Finlan (2003).

^41. Kahn (1996), ch. 10–11; Keegan (2003), ch. 4; Cruttwell (2007), pp. 115–6, 568; Larsen (2014), pp. 282–302.

^42. Ferris (1988); Popplewell (1990), pp. 139–72; Handel (1989), ch. 8; Sheffy (1998); Cruttwell (2007), pp. 356–7; Larsen (2014).

^43. Cruttwell (2007), pp. 45–6, 80; Kahn (1980); Schindler (2000).

^44. Davies (1972), pp. 89–122; Watt (1979), p. 143; Desch (2008), p. 92.

^45. Lewin (1981), pp. 501–12; Ambrose (1981); Rohwer (1999); Ferris (2005), ch. 5.

₄₆ Rohwer (1999); Drea and Richard (1999), pp. 62–83; Keegan (2003), ch. 5–6.

^47. Thomas (1987); Mulligan (1987); Glantz (1990).

^48. Aid (1999); Aid (2000).

^49. Dupuy (1978), pp. 143–50; Cohen (1998), pp. 100–40; Pollack (2004), pp. 30–1, 43–4.

^50. Dupuy (1978), pp. 231–44; Pollack (2004), pp. 62–3, 462.

^51. Shlaim (1976); Dupuy (1978), pp. 391, 406–7; Handel (1989), pp. 250–1, 260; Shalev (2010), pp. 228–9; Buckwalter (2012), p. 103.

^52. Clarke, (1983), ch. 3–4; Grant (2002); Pollack (2004), pp. 542–3.

^53. Maxwell (1970), pp. 310–1, 370–2, 392–401, 408; Calvin (1984); Palit (1991), pp. 204–5, 230–1, 289, 411–25.

^54. Wirtz (1991), p. 204; Krause (1996), pp. 305–6; Clodfelter (1995), pp. 52–3; Warner (2010).

^55. Hopple (1984); Lebow (1985); Keegan (2003), pp. 299–313.

^56. Atkinson (1993), pp. 176, 233–5, 334–46, 396; Ferris (2005), pp. 308–27.

Additional information

Notes on contributors

Kristopher Grady

Dr. Kristopher Grady is an Assistant Professor in the Department of Political Science at the University of Louisville. Dr. Grady has taught in the People’s Republic of China and served as a participant for the Department of Strategic Wargaming with respect to security issues in Northeast Asia. He earned his Ph.D. in Political Science at Michigan State University. Previous to this, Dr. Grady worked as a lobbyist and on election campaigns in Washington, DC.