‘The Great Fiasco’ of the 1948 presidential election polls: status recognition and norms conflict in social science

SUMMARY

All three ‘scientific’ pollsters (Crossley, Gallup and Roper) wrongly predicted incumbent President Harry Truman’s defeat in the 1948 presidential election, and thus faced a potentially serious legitimacy crisis. This ‘fiasco’ occurred at a most inopportune time. Social science was embroiled in a policy debate taking place in the halls of Congress. It was fighting a losing battle to be included, along with the natural sciences, in the National Science Foundation, for which legislation was being drafted. Faced with the failure of the polls, the Social Science Research Council (SSRC) intervened quickly to prevent social science’s adversaries from using this event to degrade further its status. After all, many social scientists considered the sample survey as the paramount tool of social research, and sampling as one of social science’s greatest innovation. Concurrently, there was an ongoing conflict among polling practitioners themselves—between advocates of probability sampling and users of quotas, like the pollsters. The SSRC committee appointed to evaluate the polling debacle managed to keep this contentious issue of sampling from becoming the centre of attention. Given the inauspicious environment in which this event happened, the SSRC did not wish to advertise the fact that the house of social science was in turmoil.

Acknowledgements

I would like to thank Professor W. Joseph Campbell (American University), Raul Gilbert, Professor J. Michael Hogan (Pennsylvania State University), Rita Moran and Josh Senyak for reviewing the paper at various stages of its development, and for their helpful comments and support. I wish also to express my gratitude to the anonymous reviewers of the Annals of Science for their valuable feedback and to the journal’s editors for their guidance.

ORCID

Dominic Lusinchi http://orcid.org/0000-0002-6956-2349

Notes

1 Amy Fried, Pathways to Polling: Crisis, Cooperation and the Making of Public Opinion Profession (New York: Routledge, 2012), p. 108.

2 Jean M. Converse, Survey Research in the United States: Roots and Emergence 1890-1960 (New Brunswick and London: Transaction Publishers, 2009 [1987]), pp. 374–76, 392–94; Fried, Pathways (note 1), passim; Sarah E. Igo, The Averaged American: Survey, Citizens, and the Making of a Mass Public (Cambridge, MA: Harvard University Press, 2007), pp. 153–54, 186–87, 189.

3 Igo, The Averaged American (note 2), p. 186.

4 Erving Goffman, The Presentation of Self in Everyday Life (New York: Anchor Books, 1959), pp. 208–37.

5 U.S. House of Representatives, Hearings Before a Subcommittee of the Committee on Interstate and Foreign Commerce on H.R. 6448 (Washington, DC: Government Printing Office, 1946), p. 13.

6 Mark Solovey, ‘Riding natural scientists' coattails onto the endless frontier: The SSRC and the quest for scientific legitimacy’, Journal of the History of the Behavioral Sciences, 40 (2004), 393–422 (p. 397).

7 Solovey, ‘Riding’ (note 6), p. 394.

8 Edward T. Folliard, ‘Truman's Defeat By Wide Margin Seen; 51-Million Vote Expected’, Washington Post, 2 November 1948, pp. 1, 5 (p. 1).

9 On the press’s general view of Dewey’s chances, Fried (Pathways (note 1), p. 98) writes: ‘Confidence in Dewey’s invulnerability started early. Even during the summer months, some election analysts were sure that Dewey would win. […] Through the fall months, journalists repeated their certainty of a Dewey victory.’

10 ‘The Talk of the Town: Notes and Comment’, New Yorker, 12 November 1948, p. 21.

11 Stuart Chase, ‘Are the Polls Finished?’, Nation, 4 December 1948, pp. 626–29 (p. 626).

12 A. J. Liebling, ‘The Wayward Press’, New Yorker, 20 November 1948, pp. 98–103 (p. 100). The same sentiment was expressed by members of the ‘attentive public’. The authors of the book, The People Know Best, would write: ‘While the polls are looking for the source of their error, we would like to suggest that part of it will be found in their faith that what they are doing is science.’ Morris L. Ernst and David Loth, The People Know Best: The Ballot vs. The Polls (Washington, DC: Public Affairs Press, 1949), p. 143.

13 ‘The Great Fiasco’, Time, 15 November 1948, p. 66.

14 George H. Gallup and Saul Forbes Rae, The Pulse of Democracy (New York: Simon & Schuster, 1940), p. 13–14.

15 Associated Press, ‘Old Digest Ghost in Stitches; Science, Ha! Laughs, Ex-Editor’, Washington Post, November 4 1948, p. 13. A straw poll is the predecessor of the modern poll, which came into existence in 1935 with Roper, Gallup, and Crossley. Straw polls were conducted mostly by newspapers, and during the nineteenth century, they were used more often than not as a partisan weapon. The Literary Digest began its polling activities in 1916. It differed from its predecessors in that it compiled a list of potential respondents, it conducted nationwide surveys, not only during election campaigns, but also on issues, such as Prohibition, and it was distinctly non-partisan. Its last poll was in 1936.

16 The Literary Digest poll in 1936 missed the target by nearly 20 points (see Dominic Lusinchi ‘“President” Landon and the 1936 Literary Digest Poll: Were Automobile and Telephone Owners to Blame?’, Social Science History, 36 (2012), 23–54). Before that the Digest poll had always forecast the winner of a presidential election.

17 Norman C. Meier and Harold W. Saunders, The Polls and Public Opinion: Iowa Conference on Attitude and Opinion Research (New York: Henry Holt & Company, 1949), p. 217.

18 Rensis Likert, ‘Opinion Studies and Government Policy’, Proceedings of the American Philosophical Society, 92 (1948), 341–50 (p. 341). For the same evaluation some years earlier, see Albert B. Blankenship, ‘The Case for and against the Public Opinion Poll’, The Journal of Marketing, 5 (1940), 110–13 (p. 110) (‘the public opinion poll has developed into one of the most phenomenal yardsticks ever conceived.’), and Gordon W. Allport, ‘Review of A guide to public opinion polls’, The Journal of Abnormal and Social Psychology, 40 (1945), 113–14 (p. 114) (‘Polling represents probably the greatest advance made by social science during the present century.’)

19 U.S. House of Representatives, Hearings before the Committee to Investigate Campaign Expenditures. Part 12: American Institute of Public Opinion (Gallup Poll); Dr. George Gallup, Witness. 78th Congress, 2nd Session on H. Res. 551, Dec. 28, 1944 (Washington, DC: Government Printing Office, 1945), p. 1266.

20 U.S. House of Representatives, Gallup Poll (note 19), p. 1266.

21 ‘3 Polling Groups To Release Data: Major Organizations to Make Material Available as Soon as Poll Analyses Are Done’, New York Times, 6 November 1948, p. 3.

22 Associated Press, ‘McKellar to Ask Digest Poll Inquiry’, Washington Post, 11 November1936, p. X6. For a detailed history of congressional dealings with polls in the 1930s and 1940s, see Fried, Pathways, pp. 68–79.

23 Robert Cobb Myers, ‘Vital Part of the Poll Question: The election forecasts have had all the attention. But what about the polls on the issues of our time?’, New York Times, 2 January 1949, pp. 8, 31 (p. 8).

24 Will Lissner, ‘Poll-Takers Called On to Disclose Technical Data for an Evaluation’, New York Times, 5 November 1948, pp. 1, 5.

25 New York Times, ‘3 Polling Groups’ (note 21), p. 3.

26 Converse, Survey Research (note 2), p. 393. Frederick Mosteller, The Pleasures of Statistics: The Autobiography of Frederick Mosteller (New York: Springer, 2010), p. 7.

27 Converse, Survey Research (note 2), p. 396.

28 Fried, Pathways (note 1), p. 2.

29 ‘In any system of ranked groups, participants take it as given that members of the highest group have the right to define the way things really are.’ Howard S. Becker, Outsiders: Studies in the Sociology of Deviance (New York: Free Press, 1966), p. 241.

30 Elbridge Sibley and Kenton W. Worcester, The Social Science Research Council, 1923-1998 (New York: Social Science Research Council, 2001), p. 15.

31 Solovey, ‘Riding’ (note 6), p. 404.

32 Solovey, ‘Riding’ (note 6), p. 398.

33 Sibley and Worcester, The Social Science Research Council (note 30), pp. 47, 6.

34 Samuel A. Stouffer and others, The American soldier: adjustment during army life (Princeton, NJ: Princeton University Press, 1949).

35 About this committee, see Converse, Survey Research (note 2), pp. 229–30, 336, 371 and Fried, Pathways (note 1), pp. 65–66.

36 Mosteller, Pleasures of Statistics (note 26), p. 6.

37 Frederick F. Stephan and Philip J. McCarthy, Sampling Opinions: An Analysis of Survey Procedure (New York: John Wiley & Sons, Inc., 1958), p. v.

38 My description of the Committee’s work relies heavily on chapter 1 of Mosteller’s autobiography: ‘Why Did Dewey Beat Truman in the Pre-election Polls of 1948?’ (Mosteller, Pleasures of Statistics (note 26), pp. 5–17).

39 ‘Science to Investigate Election Poll Workings’, New York Times, 10 November 1948, p. 24.

40 ‘7 Named For Study of Election Polls’, New York Times, 11 November 1948, p. 28.

41 Mosteller, Pleasures of Statistics (note 26), p. 7.

42 ‘Poll Group Augmented: 2 Professors Added to Election Inquiry Committee’, New York Times, 15 November 1948, p. 17.

43 Mosteller, Pleasures of Statistics (note 26), p. 9.

44 Frederick Mosteller and others, The Pre-Election Polls of 1948: the report to the Committee on Analysis of Pre-election Polls and Forecasts (New York: Social Science Research Council, Bulletin 60, 1949).

45 The analysis that follows owes much to Solovey, ‘Riding’ (note 6), and chapter 2 of Thomas F. Gieryn, Cultural Boundaries of Science: Credibility on the Line (Chicago: University of Chicago Press, 1999).

46 I do not mean to convey the impression that all natural scientists and all politicians that intervened in the debate were against social science—far from it. What I am presenting here is the point of view that prevailed.

47 Solovey, ‘Riding’ (note 6), p. 403.

48 Solovey, ‘Riding’ (note 6), p. 404.

49 Solovey, ‘Riding’ (note 6), p. 408.

50 U.S. House of Representatives, Hearings on H.R. 6448 (note 5), p. 11.

51 U.S. House of Representatives, Hearings on H.R. 6448 (note 5), p. 13.

52 Bowman said: ‘there are about 137,000,000 social scientists in the United States’ (U.S. House of Representatives, Hearings on H.R. 6448 (note 5) p. 11), roughly the population of the country in 1945.

53 U.S. House of Representatives, Hearings on H.R. 6448 (note 5), p. 20 and Solovey, ‘Riding’ (note 6), p. 409.

54 Gieryn, Cultural Boundaries (note 45), p. 76 and Solovey, ‘Riding’ (note 6), p. 410.

55 M. Brewster Smith, ‘The American Soldier and Its Critics: What Survives the Attack on Positivism?’ Social Psychology Quarterly, 47 (1984), p. 192–98 (p. 194).

56 Solovey, ‘Riding’ (note 6), p. 411.

57 Robert Redfield, ‘The Art of Social Science’, American Journal of Sociology, 54 (1948), p. 181–90 (p. 185). One observer deplored the fact that the polls fitted well within this mostly quantitative vision of social science. Disputing the pollsters’ claim that theirs was a ‘microscopically accurate’ instrument, and rejecting the polls as ‘push-button “techniques”’, he suggested that what was needed was ‘a little less absentee-engineering and a little more street-corner sociology’ (Werner J. Cahnman, ‘Measuring Public Opinion: Instrument Considered to Be Far From Accurate, as Described’, New York Times, 20 November 1948, p. 12.). This was an obvious reference to a classic work in participant observation, William Foote. Whyte, Street Corner Society: The Social Structure of an Italian Slum (Chicago: University of Chicago Press, 1943).

58 Solovey states that Mitchell, Ogburn, Nourse, Gaus and Yerkes were ‘the Council’s selection of witnesses’ (Solovey, ‘Riding’ (note 6), p. 404).

59 p. 741, Mitchell; p. 745, Gaus; p. 752–53, Yerkes. All the page numbers and/or names listed in the footnotes for the next three paragraphs refer to the following document: U.S. Senate, Hearings on Science Legislation—Part 4: Oct. 29-31 & Nov. 1, 1945 (Washington, DC: U.S. Government Printing Office, 1946), pp. 737–86.

60 p. 739, Mitchell; p. 773, Ogburn; p. 778, Cooper.

61 p. 741, Mitchell; p. 752, Yerkes.

62 p. 741, Mitchell; p. 748, Gaus; p. 758, Nourse; p. 771, Ogburn.

63 p. 743, Mitchell; p. 755, Yerkes; pp. 757, 758, Nourse.

64 pp. 741, 768.

65 pp. 765, 766.

66 p. 766.

67 pp. 753, 757, 741.

68 Talcott Parsons, ‘The Science Legislation and the Role of the Social Sciences’, American Sociological Review, 11 (1946), 653–66 (p. 665).

69 Talcott Parsons, ‘Science Legislation and the Social Sciences’, Political Science Quarterly, 62 (1947), 241–49 (p. 243).

70 Quinn McNemar, ‘Opinion-Attitude Methodology’, Psychological Bulletin, 43 (1946), 289–374 (pp. 299–328).

71 Converse, Survey Research (note 2), p. 126.

72 Obenauer had worked for many years at the Bureau of Labor Statistics, and conducted extensive research on the working and living conditions of male and female blue-collar workers (e.g. laundry workers, coal miners, etc.). At the time of her letter to the Post, she was joint chair of a non-profit organization called Home Owners’ Protective Enterprise in Washington, DC.

73 Marie L. Obenauer, ‘The Gallup Poll’s Sampling Technique’, Washington Post, 15 November 1937, p. 6. Deming would say the same a decade later: ‘Even though a particular judgment procedure has been used over a period of years and has apparently always given good results, if no good reasons can be assigned for such performance, it does not and never has satisfied the requirements of a probability-sample, in the sense that there never has been an objective prediction of what to expect in a future survey.’ W. Edwards Deming, ‘Some Criteria for Judging the Quality of Surveys’, The Journal of Marketing, 12 (1947), 145–57 (p. 154).

74 Eric F. Goldman, ‘Poll on the Polls’, The Public Opinion Quarterly, 8 (1944), 461–67 (p. 467).

75 Jerzy Neyman, ‘On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection’, Journal of the Royal Statistical Society, 97 (1934), 558–625.

76 Jerzy Neyman, Lectures and conferences on mathematical statistics and probability, 2nd edn rev. enl. (Washington, DC: U.S. Dept. of Agriculture, 1952 [1937]), p. 105.

77 Jerome Cornfield, ‘On Certain Biases in Samples of Human Populations’, Journal of the American Statistical Association, 37 (1942), 63–68 (p. 65).

78 U.S. House of Representatives, Gallup Poll (note 19), pp. 1295, 1298, 1295.

79 U.S. House of Representatives, Gallup Poll (note 19), pp. 1295–96.

80 On the issue of promoting the scientific image of polls see Dominic Lusinchi ‘The Rhetorical Use of Random Sampling: Crafting and Communicating the Public Image of Polls as a Science (1935-1948)’, Journal of the History of the Behavioral Sciences, 53 (2017), 113–32.

81 U.S. House of Representatives, Gallup Poll (note 19), p. 1295.

82 U.S. House of Representatives, Gallup Poll (note 19), pp. 1293, 1295, 1298.

83 Louis H. Bean, How to Predict Elections (New York: Alfred A. Knopf, 1948).

84 Philip M. Hauser and Morris H. Hansen, ‘On Sampling in Market Surveys’, The Journal of Marketing, 9 (1944), 26–31 (p. 27).

85 Hauser and Hansen, ‘On Sampling’ (note 84), pp. 28, 29.

86 Morris H. Hansen and Philip M. Hauser, ‘Area Sampling-Some Principles of Sample Design’, The Public Opinion Quarterly, 9 (1945), 183–93 (p. 185). In quota sampling the interviewers are given specific demographic quotas to fill, i.e. so many males, so many females, so many young and so many old, etc. proportional to their representation in the population 21 years and older based on Census data.

87 Hansen and Hauser, ‘Area Sampling’ (note 86), p. 192.

88 Hansen and Hauser, ‘Area Sampling’ (note 86), p. 186.

89 Norman C. Meier and Cletus J. Burke, ‘Laboratory Tests of Sampling Techniques’, Public Opinion Quarterly, 11 (1947–1948), 586–93 (p. 586).

90 National Opinion Research Center, ‘Central City Conference on Public Opinion Research’, Proceedings of the Central City Conference on Public Opinion Research (Denver, CO: National Opinion Research Center, 1946), p. 58.

91 National Opinion Research Center, ‘Central City Conference’ (note 90), p. 59.

92 Meier and Burke, ‘Laboratory Tests’ (note 89), p. 593.

93 Meier and Saunders, The Polls (note 17), p. 199.

94 Seymour Banks, Cletus J. Burke and Norman C. Meier, ‘Laboratory Tests of Sampling Techniques: Comment and Rejoinders’, The Public Opinion Quarterly, 12 (1948), 316–24 (p. 316).

95 Rensis Likert, ‘Why Opinion Polls Were So Wrong—Outdated Methods Used, Survey Expert Says’, U.S. News & World Report, 12 November 1948, pp. 24–25 (p. 24).

96 Likert, ‘Why Opinion Polls Were So Wrong’ (note 95), pp. 24, 25.

97 Rensis Likert, ‘Public Opinion Polls—Why did they fail? A leading authority assays their weaknesses and suggests some tested new techniques that would improve their accuracy’, Scientific American, December 1948, pp. 7–11 (p. 7).

98 Likert, ‘Polls—Why did they fail?’ (note 97), p. 9. At the time of these articles, Likert was the director of the Survey Research Center (SRC) at the University of Michigan. Of the SRC, Converse states that it was ‘the single most important academic practitioner of area probability sampling, second in influence only to the sampling statisticians in government (Hansen, Hurwitz, Madow, Deming, Snedecor, Cochran, and others).’ (Converse, Survey Research (note 2), pp. 370–71.)

99 Alfred N. Watson, ‘Measuring the New Market’, Printer’s Ink, 2 June 1944, 17–18, 95–96 (pp. 18, 95).

100 Alfred Politz, ‘The 1948 election forecast—a useful disaster’, Printers’ Ink, 12 November 1948, pp. 36c-36f (pp. 36c, 36c, 36d). Politiz was a member of the Advisory Subcommittee to the Study of Sampling of the SSRC’s Committee on Measurement mentioned previously (see Stephan and McCarthy, Sampling Opinions (note 37), p. v).

101 Deming, ‘Some Criteria’ (note 73), p. 147.

102 Politz, ‘Useful disaster’ (note 100), p. 36d.

103 Converse, Survey Research (note 2), p. 374.

104 Meier and Saunders, The Polls (note 17), p. 161.

105 Meier and Saunders, The Polls (note 17), pp. 221, 223.

106 George H. Gallup, ‘Should We Set Up Standards for Poll Critics?’, International Journal of Opinion and Attitude Research, 3 (1949), 348–54 (p. 348).

107 George H. Gallup, ‘Letter to The Editors’, Scientific American, February 1949, 2–3; Robert T. Bower, ‘Proceedings of the American Association for Public Opinion Research at the Fourth Annual’, Public Opinion Quarterly, 13 (1949), 737–808 (p. 765); Gallup, ‘Standards for Poll Critics’ (note 106).

108 Converse, Survey Research (note 2), p. 156. It is interesting to note that Gallup’s seven-point error in 1936 (much larger than in 1948) was considered ‘astounding accuracy’.

109 Rensis Likert, ‘Democracy in Agriculture—Why and How?’, in Farmers in a Changing World—Yearbook of Agriculture, 1940, ed. by U.S. Department of Agriculture (Washington, DC: U.S. Government Printing Office, 1940), pp. 994–1002 (p. 1000).

110 Becker, Outsiders (note 29), p. 1.

111 Aside from Becker, the interpretative framework I adopt in this section owes much to Joseph R. Gusfield, Symbolic Crusade: Status Politics and the American Temperance Movement (Urbana, IL: University of Illinois Press,1963); Joseph R. Gusfield, ‘Moral Passage: The Symbolic Process in Public Designations of Deviance’, Social Problems, 15 (1967), 175–88; and Joseph R. Gusfield, ‘On Legislating Morals: The Symbolic Process of Designating Deviance’, California Law Review, 56 (1968), 54–73.

112 Adolph Jensen, Konrad Saenger, and Arthur L. Bowley, ‘Rapport de M. Jensen sur l’Application de la méthode représentative’, Bulletin de l’Institut International de Statistique, 22 (1926), 58–61 (p. 59).

113 Adolph Jensen, ‘Report on the representative method in statistics’, Bulletin de l’Institut International de Statistique, 22 (1926), 359–78 (p. 376).

114 Joseph W. Duncan and William C. Shelton, ‘U.S. Government Contributions to Probability Sampling and Statistical Analysis’, Statistical Science, 7 (1992), 320–38 (p. 324). This article sketches the history of the adoption of probability sampling by the federal government, and the contributions to sampling theory made by individuals in federal agencies.

115 Joseph R. Hochstim and Dilman M. K. Smith, ‘Area Sampling or Quota Control?—Three Sampling Experiments’, The Public Opinion Quarterly, 12 (1948), 73–80 (p. 73). Opinion Research Corporation was the market research arm of the Gallup organization.

116 The quota samplers accepted the idea of probability sampling in its simple random sampling (SRS) version, i.e. only when a sampling frame, a list of all the elements of the population under study (e.g. all subscribers to a magazine), was available. See Archibald M. Crossley, ‘Theory and Application of Representative Sampling as Applied to Marketing’, The Journal of Marketing, 5 (1941), 456–461 (p. 458). In SRS, every member of the defined population has an equal probability of being selected into the sample. Area sampling was meant to overcome the problem when a list did not exist.

117 Banks, Burke and Meier, ‘Comment and Rejoinders’ (note 94), p. 320.

118 It was also a tacit admission on the part of politicians that they were not fully qualified to assess the scientific worth of polls. This was an important precedent that made the SSRC’s intervention in 1948 all the more justified.

119 Gusfield, ‘Moral Passage’ (note 111), p. 179. See U.S. House of Representatives, Gallup Poll (note 19), p. 1295.

120 Gusfield, Symbolic Crusade (note 111), pp. 68, 70, 111.

121 A testimony to the ascendancy of probability sampling is Roper’s attempt, in 1949, to implement it, unsuccessfully, in some of his surveys. For a description of this experiment see Igo The Averaged American (note 2), pp. 131–33.

122 Fried, Pathways (note 1), p. 126.

123 Fried, Pathways (note 1), p. 126.

124 Harold F. Gosnell, ‘Review: Report of the Social Science Research Council Committee on Analysis of Pre-Election Polls and Forecasts’, The Public Opinion Quarterly, 13 (1949), 134–36 (p. 134).

125 ‘Humiliation’ is how Arthur N. Holcombe, ‘Review of The Pollsters’, American Political Science Review, 43 (1949), 823–24 (p. 823), and Angus Campbell, ‘The Pre-Election Polls of 1948’, International Journal of Opinion and Attitude Research, 4 (1950), 27–36 (p. 27) described the 1948 polls’ failure.

126 ‘Dr. Gallup Explains Why Poll Fell Down: Cause Believed Failure to Take Into Account Last-Minute Shifts’, Los Angeles Times, 24 November 1948, p. 8.

127 United Press, ‘Gallup Adviser Says It Ain’t So’, Washington Post, 4 January 1949, p. 12.

128 George Gallup and others, ‘Here are the opinions of researchers and marketers’, Printers’ Ink, 12 November 1948, pp. 37–39 (p. 38).

129 ‘Poll Errors Laid to Poor Judgment’, New York Times, 27 December 1948, pp. 23, 34 (p. 34).

130 George H. Gallup, ‘Study of Poll Forecasts Impartial Job’, Washington Post, 27 December 1948, p. 7.

131 ‘Report on the Holes in the Polls’, Business Week, 8 January 1949, pp. 56–58 (p. 57).

132 Business Week, ‘Holes in the Polls’ (note 131), pp. 56–58.

133 Gresham M. Sykes and David Matza, ‘Techniques of Neutralization: A Theory of Delinquency’, American Sociological Review, 22 (1957), 664–70.

134 ‘Appeal to higher loyalties’, as Sykes and Matza call it: the actors see themselves ‘as caught up in a dilemma that must be resolved, unfortunately, at the cost of violating the law’ (Sykes and Matza, ‘Techniques’ (note 133), p. 669).

135 Wilson who was Director of Research at CBS also encouraged his fellow commercial researchers ‘not brush off the criticisms of critics who have never met a payroll’ (Bower, Proceedings (note 107), p. 738).

136 National Opinion Research Center, ‘Central City Conference’ (note 90), p. 61.

137 Meier and Burke, ‘Laboratory Tests’ (note 89), p. 587.

138 Hochstim and Smith, ‘Area Sampling’ (note 115), p. 74.

139 Frederick F. Stephan, ‘History of the Uses of Modern Sampling Procedures’, Journal of the American Statistical Association, 43 (1948), 12–39 (p. 35). The Committee on Polls noted the same thing: the quota methods ‘were used again in 1948, in part because they were less costly’. Committee on Analysis of Pre-Election Polls and Forecasts of the Social Science Research Council ‘Report on the Analysis of Pre-Election Polls and Forecasts’, The Public Opinion Quarterly, 12 (1948-1949), 599–622 (p. 611).

140 Watson, ‘Measuring’ (note 99), p. 18.

141 Hauser and Hansen, ‘On Sampling’ (note 84), p. 31.

142 W. Edwards Deming, ‘On Training in Sampling’, Journal of the American Statistical Association, 40 (1945), 307–16 (pp. 309–10).

143 Deming, ‘Some Criteria’ (note 73), p. 157.

144 Hansen and Hauser, ‘Area Sampling’ (note 86), pp. 191–92.

145 Louis H. Bean, ‘Review of The Pre-Election Polls of 1948 by Frederick Mosteller et al.’, Journal of the American Statistical Association, 45 (1950), 461–64 (p. 461).

146 Gieryn, Cultural Boundaries (note 45), pp. 98–100.

147 Solovey, ‘Riding’ (note 6), p. 414.

148 ‘Science’ here means, of course, the natural sciences; the social sciences were explicitly excluded. Vannevar Bush was the director of the Office of Scientific Research and Development. The role of the agency was to coordinate scientific research to support the military during WWII. His report, which came out in July of 1945, described how the federal government could continue its support of the sciences during peacetime. It is accessible at www.nsf.gov/about/history/vbush1945.htm [accessed March 2017].

149 Solovey, ‘Riding’ (note 6), pp. 414–15.

150 Solovey, ‘Riding’ (note 6), p. 410.

151 Claude E. Robinson, ‘Review of “How to Predict Elections” and of “A Guide to Public Opinion Polls”’, Annals of the American Academy of Political and Social Science, 261 (1949), 194–96 (p. 195).

152 New York Times, ‘Science to Investigate’ (note 39), p. 24.

153 p. 600. Unless otherwise indicated, all the page numbers listed in the footnotes for each of the next four paragraphs, in the order they are cited, refer to pages in the previously mentioned Committee on Analysis of Pre-Election Polls, ‘Report’ (note 139).

154 p. 601.

155 ‘Roper Still Seeks Answer on Polls: Does Not Know Why They Failed but Thinks They Were Not Used Correctly’, New York Times, 8 November 1948, p. 13.

156 pp. 602, 622, 602, 602, 602, 622, 622, 602, 602, respectively.

157 U.S. House of Representatives, Gallup Poll (note 19), p. 1294.

158 pp. 608, 611, 608, 609, respectively.

159 pp. 602, 609, 620. The quota samplers brushed aside the fact that this ‘conclusion’ was heavily qualified: it was ‘tentative’, the Committee said; based on ‘evidence now available’, implying that things could change given further evidence; noting the ‘complexity of the problem’ (p. 620).

160 Wallace had declared: ‘The sweeping assertion that social science is not science at all is nonsense.’ U.S. Senate, Hearings on Science Legislation — Part 1: Oct. 8-12, 1945 (Washington, DC: U.S. Government Printing Office, 1946), p. 82.

161 U.S. Senate, Hearings on Science Legislation — Part 1 (note 160), p. 80.

162 U.S. Senate, Hearings on Science Legislation — Part 4 (note 59), p. 767. Parsons said as much in 1946 (Parsons, ‘Role of the Social Sciences’ (note 68), p. 664). He described sampling techniques as ‘refined’. He reiterated this assessment a year later (Parsons, ‘Science Legislation’ (note 69), p. 245).

163 Goffman, The Presentation (note 4), p. 211.

164 Stouffer supported the pollsters’ version of the events. He stated: ‘I don’t think there is any doubt at all—there is no doubt in my mind—that there was a very substantial shift in the last two weeks of the campaign. (…) … it is possible that Gallup and Crossley—not Roper, he was a little too far off this time—but Gallup and Crossley were almost on the beam as of two weeks before the election’ (Meier and Saunders, The Polls (note 17), p. 211).

165 Becker, Outsiders (note 29), p. 147.

166 See for example: Frederick F. Stephan, ‘Representative Sampling in Large-Scale Surveys’, Journal of the American Statistical Association, 34 (1939), 343–52; Frederick F. Stephan, W. Edwards Deming and Morris H. Hansen, ‘The Sampling Procedure of the 1940 Population Census’, Journal of the American Statistical Association, 35 (1940), 615–30; and Frederick F. Stephan, ‘History of the Uses’ (note 139).

167 Frederick F. Stephan, ‘Sampling in Studies of Opinions, Attitudes, and Consumer Wants’, Proceedings of the American Philosophical Society, 92 (1948), 387–98 (p. 390). The research, upon which this article was based, was done under the auspices of the SSRC and the National Research Council, and was the basis for a far more extensive work published ten years later. His co-author was Philip J. McCarthy, a member of the Committee on Polls’ staff, as we saw earlier: Stephan and McCarthy, Sampling Opinions (note 37).

168 Stephan, ‘Sampling in Studies’ (note 167), pp. 394–97.

169 Stephan, ‘Sampling in Studies’ (note 167), p. 397.

170 Stephan, ‘Sampling in Studies’ (note 167), p. 392.

171 On Stephan’s ‘research diplomacy’ see Converse, Survey Research (note 2), pp. 371–72. She quotes (p. 372), what appears to be a probability fundamentalist: ‘Fred Stephan never gave up on quota sampling—he should have known better.’

172 Fried reports that because of his opposition to ‘quota polling’, Hauser made it into the Committee on Polls only ‘after some dissension’ (Fried, Pathways (note 1), p. 113). A further obstacle to the area samplers’ influence was the fact that the Committee worked in a spirit of ‘cooperative interactions’, as Fried puts, which meant that it allowed the pollsters to review the Committee’s work and provide feedback (Fried, Pathways (note 1), p. 114).

173 Committee on Analysis of Pre-Election Polls, ‘Report’ (note 139), p. 600.

174 Solovey, ‘Riding’ (note 6), p. 412.

175 This was the case for Gallup and Crossley, but not for Roper.

176 Meier and Burke, ‘Laboratory Tests’ (note 89), p. 586.

177 Likert, ‘Why Opinion Polls Were So Wrong’ (note 95), p. 24; Rensis Likert, ‘The Polls: Straw Votes or Scientific Instruments’, American Psychologist, 3 (1948), pp. 556–57 (p. 557); Daniel Katz, ‘Polling Methods and the 1948 Polling Failure’, International Journal of Opinion and Attitude Research, 2 (1948), pp. 469–80 (p. 469).

178 Politz, ‘Useful disaster’ (note 100), p. 36c.

179 Meier and Saunders, The Polls (note 17), pp. 221, 255, 260.

180 Sarah E. Igo, ‘“A Gold Mine and a Tool for Democracy”: George Gallup, Elmo Roper, and the Business of Scientific Polling, 1935–1955’, Journal of the History of the Behavioral Sciences, 42 (2006), 109–134 (p. 126).

181 For an article that reflects well the unsettled nature of sampling theory and practice see Stephan, ‘Sampling in Studies’ (note 167), particularly pp. 394–97.

182 Fried, Pathways (note 1), p. 2.

183 Of course, enforcement of a norm in light of a violation is not the only form of social control. But this is beyond the scope of this paper.

184 Gusfield, Symbolic Crusade (note 111), p. 87.

185 Gusfield, Symbolic Crusade (note 111), p. 65.

186 Michael Link, ‘The Critical Role of Transparency & Standards in Today’s World of Polling and Opinion Research’, American Association for Public Opinion Research, August 1 2014, https://www.aapor.org/Publications-Media/Public-Statements/AAPOR-Response-to-New-York-Times-CBS-News-poll.aspx [accessed May 2017].

187 In December 1958 an Office of Social Sciences was created within the National Science Foundation. In those days, the NSF was composed of ‘Divisions’, the social sciences was not one of them. See Harry Alpert, ‘The Government’s Growing Recognition of Social Science’, The Annals of the American Academy of Political and Social Science, 327 (1960), 59–67 (p. 62).