The history of the use of self-reports and the methodology of economics

Abstract

The main arguments currently held for and against the use of self-reports in economics are presented in their relation to well-known events in the history of the discipline: the ‘measurement without theory’, the ‘full-cost’, and the ‘economic expectations’ controversies. Doing so, the paper highlights the so far neglected role of George Katona's behavioral economics in these methodological discussions.

Keywords:

• self-reports
• George Katona's behavioral economics
• methodological controversies

Acknowledgements

A previous version of this essay was presented at the 2008 Conference on the History of Recent Economics. I want to thank Philippe Fontaine, Roger Backhouse and Tiago Mata (the organizers) as well as D. Wade Hands, Dan Hammond, Annie L. Cot, Harro Maas and two anonymous referees for their comments, which helped improving that version. The usual disclaimers apply.

Notes

 1. By ‘subjective data’, ‘self-reports’ or ‘self-testimonies’ it is meant survey reports of inner psychological states (e.g., ‘opinions’, ‘intentions’, ‘satisfaction’, ‘happiness’) as well as subjective valuations of goods and services (e.g., ‘willingness to pay’, ‘willingness to buy’). This category excludes survey data of past or current ‘objective’ outcomes, such as answers to questions like: ‘were you employed last month?’, or the kind of data discussed by Wallis and Friedman in ‘The Empirical Derivation of Indifference Functions’ (1942), i.e., ‘income, wealth, prices, family type, occupation, age, nationality, regional location, type of community, ownership of home or automobile, etc.’ (p. 189). See Boulier and Goldfarb (1998), for a methodological study of different types of survey data. Recent examples as well as numerous references to studies using self-reports can be found in Manski (2004), Lokshin and Ravallion (2008), Schläpfer (2008), Clark et al. (2008, 2009), and supplementary discussion in Blinder and Krueger (2004) and Ferrer-i-Carbonell and Frijters (2004).

 2. According to McCloskey (1983) one can even ‘get an audience of economists to laugh out loud by proposing ironically to send out a questionnaire on some disputed economic point’ (p. 514).

 3. George Katona [1901–1981] the ‘pioneer student and chief collector of consumer anticipations data’ (Tobin 1959, p. 1) was a Hungarian psychologist and economic journalist who emigrated from Europe in 1933. After studying and teaching Gestalt psychology at the New School (1936–1942) he worked at the Cowles Commission (1942–1944), the Division of Program Surveys of the Department of Agriculture (1945) and the Survey Research Center of the University of Michigan (1946–1980). During the last of these periods, Katona's main aim was of developing a macroeconomics-focused form of behavioral economics based on the analysis of self-reports by consumers. See Warneryd (1982) and Edwards (2012) for more detailed accounts of Katona's life and work.

 4. Accounts of the economic use of self-reports are rare. The main references found were McCloskey (1983), Blinder (1991), Hanemann (1994), Boulier and Goldfarb (1998), Bertrand and Mullainathan (2001), Hamermesh (2004), Easterlin (2004), Manski (2004), and Schläpfer (2008). Unlike all other contributors, Boulier and Goldfarb (1998) claim that the ‘majority position among economists and economic methodologists is that our unwillingness to use surveys is well founded’ (Boulier and Goldfarb 1998, p. 1).

 5. The aim of this section is to defend this point by presenting a few examples. Another important field of discussion of the use of self-reports is environmental and ecological economics. Contingent valuations have been widely discussed in the same terms proposed throughout this section, mainly in a symposium by P. Portney, W. Hanemann, P. Diamond and J. Hausman in The Journal of Economic Perspectives (1994, Vol. 8, No. 4). More recently, Schläpfer (2008) has argued that the economics profession has ‘enforced something of a prohibition on the collection of subjective data’ (Manski 2000, in Schläpfer 2008, p. 729), a ‘self-imposed rule’ explained by the dominance of ‘revealed preference techniques’ (Schläpfer 2008, p. 730). See Edwards (2009) for a more detailed account of this subject.

 6. Easterlin's (2004) position exemplifies the main views of economists invested in the analysis of subjective well-being data (i.e. happiness). See Layard (2005), Senik (2005) and Clark et al. (2008) for examples of economic studies of happiness, and Tribe (2008) for a critical review of them.

 7. For Easterlin, ‘hard’ data, such as ‘inflation rates’, ‘unemployment rates’ and estimates of ‘gross domestic product’ are all subject to serious quality problems, although economists ‘disregard the possible biases in these data or even forget about them’ (2004, p. 29). Less familiar data, like self-reports, are supposedly dismissed not because of their quality, but because of unexamined commitments (p. 29).

 8. As presented by Morgan (1990), W.C. Mitchell's pioneering work on business cycles emphasized fact observation rather than theory fitting. However, as shown below – and especially in relation to the measurement without theory controversy – that approach was ‘dismissed by economists as purely descriptive’ (Easterlin 2004, p. 10).

 9. See Dominitz and Manski (1997) and Manski and Straub (2000) for accounts of the Survey of Economic Expectations (SEE) produced since 1993 at the University of Wisconsin Survey Center.

10. Easterlin (2004) contests B&M's (2001) argument in the following terms: ‘A recent article by two economists on subjective survey data […] states as its ‘primary objective’ to turn economists’ ‘vague implicit distrust’ of subjective testimony ‘into an explicit position grounded in facts’. […], this strikes me as a rearguard action. The use of subjective testimony in economics is growing. Its use outside economics has expanded to the point that serious challenges are being made to the efficacy of economic policy. When asked by nonspecialists about such challenges, economists will not be able to hide for long behind the defense that subjective testimony is no good' (Easterlin 2004, p. 31).

11. The main elements of the measurement without theory controversy are Burns and Mitchell's (1946) book and the ensuing discussion between Koopmans (1947, 1949) and Vining (1949a,b). Secondary literature can be found in works by Epstein (1987), Morgan (1990), Christ (1994) and Mirowski (2002), which remain definitive contributions to the analysis of the controversy.

12. A similar argument was held by Koopmans against Vining's (1949a) reply to his 1947 paper. In a footnote, he claimed to be embarrassed of being credited the Cowles approach regardless the contributions of Fisher, Frisch or Tinbergen: it is ‘embarrassing to me to see the methodological approach that resulted from the cumulative effort of these men to be referred to as ‘Koopmans’ methods’ or the ‘methods of Koopmans’ group’ (Koopmans 1949, p. 86).

13. According to Mirowski's (2002) account of the controversy and the relationship between the Cowles Commission and the NBER, the ‘most standard’ narratives focus on the Cowles role in the development of econometrics. Alternatively, Mirowski highlights the importance of ‘the Walrasian system of price theory’ in the Cowles program (Mirowski 2002, p. 216). There are virtually no accounts of the controversy including Katona's studies, except for Epstein's three pages on ‘Wartime Price Controls and the Cowles Commission’ (1987, pp. 82–85).

14. The Committee members were O. Lange, J. Marschak, E. Mason, J. Viner, C. Wilcox, and T. Yntema.

15. See Edwards (2012) for a more detailed account of the inception and development of Katona's behavioral economics.

16. See Edwards (2009) for a more detailed account of Katona's Price Control and Business (1945).

17. There is a vast literature on the full-cost controversy, the main elements being Harrod (1939), Hall and Hitch (1939), Lester (1946, 1947), Machlup (1946, 1947), and secondary literature by, namely, Machlup (1955, Avery et al. 1984), Lee (1981), Langlois and Koppl (1991), Mongin (1992, 1997, 2000) and Boulier and Goldfarb (1998). This section deals only with the controversy between Lester and Machlup in 1946 and not with the earlier project of the Oxford Group (late-1930s), which (like Katona) used detailed interviews aimed at finding out ‘how businessmen would react to government intervention to control the trade cycle’ (Lee 1981, p. 340). The Oxford Group asked ‘a sample of entrepreneurs how they did in fact make up their minds what to do when faced with specified changes in their business position’ (Harrod 1939, p. 2), concluding that they charged prices ‘equal to prime cost plus an allowance for overheads’ (p. 2). Consequently, it was claimed that economists would get results ‘closer to the truth by assuming’ the full-cost principle rather than ‘equating marginal cost to marginal revenue’ (p. 2). Unfortunately for historians, ‘the entire proceedings of the Group were burned’ while it ‘disbanded’ after the Nazi invasion (Lee 1981, p. 348).

18. ‘A detailed questionnaire was mailed in June 1945, to the presidents or executive officers of 430 Southern manufacturing firms […] only 58 replies contain[ed] answers to two or more of the questions […]. Employment in these 58 firms averaged 600.’ (Lester 1946, p. 64)

19. The first part attempted to clarify the theory of the firm, which was ‘not as ambitious as is often believed’ (Machlup 1946, p. 520) for it only explained what ‘certain changes in conditions may have upon the actions of the firm’ (p. 521). It explored: ‘A. Marginal Revenue and Cost of Output’, and ‘B. Marginal Productivity and Cost of Input’ and proposed the well-known ‘driver analogy’ explaining that businessmen did not proceed according to the theory because of the ‘extreme difficulty of calculating’. Like drivers who overtake without having a ‘theory of overtaking’, businessmen would ‘“just know,” in a vague and rough way, whether or not it would pay […] to hire more men’ (p. 535). See Boulier and Goldfarb (1998) for an account of that analogy.

20. Mongin's (1992) reading of the controversy is a good example of this, for even though focused on empirical studies (i.e., Eiteman 1947; Edwards 1952; Earley 1955, 1956) it totally neglects Katona's work.

21. In his analysis of business expectations, Katona (1946) explored the distinction between ‘routine behavior’ and ‘genuine decisions’. After acknowledging the contributions of F. Knight, J.M. Keynes, J. Schumpeter, J. Tinbergen, J. Hicks, and G. Shackle to the analysis of expectations, Katona explained that, whenever conscious decisions were made, it happened simultaneously for a large number of businessmen as a function of sudden changes in expectations: ‘First, they indicate that new expectations bringing forth new economic decisions are not an everyday occurrence […]. Secondly, expectations can and even typically do change radically […]. Thirdly, it follows from the psychological findings that when expectations do change, they are likely to change at about the same time and in the same direction for many individual businessmen.’ (Katona 1946, p. 53)

22. This position is somehow analogue to Wallis and Friedman's (1942) doubt that ‘the indifference function in pure theory has any material value for the organization of empirical data.’ (p. 189)

23. See Edwards (2012) for a detailed account of the inception, development and uses of the SCF and SCAs.

24. The Consultant Committee, also known as the Smithies Committee, was composed by A. Smithies (chairman), H. Kyrk, G. Orcutt, H. Passer, B. Seidman, S. Stouffer and J. Tobin.

25. For Katona, expectations were a ‘subgroup of the more general concept of attitudes’ (Katona 1957, p. 41), which were defined as ‘generalized viewpoints’ determining consumers' ‘perceptions, cognitions, and behavior’ (p. 41). Opposing the ‘atomistic’ viewpoint of the consultant committee, Katona criticized the report's ‘assumption that each individual attitude, taken in isolation, should have a specific relation, not changing over time, to action variables’ (p. 43). He insisted in pointing out that the ‘basic tenet of Gestalt psychology’, in which his approach originated, was that ‘a part or item may change its meaning and function according to the whole to which it belongs’ (p. 43). Instead of ‘testing the predictive value of each attitude separately’, he claimed that ‘the relation of clusters of attitudes to behavior should be studied’ (p. 43).

26. See Edwards (2009) for an account of the rise and fall of Katona's behavioral economics.

27. See McNeil (1974) for an account of the rise and fall of the Surveys of Consumer Buying Intentions and Consumer Buying Expectations, both sponsored by the Federal Reserve. Avery et al. (1984) and Kennickell and Starr-McCluer (1994) deals with the current design of the SCF, as conducted by the National Opinion Research Center at the University of Chicago.