From Ontological Traits to Validity Challenges in Social Science: The Cases of Economic Experiments and Research Questionnaires

References

• AERA, APA, and NCME (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education). 2014. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
   Google Scholar
• Alm, J., G. H. McClelland, and W. D. Schulze. 1992. “Why Do People Pay Taxes?” Journal of Public Economics 48: 21–38. doi: 10.1016/0047-2727(92)90040-M
   Web of Science ®Google Scholar
• Angoff, W. H. 1988. “Validity: An Evolving Concept.” In Test Validity, edited by H. Wainer and H. I. Braun, 19–32. Hillsdale, NJ: Lawrence Erlbaum.
   Google Scholar
• APA (American Psychological Association, Committee on Test Standards). 1952. “Technical Recommendations for Psychological Tests and Diagnostic Techniques: A Preliminary Proposal.” American Psychologist 7: 461–475. doi: 10.1037/h0056631
   Web of Science ®Google Scholar
• Bardsley, N. 2005. “Experimental Economics and the Artificiality of Alteration.” Journal of Economic Methodology 12: 239–251. doi: 10.1080/13501780500086115
   Google Scholar
• Barron, K. E., A. R. Brown, T. E. Egan, C. R. Gesualdi, and K. A. Marchuck. 2008. “Validity.” In 21st Century Psychology: A Reference Handbook, edited by S. F. Davis and W. Buskist, 55–64. Thousand Oaks, CA: Sage.
   Google Scholar
• Beullens, K., and G. Loosveldt. 2014. “Interviewer Effects on Latent Constructs in Survey Research.” Journal of Survey Statistics and Methodology 2: 433–458. doi: 10.1093/jssam/smu019
   Google Scholar
• Beullens, K., and G. Loosveldt. 2016. “Interviewer Effects in the European Social Survey.” Survey Research Methods 10: 103–118.
   Web of Science ®Google Scholar
• Billiet, J. B., L. Waterplas, and G. Loosveldt. 1992. “Context Effects as Substantive Data in Social Surveys.” In Context Effects in Social and Psychological Research, edited by N. Schwarz and S. Sudman, 131–147. Berlin: Springer.
   Google Scholar
• Binham, W. V. 1937. Aptitudes and Aptitude Testing. New York: Harper.
   Google Scholar
• Borsboom, D., A. O. J. Cramer, R. A. Kievit, A. Z. Scholten, and S. Franic. 2009. “The End of Construct Validity.” In The Concept of Validity: Revisions, New Directions, and Applications, edited by R. W. Lissitz, 135–170. Charlotte, NC: Information Age Publishing.
   Google Scholar
• Boumans, M. 2016. “Methodological Ignorance: A Comment on Field Experiments and Methodological Intolerance.” Journal of Economic Methodology 23: 139–146. doi: 10.1080/1350178X.2016.1158947
   Web of Science ®Google Scholar
• Camerer, C. F. 2015. “The Promise and Success of Lab – Field Generalizability in Experimental Economics: A Critical Reply to Levitt and List.” In Handbook of Experimental Economic Methodology, edited by G. R. Fréchette and A. Schotter, 249–295. Oxford: Oxford University Press.
   Google Scholar
• Campbell, D. T. 1957. “Factors Relevant to the Validity of Experiments in Social Settings.” Psychological Bulletin 54: 297–312. doi: 10.1037/h0040950
   PubMed Web of Science ®Google Scholar
• Carmines, E. G., and R. A. Zeller. 1979. Reliability and Validity Assessment. London: Sage.
   Google Scholar
• Carpenter, J., C. Connolly, and C. K. Myers. 2008. “Altruistic Behavior in a Representative Dictator Experiment.” Experimental Economics 11: 282–298. doi: 10.1007/s10683-007-9193-x
   Web of Science ®Google Scholar
• Chang, H. 2004. Inventing Temperature: Measurement and Scientific Progress. Oxford: Oxford University Press.
   Google Scholar
• Cordeiro-dos-Santos, A. C. 2006. “The Social Epistemology of Experimental Economics.” PhD thesis, Erasmus Universiteit Rotterdam.
   Google Scholar
• Cronbach, L. J. 1989. “Construct Validation After Thirty Years.” In Intelligence: Measurement, Theory, and Public Policy, edited by R. E. Linn, 147–171. Urbana: University of Illinois Press.
   Google Scholar
• Cronbach, L. J., and P. E. Meehl. 1955. “Construct Validity in Psychological Tests.” Psychological Bulletin 52: 281–302. doi: 10.1037/h0040957
   PubMed Web of Science ®Google Scholar
• Davidson, D. [1963] 1989. “Actions, Reasons, and Causes.” In Essays on Actions and Events, edited by D. Davidson, 3–19. Oxford: Clarendon Press.
   Google Scholar
• Davidson, D. [1974] 1989. “Psychology as Philosophy.” In Essays on Actions and Events, edited by D. Davidson, 229–239. Oxford: Clarendon Press.
   Google Scholar
• Dietrich, F., and C. List. 2012. “Mentalism Versus Behaviorism in Economics: A Philosophy-of-science Perspective.” Munich Personal RePEc Archive. http://mpra.ub.uni-muenchen.de/id/eprint/43231.
   Google Scholar
• Duprè, J. 2016. “Social Science: City Center or Leafy Suburb.” Philosophy of the Social Sciences 46: 548–564. doi: 10.1177/0048393116649713
   Web of Science ®Google Scholar
• Favereau, J. 2016. “On the Analogy Between Field Experiments in Economics and Clinical Trials in Medicine.” Journal of Economic Methodology 23: 203–222. doi: 10.1080/1350178X.2016.1157202
   Web of Science ®Google Scholar
• Franklin, A. 2005. No Easy Answers: Science and the Pursuit of Knowledge. Pittsburgh: University of Pittsburgh Press.
   Google Scholar
• Galison, P. 1997. Image and Logic: A Material Culture of Microphysics. Chicago, IL: University of Chicago Press.
   Google Scholar
• García-Pérez, M. A. 2012. “Statistical Conclusion Validity: Some Common Threats and Simple Remedies.” Frontiers in Psychology 3: art. 325.
   Web of Science ®Google Scholar
• Greenhill, M., Z. Leviston, R. Leonard, and I. Walker. 2014. “Assessing Climate Change Beliefs: Response Effects of Question Wording and Response Alternatives.” Public Understanding of Science 23: 947–965. doi: 10.1177/0963662513480117
   PubMed Web of Science ®Google Scholar
• Greenwood, J. D. 1982. “On the Relation Between Laboratory Experiments and Social Behavior: Causal Explanation and Generalisation.” Journal of the Theory of Social Behavior 12: 225–250. doi: 10.1111/j.1468-5914.1982.tb00449.x
   Web of Science ®Google Scholar
• Groves, R. M., and E. Singer. 2004. “Survey Methodology.” In A Telescope on Society: Survey Research and Social Science at the University of Michigan and Beyond, edited by J. S. House, F. T. Juster, R. L. Kahn, H. Schuman, and E. Singer, 21–64. Ann Arbor: University of Michigan Press.
   Google Scholar
• Groves, R. M., E. Singer, and A. Corning. 2000. “Leverage-saliency Theory of Survey Participation: Description and an Illustration.” Public Opinion Quarterly 64: 299–308. doi: 10.1086/317990
   PubMed Web of Science ®Google Scholar
• Guala, F. 2005. The Methodology of Experimental Economics. Cambridge: Cambridge University Press.
   Google Scholar
• Guala, F. 2017. “Experimental Methodology on the Move.” Journal of Economic Methodology 24: 108–114. doi: 10.1080/1350178X.2017.1287557
   Web of Science ®Google Scholar
• Hacking, I. 1983. Representing and Intervening: Introductory Topics in the Philosophy of Natural Science. Cambridge: Cambridge University Press.
   Google Scholar
• Hacking, I. [1995] 2011. “The Looping Effects of Human Kinds.” In The Philosophy of Social Science Reader, edited by D. Steel and F. Guala, 24–38. London: Routledge.
   Google Scholar
• Hertwig, R., and A. Ortmann. 2001. “Experimental Practices in Economics: A Methodological Challenge for Psychologists?” Behavioral and Brain Sciences 24: 383–403. doi: 10.1017/S0140525X01004149
   PubMed Web of Science ®Google Scholar
• Hogarth, R. M. 2005. “The Challenge of Representative Design in Psychology and Economics.” Journal of Economic Methodology 12: 253–263. doi: 10.1080/13501780500086172
   Google Scholar
• Huff, T. E. 1982b. “On the Methodology of the Social Sciences: A Review Essay. Part III.” Philosophy of the Social Sciences 12: 205–219. doi: 10.1177/004839318201200210
   Google Scholar
• Ivarola, L. 2017. “Socioeconomic Processes as Open-ended Results: Beyond Invariance Knowledge for Interventionist Purposes.” Theoria 32: 211–229. doi: 10.1387/theoria.16184
   Web of Science ®Google Scholar
• Kagel, J. H. 2015. “Laboratory Experiments: The Lab in Relationship to Field Experiments, Field Data, and Economic Theory.” In Handbook of Experimental Economic Methodology, edited by G. R. Fréchette and A. Schotter, 339–359. Oxford: Oxford University Press.
   Google Scholar
• Kane, M. 2006b. “In Praise of Pluralism: A Comment on Borsboom.” Psychometrika 71: 441–445. doi: 10.1007/s11336-006-1491-2
   PubMed Web of Science ®Google Scholar
• Kelley, T. L. 1927. Interpretation of Educational Measurement. Yonkers-on-Hudson, NY: World Book.
   Google Scholar
• Kim, J. 1988. “What Is ‘Naturalized Epistemology’?” In Epistemology: An Anthology, edited by E. Sosa and J. Kim, 301–313. Malden, MA: Blackwell.
   Google Scholar
• Krosnick, J. A., and S. Presser. 2010. “Question and Questionnaire Design.” In Handbook of Survey Research, 2nd ed., edited by P. Marsden and J. Wright, 263–313. Bingley: Emerald.
   Google Scholar
• Landry, C. E., A. Lange, J. A. List, M. K. Price, and N. G. Rupp. 2006. “Toward an Understanding of the Economics of Charity: Evidence from a Field Experiment.” The Quarterly Journal of Economics 121: 747–782. doi: 10.1162/qjec.2006.121.2.747
   Web of Science ®Google Scholar
• Lawson, T. 2003. Reorienting Economics. London: Routledge.
   Google Scholar
• Levin, I. P., S. L. Schneider, and G. J. Gaeth. 1998. “All Frames Are Not Created Equal: A Typology and Critical Analysis of Framing Effects.” Organizational Behavior and Human Decision Process 76: 149–188. doi: 10.1006/obhd.1998.2804
   PubMed Web of Science ®Google Scholar
• Levitt, S. D., and J. A. List. 2009. “Field Experiments in Economics: The Past, the Present, and the Future.” European Economic Review 53: 1–18. doi: 10.1016/j.euroecorev.2008.12.001
   Web of Science ®Google Scholar
• Mäki, U. 2008. “Scientific Realism and Ontology.” In The New Palgrave Dictionary of Economics, 2nd ed., edited by S. N. Durlauf and L. E. Blume, Vol. 7, 334–341. Basingstoke: Palgrave Macmillan.
   Google Scholar
• Malone, E. L., J. J. Dooley, and J. A. Bradbury. 2010. “Moving From Misinformation Derived From Public Attitude Surveys on Carbon Dioxide Capture and Storage Towards Realistic Stakeholder Involvement.” International Journal of Greenhouse Gas Control 4: 419–425. doi: 10.1016/j.ijggc.2009.09.004
   Web of Science ®Google Scholar
• Mayo, D. G. 1996. Error and the Growth of Experimental Knowledge. Chicago, IL: University of Chicago Press.
   Google Scholar
• McKenzie, C. R. M., and J. T. Wixted. 2001. “Participant Skepticism: If You Can’t Beat It, Model It.” Behavioral and Brain Sciences 24: 424–425. doi: 10.1017/S0140525X01464144
   Web of Science ®Google Scholar
• Messick, S. 1989. “Validity.” In Educational Measurement, 3rd ed., edited by R. L. Linn, 13–103. New York: American Council on Education.
   Google Scholar
• Mitchell, S. 2009. “Complexity and Explanation in the Social Sciences.” In Philosophy of the Social Sciences: Philosophical Theory and Scientific Practice, edited by C. Mantzavinos, 130–145. Cambridge: Cambridge University Press.
   Google Scholar
• Morawetz, U. B., H. de Groote, and S. C. Kimenju. 2011. “Improving the Use of Experimental Auctions in Africa: Theory and Evidence.” Journal of Agricultural and Resource Economics 36: 263–279.
   Web of Science ®Google Scholar
• Morgan, S. L., and E. S. T. Poppe. 2015. “A Design and a Model for Investigating the Heterogeneity of Context Effects in Public Opinion Surveys.” Sociological Methodology 45: 184–222. doi: 10.1177/0081175015570095
   Web of Science ®Google Scholar
• Musgrave, A. 1981. “Unreal Assumptions in Economic Theory: The F-twist Untwisted.” Kyklos 34: 377–387. doi: 10.1111/j.1467-6435.1981.tb01195.x
   Web of Science ®Google Scholar
• Pelham, B. W., and H. Blanton. 2003. Conducting Research in Psychology. Measuring the Weight of Smoke. Belmont: Thomson Learning.
   Google Scholar
• Read, D. 2005. “Monetary Incentives, What Are They Good for?” Journal of Economic Methodology 12: 265–276. doi: 10.1080/13501780500086180
   Google Scholar
• Rosenberg, A. 2009. “If Economics Is a Science, What Kind of a Science Is It?” In The Oxford Handbook of Philosophy of Economics, edited by H. Kincaid and D. Ross, 55–67. Oxford: Oxford University Press.
   Google Scholar
• Rosenberg, A. 2015. Philosophy of Social Science. 5th ed. Boulder, CO: Westview Press.
   Google Scholar
• Rosenboim, M., and T. Shavit. 2012. “Whose Money Is It Anyway? Using Prepaid Incentives in Experimental Economics to Create a Natural Environment.” Experimental Economics 15: 145–157. doi: 10.1007/s10683-011-9294-4
   Web of Science ®Google Scholar
• Ross, D. 2010. “Why Economic Modelers Can’t Exclude Psychological Processing Variables.” Journal of Economic Methodology 17: 87–92. doi: 10.1080/13501780903542904
   Google Scholar
• Rulon, P. J. 1946. “On the Validity of Educational Tests.” Harvard Educational Review 16: 290–296.
   Web of Science ®Google Scholar
• Ryle, G. [1949] 2009. The Concept of Mind. London: Routledge.
   Google Scholar
• Schotter, A. 2008. “What’s So Informative About Choice?” In The Foundations of Positive and Normative Economics: A Handbook, edited by A. Caplin and A. Schotter, 70–94. Oxford: Oxford University Press.
   Google Scholar
• Shadish, W. R., T. D. Cook, and D. T. Campbell. 2002. Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin.
   Google Scholar
• Sireci, S. G. 2009. “Packing and Unpacking Sources of Validity Evidence: History Repeats Itself Again.” In The Concept of Validity: Revisions, New Directions, and Applications, edited by R. W. Lissitz, 19–37. Charlotte, NC: Information Age Publishing.
   Google Scholar
• Smith, V. L. 1962. “An Experimental Study of Competitive Market Behavior.” Journal of Political Economy 70: 111–137. doi: 10.1086/258609
   Web of Science ®Google Scholar
• Smith, V. L. 1982. “Microeconomic Systems as an Experimental Science.” American Economic Review 72: 923–955.
   Web of Science ®Google Scholar
• Smith, V. L. 2010. “Theory and Experiment: What Are the Questions?” Journal of Economic Behavior and Organization 73: 3–15. doi: 10.1016/j.jebo.2009.02.008
   Web of Science ®Google Scholar
• Smyth, J. D., D. A. Dillman, and L. M. Christian. 2009. “Context Effects in Internet Surveys: New Issues and Evidence.” In Oxford Handbook of Internet Psychology, edited by A. N. Joinson, K. Y. A. McKenna, T. Postmes, and U.-D. Reips, 429–446. New York: Oxford University Press.
   Google Scholar
• Søberg, M. 2005. “The Duhem-Quine Thesis and Experimental Economics: A Reinterpretation.” Journal of Economic Methodology 12: 581–597. doi: 10.1080/13501780500343680
   Google Scholar
• Steinle, F. 1997. “Entering New Fields: Exploratory Uses of Experimentation.” Philosophy of Science 64: S65–S74. doi: 10.1086/392587
   Web of Science ®Google Scholar
• Suppes, P. 1962. “Models of Data.” In Logic, Methodology and Philosophy of Science: Proceedings of the 1960 International Congress, edited by E. Nagel, P. Suppes, and A. Tarski, 252–261. Stanford, CA: Stanford University Press.
   Google Scholar
• Suppes, P. 1982. “Problems of Causal Analysis in the Social Sciences.” Epistemologia 5: 239–250.
   Google Scholar
• Tal, E. 2016. “Making Time: A Study in the Epistemology of Measurement.” British Journal for the Philosophy of Science 67: 297–335. doi: 10.1093/bjps/axu037
   Web of Science ®Google Scholar
• Thurstone, L. L. 1932. The Reliability and Validity of Tests. Ann Arbor, MI: Edwards Brothers.
   Google Scholar
• Tyler, T. R., and D. Amodio. 2015. “Psychology and Economics: Areas of Convergence and Difference.” In Handbook of Experimental Economic Methodology, edited by G. R. Fréchette and A. Schotter, 181–196. Oxford: Oxford University Press.
   Google Scholar
• Van Bochove, M., J. Burgers, A. Geurts, W. de Koster, and J. van der Waal. 2015. “Questioning Ethnic Identity: Interviewer Effects in Research About Immigrants’ Self-definition and Feelings of Belonging.” Journal of Cross-Cultural Psychology 46: 652–666. doi: 10.1177/0022022115576961
   Web of Science ®Google Scholar
• Weber, M. 1949. The Methodology of the Social Sciences . Translated by E. A. Shils and H. A. Finch. Glencoe, IL: Free Press.
   Google Scholar
• Wiggins, J. A. 1968. “Hypothesis Validity and Experimental Laboratory Methods.” In Methodology in Social Research, edited by H. M. Blalock and A. B. Blalock, 390–427. New York: McGraw-Hill.
   Google Scholar
• Yount, K. M., N. Halim, M. Hynes, and E. R. Hillman. 2011. “Response Effects to Attitudinal Questions About Domestic Violence Against Women: A Comparative Perspective.” Social Science Research 40: 873–884. doi: 10.1016/j.ssresearch.2010.12.009
   Web of Science ®Google Scholar
• Zaller, J., and S. Feldman. 1992. “A Simple Theory of the Survey Response: Answering Questions Versus Revealing Preferences.” American Journal of Political Science 36: 579–616. doi: 10.2307/2111583
   Web of Science ®Google Scholar