Who can serve as the proxy for public employees in public administration experiments? a cross-sample comparison

References

• Abdel-Khalik, A. R. 1974. “On the Efficiency of Subject Surrogation in Accounting Research.” The Accounting Review 49: 743–750.
   Web of Science ®Google Scholar
• Anderson, D. M., and J. M. Stritch. 2015. “Goal Clarity, Task Significance, and Performance: Evidence from a Laboratory Experiment.” Journal of Public Administration Research and Theory 26 (2): 211–225. doi:10.1093/jopart/muv019.
   Web of Science ®Google Scholar
• Aronow, P. M., J. Baron, and L. Pinson. 2019. “A Note on Dropping Experimental Subjects Who Fail A Manipulation Check.” Political Analysis 27 (4): 572–589. doi:10.1017/pan.2019.5.
   Web of Science ®Google Scholar
• Bai, H. (2018). Evidence that a Large Amount of Low Quality Responses on MTurk Can Be Detected with Repeated GPS Coordinates. https://www.maxhuibai.com/blog/evidence-that-responses-from-repeating-gps-are-random
   Google Scholar
• Belle, N., and P. Cantarelli. 2019. “Do Ethical Leadership, Visibility, External Regulation, and Prosocial Impact Affect Unethical Behavior? Evidence from a Laboratory and a Field Experiment.” Review of Public Personnel Administration 39 (3): 349–371. doi:10.1177/0734371X17721301.
   Web of Science ®Google Scholar
• Berinsky, A. J., G. A. Huber, and G. S. Lenz. 2017. “Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk.” Political Analysis 20: 351–368.
   Google Scholar
• Buhrmester, M., T. Kwang, and S. D. Gosling. 2011. “Amazon’s Mechanical Turk: A New Source of Inexpensive, yet High-quality, Data?” Perspectives on Psychological Science 6 (1): 3–5. doi:10.1177/1745691610393980.
   Web of Science ®Google Scholar
• Casler, K., L. Bickel, and E. Hackett. 2013. “Separate but Equal? A Comparison of Participants and Data Gathered via Amazon’s MTurk, Social Media, and Face-to-face Behavioral Testing.” Computers in Human Behavior 29 (6): 2156–2160. doi:10.1016/j.chb.2013.05.009.
   Web of Science ®Google Scholar
• Chandler, J., G. Paolacci, E. Peer, P. Mueller, and K. A. Ratliff. 2015. “Using Nonnaive Participants Can Reduce Effect Sizes.” Psychological Science 26 (7): 1131–1139. doi:10.1177/0956797615585115.
   PubMed Web of Science ®Google Scholar
• Clifford, S., R. M. Jewell, and P. D. Waggoner. 2015. “Are Samples Drawn from Mechanical Turk Valid for Research on Political Ideology?” Research & Politics 2 (4): 1–9. doi:10.1177/2053168015622072.
   Web of Science ®Google Scholar
• Dreyfuss, E. (2018). A Bot Panic Hits Amazon’s Mechanical Turk. https://www.wired.com/story/amazon-mechanical-turk-bot-panic/
   Google Scholar
• Druckman, J. N., and C. D. Kam. 2011. “Students as Experimental Participants: A Defense of the ‘Narrow Data Base’.” In Handbook of Experimental Political Science, edited by J. N. Druckman, D. P. Green, J. H. Kuklinski, and A. Lupia, 41–57. New York: Cambridge University Press.
   Google Scholar
• Elliott, W. B., F. D. Hodge, J. J. Kennedy, and M. Pronk. 2007. “Are M.B.A. Students a Good Proxy for Nonprofessional Investors?” The Accounting Review 82 (1): 139–168. doi:10.2308/accr.2007.82.1.139.
   Web of Science ®Google Scholar
• Esteve, M., D. Urbig, A. Van Witteloostuijn, and G. Boyne. 2016. “Prosocial Behavior and Public Service Motivation.” Public Administration Review 76: 177–187.
   Web of Science ®Google Scholar
• European Union. (2016). Behavioural Insights Applied to Policy: Overview across 32 European Countries. http://publications.jrc.ec.europa.eu/repository/bitstream/JRC100146/kjna27726enn_new.pdf
   Google Scholar
• Feitosa, J., D. L. Joseph, and D. A. Newman. 2015. “Crowdsourcing and Personality Measurement Equivalence: A Warning about Countries Whose Primary Language Is Not English.” Personality and Individual Differences 75: 47–52. doi:10.1016/j.paid.2014.11.017.
   Web of Science ®Google Scholar
• Filtenborg, A. F., F. Gaardboe, and J. Sigsgaard-Rasmussen. 2017. “Experimental Replication: An Experimental Test of the Expectancy Disconfirmation Theory of Citizen Satisfaction.” Public Management Review 19 (9): 1235–1250. doi:10.1080/14719037.2017.1295099.
   Web of Science ®Google Scholar
• Hainmueller, J., D. Hangartner, and T. Yamamoto (2015). Validating Vignette and Conjoint Survey Experiments against Real-world Behavior. Proceedings of the National Academy of Sciences, 112, 2395–2400.
   Google Scholar
• Hermann, M. G., and B. Ozkececi-Taner. 2011. “The Experiment and Foreign Policy Decision Making.” In Cambridge Handbook of Experimental Political Science, edited by J. N. Druckman, D. P. Green, J. H. Kuklinski, and A. Lupia, 430–442. New York, NY: Cambridge University Press.
   Google Scholar
• Hillygus, D. S., N. Jackson, and M. Young. 2014. “Professional Respondents in Nonprobability Online Panels.” In Online Panel Research: A Data Quality Perspective, edited by M. Callegaro, R. Baker, P. Lavrakas, J. Krosnick, J. Bethlehem, and A. Gritz, 219–237. West Sussex, UK: Wiley.
   Google Scholar
• Hitlin, P. (2016). Research in the Crowdsourcing Age, a Case Study. https://www.pewinternet.org/2016/07/11/research-in-the-crowdsourcing-age-a-case-study/
   Google Scholar
• Huff, C., and D. Tingley. 2015. ““Who are These People?” Evaluating the Demographic Characteristics and Political Preferences of MTurk Survey Respondents.” Research & Politics 2 (3): 1–12. doi:10.1177/2053168015604648.
   Web of Science ®Google Scholar
• Hughes, C. T., and M. L. Gibson. 1991. “Students as Surrogates for Managers in a Decision-making Environment: An Experimental Study.” Journal of Management Information Systems 8 (2): 153–166. doi:10.1080/07421222.1991.11517925.
   Google Scholar
• Jacobsen, C. B., and L. E. Jensen. 2017. “Why Not “Just for the Money”? an Experimental Vignette Study of the Cognitive Price Effects and Crowding Effects of Performance-related Pay.” Public Performance & Management Review 40 (3): 551–580. doi:10.1080/15309576.2017.1289850.
   Web of Science ®Google Scholar
• James, O., S. R. Jilke, and G. G. Van Ryzin. 2017. “Behavioural and Experimental Public Administration: Emerging Contributions and New Directions.” Public Administration 75 (4): 865–873. doi: 10.1111/padm.12363.
   Google Scholar
• Jilke, S., and L. Tummers. 2018. “Which Clients are Deserving of Help? A Theoretical Model and Experimental Test.” Journal of Public Administration Research and Theory 28 (2):226–238. doi: 10.1093/jopart/muy002
   Web of Science ®Google Scholar
• Johnson, D. R., and L. A. Borden. 2012. “Participants at Your Fingertips: UsingAmazon’s Mechanical Turk to Increase Student–faculty Collaborative Research.” Teaching of Psychology 39 (4): 245–251. doi:10.1177/0098628312456615.
   Web of Science ®Google Scholar
• Jones, W. L., and B. S. Sonner. 2001. “Just Say No to Traditional Student Samples.” Journal of Advertising Research 41 (5): 63–71. doi:10.2501/JAR-41-5-63-71.
   Web of Science ®Google Scholar
• Kennedy, R., S. Clifford, T. Burleigh, P. Waggoner, and R. Jewell (2018). The Shape of and Solutions to the MTurk Quality Crisis. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3272468
   Google Scholar
• Kroll, A., and G. A. Porumbescu. 2019. “When Extrinsic Rewards Become “Sour Grapes”: Anexperimental Study of Adjustments in Intrinsic and Prosocial Motivation.” Review of Public Personnel Administration 39: 467–486. doi:10.1177/0734371X15608419.
   Web of Science ®Google Scholar
• Krupnikov, Y., and A. S. Levine. 2014. “Cross-sample Comparisons and External Validity.” Journal of Experimental Political Science 1 (1): 59–80. doi:10.1017/xps.2014.7.
   Google Scholar
• Lee, M. J., M. J. Moon, and J. Kim. 2017. “Insights from Experiments with Duopoly Games: Rational Incremental Decision-making.” Public Management Review 19 (9): 1328–1351. doi:10.1080/14719037.2017.1282002.
   Web of Science ®Google Scholar
• Levay, K. E., J. Freese, and J. N. Druckman. 2016. “The Demographic and Political Composition of Mechanical Turk Samples.” SAGE Open 6 (1): 1–17. doi:10.1177/2158244016636433.
   Web of Science ®Google Scholar
• Levitt, S. D., and J. A. List. 2007. “What Do Laboratory Experiments Measuring Social Preferences Reveal about the Real World?” Journal of Economic Perspectives 21 (2): 153–174. doi:10.1257/jep.21.2.153.
   Web of Science ®Google Scholar
• Li, H., and G. G. Van Ryzin. 2017. “A Systematic Review of Experimental Studies in Public Management Journals.” In Experiments in Public Management Research: Challenges & Contributions, edited by O. James, S. Jilke, and G. G. Van Ryzin, 20–36. New York, NY: Cambridge University Press.
   Google Scholar
• Li, H., C. Kuo, and M. G. Rusell. 1999. “The Impact of Perceived Channel Utilities, Shopping Orientations, and Demographics on the Consumer’s Online Buying Behavior.” Journal of Computer-Mediated Communication 5: 1–20.
   Google Scholar
• Li, H., J. Liang, H. Xu, and Y. Liu. 2018. “Does Windfall Money Encourage Charitable Giving? An Experimental Study.” VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations 30 (4): 841–848. doi:10.1007/s11266-018-9985-y.
   Web of Science ®Google Scholar
• Li, X., W. Shi, and B. Zhu. 2018. “The Face of Internet Recruitment: Evaluating the Labor Markets of Online Crowdsourcing Platforms in China.” Research & Politics 5 (1): 1–8. doi:10.1177/2053168018759127.
   Web of Science ®Google Scholar
• Liyanarachchi, G. A. 2007. “Feasibility of Using Student Subjects in Accounting Experiments: A Review.” Pacific Accounting Review 19 (1): 47–67. doi:10.1108/01140580710754647.
   Google Scholar
• Lu, X., H. Xu, and W. Wang. 2019. “Clients’ Help Deservingness, Crowd Situational Stress and Discretionary Decision-making: An Experimental Study of Regulatory Street-level Bureaucrats in China.” International Public Management Journal 1–26. doi:10.1080/10967494.2019.1661892.
   Web of Science ®Google Scholar
• Lupton, D. L. 2018. “The External Validity of College Student Subject Pools in Experimental Research: A Cross-sample Comparison of Treatment Effect Heterogeneity.” Political Analysis 27 (1): 90–97. doi:10.1017/pan.2018.42.
   Web of Science ®Google Scholar
• Marvit, M. Z. (2014). How Crowdworkers Became the Ghosts in the Digital Machine. The Nation. http://www.thenation.com/article/how-crowdworkers-became-ghosts-digital-machine/
   Google Scholar
• Mintz, A., S. B. Redd, and A. Vedlitz. 2006. “Can We Generalize from Student Experiments to the Real World in Political Science, Military Affairs, and International Relations?” Journal of Conflict Resolution 50 (5): 757–776. doi:10.1177/0022002706291052.
   Web of Science ®Google Scholar
• Mullinix, K. J., T. J. Leeper, J. N. Druckman, and J. Freese. 2016. “The Generalizability of Survey Experiments.” Journal of Experimental Political Science 2 (2): 109–138. doi:10.1017/XPS.2015.19.
   Google Scholar
• Oppenheimer, D. M., T. Meyvis, and N. Davidenko. 2009. “Instructional Manipulation Checks: Detecting Satisficing to Increase Statistical Power.” Journal of Experimental Social Psychology 45 (4): 867–872. doi:10.1016/j.jesp.2009.03.009.
   Web of Science ®Google Scholar
• Paolacci, G., and J. Chandler. 2014. “Inside the Turk: UnderstandingMechanical Turk as a Participant Pool.” Current Directions in Psychological Science 23 (3): 184–188. doi:10.1177/0963721414531598.
   Web of Science ®Google Scholar
• Porter, C. O. L. H., R. Outlaw, J. P. Gale, and T. S. Cho. 2019. “The Use of Online Panel Data in Management Research: A Review and Recommendations.” Journal of Management 45 (1): 319–344. doi:10.1177/0149206318811569.
   Web of Science ®Google Scholar
• Remus, W. 1986. “Graduate Students as Surrogates for Managers in Experiments on Business Decision Making.” Journal of Business Research 14 (1): 19–25. doi:10.1016/0148-2963(86)90053-6.
   Web of Science ®Google Scholar
• Rosenthal, R. W., and R. L. Rosnow. 1969. Artifact in Behavioral Research. New York: Academic Press.
   Google Scholar
• Ryan, T. J. (2018). Data Contamination on MTurk. http://timryan.web.unc.edu/2018/08/12/data-contamination-on-mturk/
   Google Scholar
• Sears, D. O. 1986. “College Sophomores in the Laboratory: Influences of a Narrow Data Base on Social Psychology’s View of Human Nature.” Journal of Personality and Social Psychology 51 (3): 515–530. doi:10.1037/0022-3514.51.3.515.
   Web of Science ®Google Scholar
• Smith, N. A., I. E. Sabat, L. R. Martinez, K. Weaver, and S. Xu. 2015. “A Convenient Solution: Using MTurk to Sample from Hard-to-reach Populations.” Industrial and Organizational Psychology 8 (2): 220–228. doi:10.1017/iop.2015.29.
   Web of Science ®Google Scholar
• Stokel-Walker, C. (2018). Bots on Amazon’s Mechanical Turk are Ruining Psychology Studies. https://www.newscientist.com/article/2176436-bots-on-amazons-mechanical-turk-are-ruining-psychology-studies/
   Google Scholar
• Stritch, J. M., M. J. Pedersen, and G. Taggart. 2017. “The Opportunities and Limitations of Using Mechanical Turk (Mturk) in Public Administration and Management Scholarship.” International Public Management Journal 20 (3): 489–511. doi:10.1080/10967494.2016.1276493.
   Web of Science ®Google Scholar
• Tsang, E. W. K., and K.-M. Kwan. 1999. “Replication and Theory Development in Organizational Science: A Critical Realist Perspective.” Academy of Management Review 24 (4): 759–780. doi:10.5465/amr.1999.2553252.
   Web of Science ®Google Scholar
• Van Ryzin, G. G., N. M. Riccucci, and H. Li. 2017. “Representative Bureaucracy and Its Symbolic Effect on Citizens: A Conceptual Replication.” Public Management Review 19 (9): 1365–1379. doi:10.1080/14719037.2016.1195009.
   Web of Science ®Google Scholar
• Walker, R. M., O. James, and G. A. Brewer. 2017. “Replication, Experiments and Knowledge in Public Management Research.” Public Management Review 19 (9): 1221–1234. doi:10.1080/14719037.2017.1282003.
   Web of Science ®Google Scholar
• Walter, S. L., S. E. Seibert, D. Goering, and E. H. O’Boyle. 2019. “A Tale of Two Sample Sources: Do Results from Online Panel Data and Conventional Data Converge?” Journal of Business and Psychology 34 (4): 425–452. doi:10.1007/s10869-018-9552-y
   Web of Science ®Google Scholar