Online field experiments

ABSTRACT

Changes in information communication technology across the Asian region have altered our field substantively and methodologically. The rapid growth of digitized communications allows us to find new purchase in examining questions fundamental to our understanding of communication theories, norms, and practices across Asia. While methods such as text mining and user analytics are increasingly popular among computational scholars, here, we focus on online field experiments, an approach to studying communication that has the potential to overcome many existing obstacles to social scientific inquiry but one that has been used relatively rarely in Asia. In this paper, we discuss what online field experiments are and how they differ from traditional experiments as well as online lab and survey experiments. We show how researchers can go about designing and implementing online field experiments, focusing on issues where online field experiments differ from their traditional counterparts – legal and ethical considerations, construct validity, randomization and spillover, and statistical analyses. Finally we discuss how online field experiments can advance our understanding of communication in Asia by helping researchers to gain insight and make causal inferences on attitudes, behaviors, and interactions that were previously unobservable ℘.

KEYWORDS:

• Asia
• online experiments
• field experiments
• experimental design

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Daniel Muise is a Ph.D. student in the Department of Communication at Stanford University. His research focuses on how smartphone usage impacts the development of public goods and democratic institutions. He graduated summa cum laude from the University of Massachusetts in 2016, with degrees in Political Science and Economics. Prior to studying at Stanford University, Muise was a research assistant at Harvard University, co-developing privacy preservation algorithms for social data.

Jennifer Pan is an Assistant Professor of Communication, and an Assistant Professor, by courtesy, of Political Science and Sociology at Stanford University. Her research focuses on the politics of authoritarian countries in the digital age, combining experimental and computational methods with large-scale datasets on political activity in China and other authoritarian regimes. Pan’s work has appeared in peer reviewed publications such as the American Political Science Review, American Journal of Political Science, Comparative Political Studies, Journal of Politics, and Science. Pan received her Ph.D. from Harvard University’s Department of Government in 2015. She graduated from Princeton University, summa cum laude, in 2004, and until 2009, she was a consultant at McKinsey & Company.

Notes

1. See Green, Calfano, and Aronow (2014) for description on the state of field experiments in media effects research.

2. Also, online lab and survey experiments are relatively similar to traditional lab and survey experiments in their design and implementation.

3. For details on value relevance affirmation interventions, see Kizilcec, Saltarelli, Reich, and Cohen (2017).