Moving the campaign from the front door to the front pocket: field experimental evidence on the effect of phrasing and timing of text messages on voter turnout

ABSTRACT

Despite the widespread scholarly attention given to get-out-the-vote tactics the recent one and a half decade, few have studied the effect of short text messages (SMS) on voter turnout, and no previous such study has been conducted outside the US. We analyze four SMS experiments with more than 300,000 voters conducted in relation to two elections in Denmark and find intention-to-treat (ITT) effects between 0.33 and 1.82 percentage points with a pooled effect of 0.74 percentage points. Furthermore, we vary the timing and the content of the messages to test existing theories of text messages as mobilization tools. In one experiment, we find messages delivered before Election Day to have a higher effect than those delivered on Election Day, while we find no additional effect of delivering multiple messages. We also vary message content and in general find no significant differences from sending different messages.

Acknowledgements

An earlier version of the paper was presented at the American Political Science Association's Annual Meeting 2015, The Research Workshop in American Politics 2015 at UC, Berkeley, The Experiments and Natural Experiments Research Seminar 2015 at the University of Copenhagen, and at The 2nd Barcelona-Gothenburg Workshop on Experimental Political Science 2016. We thank the participants at all three events for useful suggestions. We also thank Donald P. Green and Melissa Michelson as well as the reviewers for reading and commenting on earlier versions of the manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Yosef Bhatti is a senior researcher at KORA - Danish Institute for Local and Regional Government Research. His research interests includes voter turnout and public reforms.

Jens Olav Dahlgaard is a PhD Candidate at the Department of Political Science at the University of Copenhagen.

Jonas Hedegaard Hansen is a PhD Candidate at the Department of Political Science at the University of Copenhagen. He researches political behavior with a focus on voter mobilization and political participation.

Kasper M. Hansen is a Professor at the Department of Political Science at the University of Copenhagen. Among his research interests are electoral turnout, voting behavior and opinion formation. www.kaspermhansen.eu

Notes

1 In comparison, the enrichment rates in Malhotra et al.’s (2011) two experiments are 5% (experiment 1) and 10% (experiment 2) (own calculations – number of citizens sample with valid cell phone number/registered voters). Dale and Strauss (2009) rely on opt-ins, which, if we consider all voters in some given area as potential subjects, probably implies an even lower enrichment rate.

2 The data for this study is stored on servers at Statistics Denmark. Due to security and privacy implications the data cannot be made available on the Internet. Researchers interested in replicating the findings are welcome to visit and work under supervision.

3 In the statistical tests throughout the paper, we use a 0.05 alpha level. When we compare treatment groups to the control groups, we use one-sided tests as we have a clear one-sided hypothesis regarding the positive general effect of receiving a text message on turnout, backed by prior evidence. We use two-sided tests when comparing different treatment groups. While we do have hypothesis regarding the relative strength of some of these, it is in most cases possible to make a hypothesis pointing in the other direction. We do not take into account in the tests that we make multiple inferences.

4 If we convert the effects to probits where we allow the probability of voting to be a non-linear function of some underlying function, they are slightly smaller than in Malhotra et al. (2011).

5 One might question the appropriateness of including experiment 2 in the meta-analysis due to its different study population, that is, an oversampling of non-Western immigrants and also with no age limit. However, since the electoral context, treatment and dependable variable still are quite similar (see also Gerber and Green 2012, 361–365), we find it acceptable to pool the experiments together.

6 We have examined heterogeneity based on age in our experiments by interacting a continuous age variable with the treatment indicators. We found three negative and one positive interaction coefficient and no significant effects. However, these tests are of course limited by the age restrictions in our samples, and there could be heterogeneity when looking at broader age spans.

Additional information

Funding

This work was supported by the Danish Council for Independent Research [grant number 12-124983]. The project has also received funding from the Danish Youth Council, The Ministry of Children, Equality, Integration & Social Affairs and the Danish Parliament.