Do Online Ads Influence Vote Choice?

Abstract

Do online ads influence vote choice? We partner with a German party to evaluate the effectiveness of online ads using a cluster-randomized experiment. During the 2016 Berlin state election, 189 postal districts were randomly assigned to (a) emotional ads; (b) factual ads; or (c) no ads. Analyzing electoral results at the postal district level, we find that the overall campaign weakly increased the party’s vote share by 0.7 percentage points (p-value = 0.155). We also estimate a negative effect of the campaign on the vote share of the party’s main competitors of 1.4 percentage points (p-value = 0.094). Turning to the mechanism of persuasion, we find that the factual ads, if anything, fared slightly better than the emotional ads. Our evidence thus provides tentative support that online ads positively affect vote choice.

Keywords:

• field experiment
• online communication
• campaigns
• persuasion

Disclosure statement

No potential conflict of interest was reported by the author.

Supplemental material

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/10584609.2018.1548529.

Correction Statement

The author received valuable feedback from Donald Green, Alex Coppock, Macartan Humphreys, Florian Foos, Jens Olav Dahlgaard, Hanno Hilbig, Thomas Tichelbaecker and Ruth Dittlmann. The design for this experiment was pre-registered at EGAP (20160913AA). Please send correspondence to Anselm Hager at [email protected]. The data and code are available on the Harvard Dataverse at https://doi.org/10.7910/DVN/QOEBOCThis article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

1. Among the few empirical studies, Broockman and Green (2014) find weak evidence that voters are more likely to recognize candidates when randomly exposed to ads on Facebook. Relatedly, Collins, Kalla, and Keane (2014) randomly exposed voters to get-out-the-vote Facebook ads, and find null and negative effects, respectively. Only one study has linked online ads to administrative outcomes, namely, turnout. In particular, Bond and colleagues (2012) find that Facebook voting reminders do not increase turnout (see also Konitzer, Rothschild, Hill, & Wilbur, 2018).

2. This analysis was not preregistered.

3. Pride figures prominently among the emotions hypothesized to spur individuals’ willingness to process political information. Panagopoulos (2010), for instance, uses a large-scale field experiment to show that the activation of pride motivates compliance with voting norms as compared with a different emotion, namely, shame (see also Gerber, Green, & Larimer, 2010). A related literature in social psychology has presented evidence that pride motivates prosocial behavior (e.g., Williams & DeSteno, 2008). Finally, scholars of advertising have argued that the activation of pride is a powerful emotion in facilitating persuasion (Ambler, 2000).

4. Given that the online experiment also enables us to analyze clicks rates across the two treatments, we added a final third hypothesis: “The emotional campaign generates a higher click rate as compared with the factual campaign.” To save space, this hypothesis is discussed in Section A.8 in the online supplemental Appendix.

5. Facebook can offer more reliable and fine-grained targeting as its users frequently share their whereabouts. Google users, on the other hand, do not typically communicate comparable information, thereby making local-level targeting less reliable. In general, administering ads to postal districts poses a significant technological challenge to ad providers. Targeting mobile machines (i.e., phones or tablets) is typically less difficult, given that these devices submit GPS signals (e.g., when using apps). This enables providers to locate individuals’ likely home addresses. The case is more challenging for desktop computers. Here, Facebook and Google have to rely on IP addresses, which are more difficult to link to postal districts. Thus, unless ad providers can link phones or tablets to desktop computers—the most common and efficient technique—they have to rely on databases linking IP address ranges to postal districts. There can thus be no doubt that there was significant spillover across postal districts due to imperfect targeting by the ad providers. Unfortunately, neither Facebook nor Google offer measures that would allow one to assess the extent of the spillover. If anything, however, such fuzzy treatment assignment works to dampen potential treatment effects. Our effect sizes are thus likely a lower bound of the overall effectiveness. One potential, although imperfect, way to investigate possible spillovers is to assess whether the preregistered model (discussed later and shown in Table 1) systematically underestimates the party’s vote share in control districts that share borders with treatment districts. We conduct such an analysis in Section A.7 in the online supplemental Appendix. Indeed, we confirm a positive correlation between model residuals and the percentage of control districts’ shared borders with treatment districts. However, whether this spillover effect is due to fuzzy treatment assignment or due to social networks is unclear.

6. The exact allocation of funds is laid out in the preregistration document. Further details and key metrics of the online campaign are discussed in Section A.2 in the online supplemental Appendix.

7. The party requested that the online campaign also include videos. Importantly, as we detail in Section A.2 “Online campaign”, the videos shared elements of both the factual and emotional condition. The videos, therefore, do not compromise the causal identification of the two main treatment conditions; they may, however, have washed out treatment effect differences across the two conditions.

8. A manipulation check, discussed in detail later, shows that the images are a likely source for the variation between the two ads.

9. We believe that such an interpretation is rather unlikely, given that the subheading of the banner explicitly states “We provide the most funding to schools, teachers and school renovations.”

10. The reason for the reduction in effect sizes is the inclusion of the party’s historic vote share. This can be seen when analyzing added variable plots, provided in Figures A14 and A15 in the online supplemental Appendix. In addition, in Table A12 in the online supplemental Appendix, we show that results are similar (and insignificant) when combining the first and second vote to a standardized index.

11. Table 2 was not included in the pre-analysis plan.

12. Specifically, the priors for the aggregate treatment are $Treatmen{t}_{j=1[i]}-Treatmen{t}_{j=0[i]}\sim N(\mu ,{\sigma }^2),$ where $\mu \in [0,0.05]$ and ${\sigma }^2$ is equal to the standard deviation of the frequentist estimate.

13. While we cannot communicate the precise number, in Section A.6 in the online supplemental Appendix we report the overall campaign spending by all major parties during the election so as to situate the study in the broader political context.

14. In Table A7 in the online supplemental Appendix we show that the results are corroborated when using the second vote as the outcome. This analysis was not preregistered.

15. We should note, however, that Berlin has an extraordinarily high number of swing voters (see Section A.6 in the online supplemental Appendix for details).

Additional information

Notes on contributors

Anselm Hager

Anselm Hager is Assistant Professor of Political Economy, Department of Politics and Public Administration, University of Konstanz, 78457 Konstanz, Germany.