Information Campaigns and (Under)Privileged Citizens: An Experiment on the Differential Effects of a Voting Advice Application

Abstract

Information campaigns are key elements of elections. Past research has established the importance of campaigns in informing and educating citizens, and ultimately strengthening participatory democracy. While the Internet has increased the possibilities to disseminate information campaigns and eased access to political information, it is still debated whether online campaigns are effective in stimulating political interest and participation among the general public. The issue is not only one of access, but also of use of information. The investigation of main effects of campaigns obscures the fact that citizens may not use information in the same way and reap the same political benefits. In this study, I examine the conditional effects of a new type of Web information campaign, Voting Advice Applications (VAAs), on the political engagement and electoral participation of citizens with varying levels of education. By investigating who benefits most from using these apps, I evaluate whether VAAs reinforce patterns of participation or mobilize new people in politics. Building on political behavior research, communication theory, and social psychology, I study the differential effects of VAAs with an innovative randomized field experiment design. The results confirm that VAAs can stimulate the political engagement of the public. However, there is no significant impact on electoral participation. In addition, the evidence shows that VAAs work differently for more or less educated citizens, and that the lower educated users benefit the most from VAAs as they become more interested in the election and more motivated to vote.

Keywords:

• education
• experiment
• political engagement
• voting
• Voting Advice Application

Supplemental Material

Supplemental data for this article can be accessed on the publisher’s website at http://dx.doi.org/10.1080/10584609.2017.1282560

Notes

^1. See the Appendix for images of the VAA used in the study: welcome page, issue questions, and results page.

^2. However, different concepts are used to define these abilities: education, political awareness, sophistication, interest (Krause, 1997; Luskin, 1990; Valentino, Hutchings, & Williams, 2004; Xenos & Moy, 2007; Zaller, 1992).

^3. While in many social science studies lower socioeconomic status individuals are underrepresented, this study is based on a sample that reflects the Canadian population in terms of education. In the study, 30% of participants have a high school degree or less, while 70% have some postsecondary education. In the Canadian population, 35.9% of the adult population has a high school diploma or less, while 64.1% have some post-secondary qualification (Statistics Canada, 2011). The neighborhood’s composition offered several advantages for the recruitment of Canadian citizens with lower levels of formal education as 92% of the residents are Canadian citizens and 52% of the local population has a high school degree or less (Statistics Canada, 2011).

^4. The full list of recruitment sites is presented in the Appendix.

^5. In line with the collaboration with the electoral management body of Quebec, to gain access to the official voting records for the participants of this study, all individuals had to be residents of the same electoral district.

^6. Deception was used to avoid any selective opt out due to the political nature of the research project. All participants were debriefed after the last survey, following the election.

^7. Randomization was embedded in the online survey program (i.e., there was no human involvement).

^8. See the Appendix for images of both websites: the Vote Compass and the Movie Quiz.

^9. However, there was a ratio of one research assistant per participant, to ensure that technical problems or computer proficiency would not prevent individuals from using the tablet or the Vote Compass.

^10. The retention rates for the second post survey are relatively similar across experimental conditions: 78.3% in the control group and 76.1% in the treatment group. The individuals who dropped out of the study after the experiment differed in some ways from the individuals who remained in the study (and took part in the last survey). On average, the individuals who dropped out after the second survey were younger, less likely to be registered to vote, less likely to read news about politics, had less interest in politics, paid less attention to the electoral campaign, and voted at a lower rate in the 2012 election, compared to those who participated in all steps of the study (these differences were significant at the p < 0.05 level). To evaluate the effect of this attrition on the results, I ran regressions modeling the short-term effects of the VAA with and then without the individuals who dropped out after the second survey. The size of the regression coefficients varies a little, but the sign of the coefficients and the significance levels remain the same, and so attrition is not believed to affect the results substantively.

^11. See the Appendix for question wording and the coding of measures.

^12. On the 316 participants with a valid name and complete home address, the electoral management body of Quebec was missing voting records for 24 persons. So the sample size for official voter turnout is N = 292.

^13. The education breakdown of the sample is as follows: 30% of the sample has a high school degree or less, while 70% has postsecondary education.

^14. Controlling for individuals’ baseline levels in the dependent variable (in the pretest) further helps to limit the impact of overreporting in surveys, because if one has a tendency to overreport political interest and behavior, they will likely do so in the different surveys. In this sense it is important to look at individuals’ relative change in political engagement and behavior, using the proposed strategy of change models. The use of deception (i.e., making this information activity about “various current events” and having survey questions on different current events) further limits social desirability, and thus the tendency of overreporting (Belli et al., 1999).

^15. Descriptive statistics for all variables are presented in the Appendix and show some imbalances between the control and treatment groups. Furthermore, a logistic regression model predicting assignment to the experimental conditions revealed that three variables were statistically significant in explaining allocation to the treatment and will thus be included in all regression analyses in order to control for the imbalances.

^16. The marginal effects of the Vote Compass are calculated for each education group using the command margins in Stata (following the estimation of the regression model including the interaction term between education and Vote Compass use), and are presented in Table A3 in the supplemental Appendix.

^17. Two other specifications of the education breakdown were tested: using non-university college education and university education as the cut-off points. When using different operationalizations of education, we note that the direction of the differential effect remains the same (i.e., for each political outcome, the lower educated consistently benefit most from the VAA, or alternatively the higher educated consistently benefit most), but the size of the VAA’s marginal effect and the level of statistical significance vary (as presented in Table A3 in the supplemental Appendix). For campaign attention and vote intentions, the lower educated group (using any of the three education specifications) always benefits most from the VAA, but the effect of the VAA is the largest and only statistically significant when individuals have a high school degree or less. This implies that lower educated individuals tend to benefit more from the VAA than higher educated individuals, in general, but this effect is specifically located among those with especially low levels of education (i.e., those with only secondary education or less).

Additional information

Notes on contributors

Valérie-Anne Mahéo

Valérie-Anne Mahéo is a postdoctoral researcher, Department of Political Science, Université de Montréal.