Do Online Ads Sway Voters? Understanding the Persuasiveness of Online Political Ads

ABSTRACT

This study investigates the effect of online political ads on party preference, and whether this effect is more pronounced for newer political parties and voters who are less politically knowledgeable and literate regarding online privacy. A mixed-method approach, combining Facebook browser tracking data and a four-wave panel survey, was adopted during the 2021 Dutch General Election campaign. The results showed that the number of political ads received from a specific party has a positive effect on both the propensity and choice to vote for that party. In addition, people with less political knowledge and online privacy literacy are more likely to be persuaded by online political ads. However, at the party level, there is no evidence indicating that the effect of political ads on party preference is stronger for new parties than for established parties. Overall, this study shows that voters can be persuaded via the frequency of exposure to online political ads, but the extent to which they are affected can vary.

KEYWORDS:

• Online political ads
• propensity to vote
• vote choice
• new party
• political knowledge
• online privacy literacy
• browser tracking
• panel survey

Recent years have seen a growing interest in the utilization of online ads in political campaigns (Coppock et al., 2022; Fowler et al., 2020; Jain & Wood, 2020). Political parties in many countries use social media, for instance, Facebook, to reach and target specific audiences (Bradshaw & Howard, 2019). The main goal of political advertising is to inform citizens about the diverse policy stances of different parties, to mobilize voters to cast a ballot, and most importantly, to persuade the public to vote for a specific party (Berridge, 2015; Lavigne, 2020). Whilst previous research has largely focused on the extent to which political ads mobilize citizens by studying general turnout (Franz et al., 2008; Haenschen, 2022; Krasno & Green, 2008; Llaudet, 2018), the persuasiveness, as reflected in changes in the likelihood to vote for a specific party and vote choice, has been understudied, especially in multi-party electoral systems. Online political advertising has mainly been studied in a US-centric and two-party electoral context (Haenschen & Jennings, 2019; Konitzer et al., 2019), while its deployment in multi-party democracies has remained largely unexplored. Understanding the country context is important because electoral systems saliently affect party characteristics and their capacity to practice online advertising strategies (Dobber et al., 2017; Zuiderveen Borgesius et al., 2018). Therefore, this study aims to investigate the persuasiveness of online political ads in a multi-party democracy where many parties compete for visibility online.

In addition, this study investigates the conditional impacts from two different perspectives. Valkenburg and Peter (2013) posit that social contexts can moderate media responses in both interpersonal and institutional contexts, but previous studies have focused on the moderating role of either merely individual-level factors (Levy et al., 2021; Liu, 2019) or merely contextual-level factor (Haenschen & Jennings, 2019). This study examines how the effects of online political advertising differ among parties and individuals simultaneously. On the one hand, we ask: which parties benefit more from online advertising? Party features such as party age are essential factors when launching online political advertising. Long-standing parties tend to hold a position of dominance in terms of party visibility and political viewpoint displayed in mainstream media, while new parties are perceived to be unimpressive and less resourceful although this disadvantage may be overcome in an online environment (Dobber et al., 2017; Gibson & McAllister, 2015). Accordingly, this study examines whether the persuasiveness of online political ads on party preferences is different between new parties and established parties. On the other hand, we ask: which citizens are more influenced by online political ads? Citizens with less knowledge about politics and those who are less literate regarding their privacy online might be more affected by political ads (Hersh & Schaffner, 2013; Zuiderveen Borgesius et al., 2018), which subsequently influences their behavioral responses. Hence, this study also examines the conditional role of political knowledge and privacy-related online literacy.

This study applies an innovative mixed-method approach during the 2021 Dutch General Election to fully understand the interplay between online political advertising and voters. Instead of measuring online media exposure with self-reported data heavily dependent on people’s ability to recall, we provide a promising approach to observing exposure to ads within online environments. By combining browser tracking data and a four-wave panel survey, we passively captured the number and source of political ads on the desktop-version Facebook, and measured outcome variables and moderators. It should be emphasized that this study focused on online political ad exposure on desktops, and mobile exposure to political ads was not taken into account. Besides, the 2021 Dutch Election, as an exemplary case of a multi-party electoral system, had 37 parties in the race for seats in the House of Representatives, which allowed us to investigate how new parties compete with long-standing parties in elections. With the application of our data, we can extend the existing research to a multi-party electoral context.

Utilization of Online Political Advertising

Online resources have been extensively utilized in political campaigns. Compared to traditional advertising, online political advertising is a relatively flexible and low-cost endeavor with a wide audience reach (Sances, 2019). Most importantly, it is entitled to the power of targeting (Fowler et al., 2021). Social media tend to segment audiences by their demographics, partisanship, interest, and even psychometrics, based on which audiences are exposed to individually geared election promises (Phillips et al., 2010; Russmann, 2020; Sances, 2019). Such hyper-intensive campaign practices are reckoned to help educate the electorate and accelerate turnout, but concerns linger about voter manipulation. Evidence showed that political parties actually do not have the capacity to execute accurate targeting strategies (Kefford et al., 2022), and thus studies should be conducted to support the effectiveness of online campaigning.

In the meanwhile, there is limited empirical research on the multi-party context. Before we move to the literature available on the (conditionality of the) impact of political ads, we first argue that more attention should be paid to the impact of ads in a multi-party system. The reason is fourfold, and each reason, respectively, leads to a key variable in this study. First, as there is less at stake in elections, multi-party democracies usually have a less entrenched political divide and a higher turnout (Dobber et al., 2019), which could affect the effectiveness of political campaigns. Second, compared to two-party systems, the size of parties can play a crucial role in deploying online advertising in multi-party systems, as longstanding and major parties tend to have more resources and a higher budget to boost their visibility (Gibson & McAllister, 2015). Third, political knowledge in multi-party systems is of greater importance as voters are often overloaded with information (Galston, 2001). A higher level of knowledge helps voters to locate useful information from diverse viewpoints in the marketplace of ideas. Fourth, a number of multi-party democracies in Europe have stricter data protection laws impacting how voters perceive ads (higher privacy concerns might make them less receptive), and this in turn might determine how they are affected by it (Zuiderveen Borgesius et al., 2018). In the following sections, we will elaborate on the aforementioned four aspects (i.e., voting behavior, party feature, political knowledge, and digital privacy literacy), while taking into account the focus on multi-party electoral systems.

Persuasiveness of Online Political Ads

The key purpose for political parties to purchase online ads is to increase the preference to vote for a particular party. Prior studies have investigated the effect of advertising on party preference, but we know little about this effect in an online environment. In election studies, party preference is defined as the “ratings for each of the parties of a political system” (Van der Eijk & Marsh, 2007, p. 2). Political parties’ attempt to gain higher party preference can be achieved through campaigning (Lavigne, 2020). As we discussed earlier, online political advertising has accounted for a considerable proportion of campaigning activities. The deployment of online ads has focused on two aspects: quality and quantity. Previous studies have largely focused on the former aspect, such as negativity (Franz et al., 2008), framing appeals (Burge et al., 2020), rhetorical characteristics (Savoy & Wehren, 2021), and the type of targeted information (Konitzer et al., 2019; Lavigne, 2020; Zarouali et al., 2020). However, few studies have investigated the effect of the frequency of exposure to political ads online. Studying the number of political ads voters encounter online is important because many parties tend to invest a great amount of money in deploying more online ads as well as targeting citizens with repeated exposure (Shen et al., 2011).

On the one hand, online political advertising allows political parties to reach out to citizens and stay visible, especially in a hybrid media system, and thus provide voters with information to perceive and evaluate the parties or the candidates. Visibility is key in multi-party systems. Previous research has found that the more visible a party is, the more likely voters will vote for this party (Hopmann et al., 2010). Such visibility can be measured in terms of the presence and magnitude of online advertising. The power of social media is to quickly and widely spread online advertising among audiences, and thus can boost the salience and visibility of the parties. Moreover, the targeting feature of online advertising helps parties to gain more visibility because it provides a better chance to reach the targeted voters (Thorson et al., 2019).

On the other hand, online political advertising also evokes short-term affection. Farha and Hamdar (2008) point out that positive attraction to the public could be “a function of interpersonal attraction rather than an ideological-based evaluation” (p.5). Hence, instead of forming an attitude toward a party based on a long-term ideological stance, a high frequency of exposure to advertising also helps to develop short-term attraction toward a party. Mere Exposure Theory can be used to explain this effect, such that repeated exposure to an ad regarding a certain party could enhance people’s positive affect for that party (Fowles, 1992; Zajonc, 1968), especially in an online environment where citizens are increasingly encountering political ads while browsing through their social media feed multiple times a day. Therefore, citizens that are more frequently exposed to online political ads from a certain party could develop a positive affect toward the party, which can be reflected in their party preference.

Moreover, political ads on social media platforms are often featured with targeting techniques, for instance, the lookalike audiences on Facebook (Cano-Orón et al., 2021; Dommett, 2019). This feature allows political parties to purchase inferred and modeled user data, and enables them to select the most susceptible segment of audiences to disseminate geared campaigning messages. As a few studies have found positive effects of targeting on party partisanship and vote behavior (Endres, 2020; Krotzek, 2019; Lavigne, 2020), we argue that social media platforms, presumably equipped with targeting features, help to enhance the persuasiveness of political ads.

In sum, political parties in multi-party settings increase their visibility to the public via a higher volume of online political ads. Hence, voters are more informed about the parties with higher visibility, and they also tend to develop a more positive evaluation toward the parties they encounter the most during the election campaigning stage. We thus formulate the following hypothesis:

H1:

The more online political ads people receive from a party, the higher the party preference people have for that party.

Are New Parties the Big Winners?

As online political ads are reckoned to be one of the most effective campaigning tools to persuade voters, it seems to be even more tempting for newer parties to purchase ads on social media platforms such as Facebook. In multi-party democracies, it is difficult for new parties to make an impressive entrance because the media normally pay very little attention to the newcomers compared to the longstanding ones (Van Spanje & Azrout, 2020). Due to a relatively low budget, new parties are often unable to invest in massive campaigning events (Zuiderveen Borgesius et al., 2018). The pattern where established and major political parties dominate the market of visibility is often labeled as “normalization” (Gibson & McAllister, 2015). However, the unbalanced party competition is alleged to be adjusted and equalized in the online environment. Social media provide new parties with a relatively cheaper and less risky means of promoting themselves to likely supporters, and thus improve visibility and recognition (Small & Giasson, 2020). Via online ads, voters can learn more about the policy stances of the new parties, so are more likely to cast a ballot if the legitimacy and effectiveness of the newcomers are well perceived. The scenario where the new parties are entitled to equal visibility is reckoned as “equalization” (Gibson & McAllister, 2015).

Although the marginalization of new parties can be remedied using online advertising, new parties are still restricted to a very limited audience reach due to a shortage of budget and resources compared to established parties (Zuiderveen Borgesius et al., 2018). Still, previous studies have found that political advertising on social media is more effective for new parties than for established parties. Gibson and McAllister (2015) examined Australian surveys of election candidates and found that although well-established parties have superiority regarding the usage of online resources, minor parties tend to use social media to successfully gain votes. We expect that new parties are more likely to gain from advertising online and suggest that there is a ceiling effect. As voters’ familiarity with and attitudes toward long-lasting parties are preexisted and often saturated, the extent to which voters change their attitudes toward certain parties can be limited. Thus, the persuasiveness of online advertising on party preference might be more constrained for established parties. In contrast, voters often do not have enough awareness and understanding about the standpoints of parties that were newly founded, so online ads help to deliver a great amount of new information regarding the new parties. Thus, there is more room for voters to place their assessment toward the newcomers, and thus the effect of online ads is stronger for newer parties. We hypothesize the following.

H2:

The positive effect of online political ads on party preference is stronger for new parties compared to established parties.

Too Knowledgeable to Be Persuaded?

Aside from party-level features, the impact of political ads is not the same for everyone. This section focuses on two individual characteristics that explain why some citizens are more affected by political ads than others: political knowledge and online privacy literacy.

Political knowledge is the most important moderator in the process of perceiving and evaluating politically persuasive messages (Cappella & Jamieson, 1997). In this study, we define political knowledge, also known to be part of political sophistication, as the factually correct information about politics an individual can recall from the conscious memory (Carpini & Keeter, 1996). Political knowledge is an important individual characteristic in a democratic society to keep citizens politically involved (Van Erkel & Van Aelst, 2021), because voters with less voting experience and political knowledge may be deterred from participation when encountering difficulties in understanding the complex voting procedure (Dermont & Stadelmann-Steffen, 2018). It is often assumed that voters should have at least some basic political knowledge to “make a full range of reasoned civic judgments” (Galston, 2001, p. 218). Previous studies have found that less knowledgeable individuals are more likely to be persuaded. Less knowledgeable citizens are often unfamiliar with the fundamental political standpoints and institutional norms, thus when they encounter a persuasive message, they have no background knowledge to refer back to (Zaller, 1992). Accordingly, citizens with lower levels of political knowledge are less able to make well-informed decisions, and thus they are more likely to be persuaded. Work conducted by Lecheler and de Vreese (2012) confirmed this argument, where they found that citizens with lower levels of political knowledge are most susceptible to persuasive messages. Therefore, the persuasiveness of online political ads on party preference is contingent on voters’ political knowledge, in the sense that politically knowledgeable individuals are less likely to be persuaded by online ads, and hence score lower in vote propensity and vote choice. The hypothesis is derived as follows:

H3:

The higher an individual’s political knowledge, the weaker the effect of online political ads on party preference is.

Perceived behavioral control could play an important role in shaping an individual’s behavioral intention (Ajzen, 1991). Given the online setting, we expect that the perceived ability to engage in personal data protection is another important characteristic that influences the effects of online political advertising. Online privacy literacy refers to users’ knowledge about technical and institutional aspects of online data protection, as well as their ability to practice data protection strategies online (Boerman et al., 2017; Trepte et al., 2015). We believe that people who are more literate about their privacy are also more skeptical when receiving (personalized) ads. We expect that this works in the following way: When individuals have more insights into the advertising strategies that political parties use, they are expected to be more aware of the tactics behind the ads (i.e., the fact that personalized data is used for targeting purposes). In other words, they might be more knowledgeable and aware of the fact that the ad tries to persuade them. As a consequence, they are likely to critically process the ad and are more likely to resist the persuasion attempt of the ad. Thus, as a result, individuals that are more privacy literate, are less likely to be influenced by political ads, because they are more likely to resist the persuasion attempt. The findings on the effect of media or privacy literacy on online content evaluation are mixed. A survey study conducted by Craft et al. (2017) found that greater media privacy affects conspiracy theory endorsement in a negative manner, meaning that individuals with a higher level of media literacy are apt to trust online information. However, others found the opposite. They argue that individuals who have higher levels of online privacy literacy tend to generate distrust and skepticism toward advertising (Ashley et al., 2017; Potter, 2010), and thus “navigate their information environment in a mindful and critical way” (Van der Meer & Hameleers, 2020, p. 4). In other words, privacy literacy positively predicts skepticism toward the information on social media (Vraga & Tully, 2021). While the effects of literacy have been often studied, the conditional role of literacy has been rarely examined. Based on the discussion above and in light of the current literature, we assume that, during the election period, voters with higher levels of online privacy literacy are apt to recognize, interpret, and evaluate online ads with careful and more critical thoughts, and thus they are more likely to resist being persuaded by the ads. Therefore, we assume the following hypothesis.

H4:

The higher an individual’s online privacy literacy, the weaker the effect of online political ads on party preference is.

Methods

Background of the 2021 Dutch General Election

This study was conducted around the 2021 Dutch General Election, which took place on March 17, 2021. The Netherlands is an exemplary case of multi-party electoral systems: it carries a nature of proportional representation, and governments are formed by multi-party coalition. It has been over a century since a party ever won enough seats to win an outright majority in the House of Representatives. Specifically, the 2021 Dutch Election is an extreme example: 89 parties registered with the Electoral Council; 37 parties achieved ballot access and participated in the election, and 17 parties got elected into parliament; the final turnout was 78.7%.

Regarding political ads online, the Netherlands has some unique features. First, we expect that political advertising online is popular, because the Netherlands is one of the countries with the highest Internet use in the world (Wittenborg, 2018). In 2020, 97% of Dutch households had Internet access (Statista, 2020), and due to the COVID-19 pandemic, almost all campaigning events were held online. Second, in February 2021, 11 Dutch political parties and four online platforms (i.e., Facebook, Google, TikTok, and Snapchat) voluntarily signed a Code of Conduct for online political ads, which aims to avoid the spreading of misinformation, as well as refraining from psychological profiling when performing targeting campaigning (IDEA, 2021). Third, party financing in the Netherlands is lower than that of many other countries (Vliegenthart & Kruikemeier, 2017; Zuiderveen Borgesius et al., 2018). During the 2021 Dutch General Election, all the Dutch political parties collectively spent around 2.5 million euros on Facebook ads and 1 million euros on Google ads (Votta, 2021), meaning that Facebook ads accounted for a large proportion in the online campaigning.

Sample

The browser tracking study and panel survey took place between January 10 and March 28, 2021 (see Figure A1 in Appendix A for phases of data collection and pre-processing). Data collection was approved by the Ethics Review Board of The University of Amsterdam (project filed as 2021-PCJ-13104). Participants were recruited via a well-trusted audience research organization I&O. Participants were selected based on a screening question of whether they use a desktop version of Facebook and would be willing to install the plug-in. To participate in the study, participants were asked to read and sign the informed consent form. The panel survey was conducted in four waves, respectively, on January 18, February 16, March 4 (prior to the Dutch Election) and March 18 (in the aftermath of the Dutch Election). A total of 781 participants participated in our survey, among which 659 completed all four waves. In the meanwhile, 596 participants managed to successfully install and enable a tracking extension on their laptops or computers. The extension was developed by the Algorithms Exposed (ALEX) team at the Department of Media Studies and the Amsterdam School of Communication Research (ASCOR). After removing participants who only actively used the browser extension for one day (N = 75) or two consecutive days (N = 8), the final number of active participants was reduced to 513.

In the end, 505 participants both activated the browser plug-in and participated in the panel survey study (39.6% was female and 60.4% male, 16.6% under 34 years, 51.3% between 35 and 64 years, and 32.1% above 65 years). In addition, 63.4% held a bachelor’s degree or higher. 88.1% was Dutch, 7.3% western immigrants, and 2.0% non-western immigrants. According to a report on Facebook demographics in the Netherlands, 52.8% of the Facebook users in the Netherlands is female, 47.2% male; 45.8% under 34 years, 43.9% between 35 and 64 years, and 10.2% above 65 years (Facebook users in Netherlands 2021, 2021). During the four waves, participants received 8,703 political ads from Dutch political parties and politicians on Facebook.

Data Collection and Procedure

The browser extension passively recorded all kinds of ad impressions (including the ad source, ad text, time and date, associated links, and images) on participants’ Facebook feeds, which was subsequently sent to a secured non-SQL Mongo Database for data storage. We called this set of inclusive data the “metadata”. We then compared the ad sources with political ad publishers in the Facebook Ad Library, which gave us a list of Dutch political ad publishers on Facebook. We manually recognized and categorized the ad publishers,¹ and finally narrowed the metadata down to only political ads by filtering out ads from a nonpolitical publisher. Using browser tracking data, the real-time exposure to online political advertising could be grasped. Panel survey was conducted to capture participants’ characteristics as well as measure users’ attitudinal and behavioral responses in each wave. The order of measures was: vote propensity (wave 1, 2, and 3), vote choice (wave 1, 2, 3, and 4), political knowledge (wave 2), digital literacy (wave 2), media use (all the waves), and demographics (wave 1). By weaving together browser tracking and survey data, we are able to match the number of received political ads, party features, personal characteristics, and the consequent party preference.

Measures

Number of Received Ads

We treated the number of received ad impressions a respondent received from a specific political party between two waves as the independent variable (M = 0.17, SD = 1.33). An ad impression refers to the occasion when an ad is shown on the site. As the browser extension allowed us to capture the source and time point of each ad impression, the total of ad exposures from each political party within each wave was calculated for each respondent.

Party Preference

Previous studies have shown that party preference can be reviewed either in an ipsative or in a non-ipsative (or normative) scale. An ipsative scale showcases the real scenario and forces the voters to determine one party to vote for, so is useful when evaluating within-individual responses (Baron, 1996). In contrast, normative measures require voters to rate their feelings or support toward a specific party, reflecting the distribution for all scales, so can be used to assess between-individual responses (Baron, 1996). Although there are three popular non-ipsative definitions of party preference, known as the thermometer ratings, liking scores, and vote propensity, only vote propensity captures the actual voting behavior in multi-party systems (Bartle & Bellucci, 2014; Van der Eijk & Marsh, 2007; Van der Eijk & Niemoller, 1983).

Vote propensity, or propensity-to-vote (PTV), describes the voting likelihood for a party, and has been long used to examine party preference patterns in multi-party electoral scenarios (Van der Eijk & Niemoller, 1984). Given our focus on multi-party democracies, we argue that the impact of online political ads on party preference should be studied with a focus on both the vote choice and propensity to vote. Consequently, participants’ propensity to vote (M = 2.26, SD = 3.08) was measured across the first three waves of the panel survey. With a measurement employed in the European Election Studies (Schmitt et al., 2015), participants were asked to indicate how likely it is that they will vote for a political party in the General Election on a 12-point scale (0=I will never vote for this party, 11=I will definitely vote for this party). A list of 18 Dutch political parties was provided. This list consists of 13 parties that won seats during the 2017 Dutch General Election, and five new parties founded after the last election (see Table B1 in Appendix B).

Vote choice (M = 0.04, SD = 0.20) was measured across all four waves of the panel survey. Within a wave, the average possibility for a participant to vote for a particular party is 4%. For the first three waves, participants were asked to indicate which party they would vote for if there were elections tomorrow. The actual vote choice was measured in the last wave, where participants were asked to indicate which party they finally voted for. Participants were asked to choose only one party from the list of Dutch political parties, and could also name their preferred party if it was not listed.

Moderators

We define new parties as parties established after the previous election. New parties are categorized with a pre-defined list which includes seven Dutch political parties JA21, BIJ1, Code Oranje, Lijst Henk Krol, Volt, Splinter, and BoerBurgerBeweging. These parties were only established after the 2017 Dutch General Election, with JA21, Lijst Henk Krol and Splinter founded less than six months before the 2021 Dutch Election.

To measure political knowledge, participants were asked to select the correct answer to three questions regarding the Dutch politics (see Table C1 in Appendix C). A correct answer was coded as 1, an incorrect answer was coded as 0. A sum score was calculated to measure political knowledge, a higher score indicating a higher level of political knowledge. 428 participants answered the first question correctly, 387 participants answered the second question correctly, and 332 participants answered the third question correctly. As the knowledge items are dichotomous, Mokken scale analysis (Van Schuur, 2003) was performed to test the reliability among items (Loevinger’s coefficient H = .35; M = 2.43, SD = 0.79).

For online privacy literacy, according to social privacy literacy, self-perceived skill to regulate privacy settings online was measured (Bartsch & Dienlin, 2016). Participants were asked to what extent they agree with the statements on a seven-point scale (1=strongly disagree, 7= strongly agree). Four items, i.e., “I know how to operate Facebook,” “I can make an informed decision about what information to disclose online,” etc., were used to measure their self-perceived online privacy literacy (see Table C1 in Appendix C). A mean score was calculated (Cronbach’s α = .65; M = 5.01, SD = 1.02).

Control Variables

First, as we conducted a four-wave panel survey, previous attitude and behavior influence the current attitudinal and behavioral consequences. Therefore, lagged dependent variables (i.e., vote propensity and vote choice from the previous wave) were controlled for. Second, at election times, voters do not only get information about politics from Facebook, but also from other sources such as other online platforms and offline media (Zuiderveen Borgesius et al., 2018). Therefore, in each survey, we also controlled for online media use (Cronbach’s α = .51; M = 2.28, SD = 0.94), newspaper use (including newspaper websites; Cronbach’s α = .27; M = 1.63, SD = 0.73, and TV news channel use (Cronbach’s α = .73; M = 1.95, SD = 0.88) (see Table C1 in Appendix C). Third, participants’ personal information, such as age, gender, educational level and income, were included as control variables as our sample was not identical to the Facebook demographics. In addition, the wave number was included as a control variable to control for order effects.

Results

Descriptive Statistics

Previous research found that the number of Facebook political ads ramped up toward the end of the election campaign (Ridout et al., 2021). Figure E1 shows the evolution of the proportion of political ads among ads on Facebook. We can see that the closer it got to election day, the larger proportion political ads accounted for, and it reached its peak at 13.82% on March 16, 2021, which was one day before the Dutch Election. Similar evidence was found with the chronological change in the proportion of participants who received political ads among those who received all kinds of ads on Facebook. More participants started to receive political ads leading up to election day, and it reached a peak at 62.67% on March 16, 2021. These findings are in line with evidence from previous studies.

Furthermore, we calculated the quantity differences of the vote choice among our participants for each party between wave 1 and wave 4. Figure E2 shows the change in vote choice from wave 1 to wave 4. As seen in the figure, longstanding parties, especially parties that formed the previous coalition (i.e., VVD, CDA, Christian Union, etc.), were among the parties that lost the most votes during the election campaign period. In comparison, most parties that gained votes after four waves were new parties (i.e., Volt, JA21, BIJ1, etc.).

Data Analysis

Our data resembles a panel structure, which means that four waves were nested within each respondent.² More precisely, the data structure was reconstructed in such a way that the unit of analysis is the combination of user, wave and political party. In other words, each row in our analysis data set refers to a respondent’s received political ads from a specific party in a specific wave, as well as the respondent’s reported dependent variables, moderators and control variables.

To determine the analytical approach, we first calculated the intra-class correlation (ICC) with three null-models for all the independent and dependent variables. We found that ICC for the number of received political ad impressions, vote propensity, and vote choice are, respectively, .10, .05, and .00, which means that the majority of the variance can be explained by situational factors, while only a small fraction of variation is accounted for by between-respondent variation. Given that all the ICC scores are below .10, we performed regression models with clustered standard errors to analyze the causal effect between vote responses and received political ads, as well as the moderating effect of party age, political knowledge, and online privacy literacy. Specifically, ordinary least squared (OLS) regression models were performed to test the effect on the propensity to vote from wave 1 to wave 3 (see Table 1) and logistic regression models were performed to test the effect on vote choice from wave 1 to wave 4 (see Table 2). For each dependent variable, we performed five regression models, where clustered standard errors per respondent were included. For all five models, the independent variable remained the number of received online political ads, while we controlled for lagged dependent variables (propensity to vote and vote choice from the previous wave), three different types of media use, and participants’ demographic information. Across the five models, the moderating variables were entered in the regression model in a stepwise manner. Appendix D reports the bivariate correlations among variables. For robustness check, we also conducted multilevel regression analyses (see Appendix F) and fixed effects models (see Appendix G).

Table 1. Linear regressions for effects on the propensity to vote.

	Model 1	Model 2	Model 3	Model 4	Model 5
	B(Robust SE)	B(Robust SE)	B(Robust SE)	B(Robust SE)	B(Robust SE)
Constant	0.170 (0.153)	0.232 (0.157)	0.121 (0.157)	0.272 (0.189)	0.284 (0.198)
Ad count	0.049*** (0.015)	0.044*** (0.015)	0.112** (0.055)	0.133** (0.066)	0.201*** (0.073)
New party		−0.175*** (0.035)			−0.175*** (0.035)
Political knowledge			0.039 (0.027)		0.036 (0.027)
Online literacy				−0.021 (0.019)	−0.020 (0.019)
Interactions
Ad count × New party		0.096 (0.083)			0.102 (0.078)
Ad count × Knowledge			−0.024 (0.019)		−0.024 (0.018)
Ad count × Literacy				−0.016 (0.013)	−0.017 (0.012)
Controls
Previous propensity	0.860*** (0.012)	0.853*** (0.012)	0.860*** (0.012)	0.859*** (0.012)	0.852*** (0.012)
Online media use	0.020 (0.023)	0.019 (0.023)	0.018 (0.023)	0.024 (0.023)	0.021 (0.023)
Newspaper use	−0.006 (0.027)	−0.006 (0.027)	−0.010 (0.027)	−0.007 (0.027)	−0.011 (0.027)
TV use	0.002 (0.025)	0.003 (0.025)	0.0004 (0.025)	0.002 (0.025)	0.002 (0.025)
Age	0.019 (0.019)	0.017 (0.019)	0.015 (0.019)	0.018 (0.020)	0.012 (0.019)
Gender	0.082** (0.040)	0.081** (0.040)	0.093** (0.041)	0.076* (0.040)	0.087** (0.041)
Education	−0.024 (0.032)	−0.024 (0.032)	−0.029 (0.032)	−0.024 (0.032)	−0.029 (0.032)
Income	0.012 (0.015)	0.012 (0.015)	0.010 (0.015)	0.012 (0.015)	0.011 (0.015)
Wave	0.036 (0.064)	0.037 (0.063)	0.034 (0.064)	0.036 (0.064)	0.035 (0.063)
N of observations	13158	13158	13158	13158	13158
N of participants	410	410	410	410	410
R-squared	.745	.746	.745	.745	.746

*p < .1; **p < .05; ***p < .01.

Table 2. Logistic regressions for effects on vote choice.

	Model 1	Model 2	Model 3	Model 4	Model 5
	OR(Robust SE)	OR(Robust SE)	OR(Robust SE)	OR(Robust SE)	OR(Robust SE)
Constant	0.008*** (0.001)	0.010*** (0.001)	0.008*** (0.001)	0.008*** (0.001)	0.010*** (0.001)
Ad count	1.084*** (0.015)	1.079*** (0.016)	1.198*** (0.036)	1.331*** (0.104)	1.460*** (0.122)
New party		0.427*** (0.061)			0.429*** (0.061)
Political knowledge			0.960 (0.026)		0.965 (0.028)
Online literacy				1.001 (0.015)	1.003 (0.016)
Interactions
Ad count × New party		1.000 (0.026)			1.015 (0.028)
Ad count × Knowledge			0.964*** (0.011)		0.965*** (0.010)
Ad count × Literacy				0.962*** (0.014)	0.962*** (0.012)
Controls
Previous vote choice	382.785*** (62.090)	327.090*** (53.356)	384.312*** (62.336)	382.606*** (62.134)	328.915*** (53.736)
Online media use	1.007 (0.015)	1.006 (0.015)	1.013 (0.015)	1.010 (0.016)	1.014 (0.016)
Newspaper use	1.000 (0.020)	1.002 (0.021)	1.006 (0.020)	0.996 (0.020)	1.002 (0.020)
TV use	1.009 (0.015)	1.011 (0.016)	1.017 (0.015)	1.014 (0.014)	1.023 (0.015)
Age	0.972** (0.013)	0.965** (0.014)	0.978* (0.013)	0.971** (0.014)	0.969** (0.014)
Gender	0.998 (0.027)	0.987 (0.028)	0.981 (0.030)	0.983 (0.028)	0.960 (0.032)
Education	1.003 (0.020)	1.011 (0.021)	1.014 (0.021)	0.999 (0.021)	1.017 (0.022)
Income	1.027** (0.011)	1.023** (0.012)	1.034*** (0.011)	1.028*** (0.011)	1.031*** (0.012)
Wave	1.062** (0.027)	1.076*** (0.028)	1.062** (0.027)	1.059** (0.027)	1.074*** (0.028)
N of observations	24838	24838	24838	24838	24838
N of participants	410	410	410	410	410
Pseudo R-squared	.576	.580	.576	.576	.581

*p < .1; **p < .05; ***p < .01.

Main Effect of Political Ads on Party Preference

First, we tested the main effect of receiving online political ads on individuals’ propensity to vote and vote choice. We found that the number of received online political ads has a positive effect on both the propensity to vote (B = 0.049, p = .001, F(10,409) = 833.69, R² = .745) and vote choice (OR = 1.084, p < .001, Pseudo R² = .576). Our findings demonstrate that receiving one more political ad from a specific party yields 8.4% higher odds of casting a vote for the party, supporting our first hypothesis.

Conditional Impact of New Parties

At the party-level, our second hypothesis presumes a moderating effect of new party, such that the positive effect of online political ads on party preference is stronger for newer parties compared to established parties. However, we did not find support for the proposed conditional impact of new parties on the effect of the number of received political ads on vote propensity (B = 0.096, p = .249, F(12,409) = 900.86, R² = .746) and vote choice (OR = 1.000, p = .992, Pseudo R² = .580). Therefore, we do not find support for our second hypothesis.

Conditional Impact of Knowledge and Literacy

At the individual-level, we examined the moderating role of political knowledge. As we hypothesized (see H3), citizens’ political knowledge weakens, to some extent, the effect of online political advertising on party preference. While there were no significant differences among the moderating effects of political knowledge on the propensity to vote (B = −0.024, p = .216, F(12,409) = 704.65, R² = .745), the interaction effect between political knowledge and the number of received political ads was significant on vote choice (OR = 0.964, p = .001, Pseudo R² = .576). Figure 1 shows the interaction effect of political knowledge on the effect of ad exposures on vote choice. When receiving no online political ads from a party, the predicted likelihood of voting for this party is 0.8%. When receiving more ads from a specific party, the chance for less knowledgeable people to vote for that party is higher than that for more knowledgeable people. However, the effect size of ad exposure on vote choice is small, in the sense that receiving five ads from a specific party leads to no more than 5% of the increase in the likelihood of voting for that party regardless of the level of knowledge. Thus, hypothesis three is partially supported.

Figure 1. Moderating effect of personal characteristics political knowledge and online privacy literacy.

Next, hypothesis four proposes a conditional impact of online privacy literacy on the effect of online political advertising on vote propensity and vote choice. We found, similar to political knowledge, no significant moderating effect of online literacy on the relationship between vote propensity and online political ads (B = −0.016, p = .216, F(12,409) = 733.36, R² = .745), but we found a significant moderating effect on vote choice (OR = 0.962, p = .008, Pseudo R² = .576). Figure 1 shows that when receiving no online political ads, the predicted vote choice for people with different levels of online privacy literacy is 0.008. When receiving more ads from a certain party, our model predicts that less literate people are more likely to vote for the party than more literate people. A small effect size is also observed where the likelihood of voting for the promoted party only increases by no more than 5% when receiving five political ads. Therefore, hypothesis four is partially supported.

In the last step of the analysis, we explored the joint impact of all the interactions. The moderating effect of both individual-level moderators, political knowledge and online privacy literacy, remained similar results as models including each one of the interaction terms.

Discussion

Although online advertising has gained traction in political campaigns in recent years, it remains unclear whether online political advertising, especially in multi-party systems, is indeed a campaign strategy as effective as alleged. The main purpose of this study was to investigate the impact of receiving online political ads on individuals’ party preferences in a multiparty system where voters are possibly targeted by many different political parties. Overall, our results indicate that the number of online political ads has a positive, but small, impact on party preference, which confirmed previous expectations in terms of offline advertising effects (Hopmann et al., 2010; Pfau et al., 2002). We further expected that the persuasiveness of online political ads depends on party features and individual characteristics. Regarding party-level factors, we did not find support for our expectation that new parties, compared to established parties, benefit more from online political ads. This finding indicates that new parties may not profit from online ads, as it has been observed that successful new parties do not necessarily use social media (Vergeer, 2015). Having said that, we did observe that the number of released online political ads from new parties is sparse compared to that of established parties (see Table B1 in Appendix B). It means that participants in this study were rarely exposed to political ads from new parties, which made it less likely for them to be affected by these ads, and this leads to issues related to the lack of power. Besides, the quality of ads can play an important role in the effect on party preference (Galasso et al., 2021; Kendall et al., 2015). A higher level of budgets and resources supplied to the established parties helps to generate better ad content as well as higher preciseness of the targeted audience (Margolis et al., 2003; Zuiderveen Borgesius et al., 2018). In future research, the effect of new parties can be reevaluated either in a real-life setting where extensive investigations can be conducted focusing on these parties, or in laboratory experiments where the ad quality is controlled.

Individual characteristics do seem to matter: voters with a higher level of political knowledge and online privacy literacy are less likely to be persuaded by online political ads. This conditional influence was found only for vote choice, but not for the propensity to vote. While the determinants of these two variables are compatible, they still differ in terms of within-individual stability. Vote choice is more volatile than the propensity to vote, as the former describes a behavioral voting consequence, while the latter indicates an attitudinal outcome (Van der Eijk & Franklin, 1996). It is likely that voters hold relatively stable vote propensities to a certain group of parties with similar issue agenda or within a certain ideology, and meanwhile the update of vote choice can be volatile depending on short-term factors (Paparo & Sio, 2017), such as political ads online. Therefore, compared to citizens with higher levels of political knowledge and privacy literacy, less knowledgeable and literate citizens are more likely to alter their vote choices due to more exposure to online ads, but there is no difference in the extent to which the attitude toward certain parties is affected.

Theoretical and Methodological Implications

This study has three important implications. First, the persuasiveness of online political ads has been understudied, as previous research studying the US-centric two-party systems has mainly focused on the mobilizing function (Haenschen & Jennings, 2019; Krasno & Green, 2008; Llaudet, 2018). In real-life settings, two aspects of political ads are essential to study the persuasiveness of ad exposure: the number and source of received ads. To cover these two aspects, previous research has largely relied upon participants’ ability to recall by asking questions such as: “How many political ads have you encountered in the last week?” (Thorson et al., 2019), or “Have you been contacted by this party?” (Lavigne, 2020). These self-reported measures of ad exposure could be problematic with potential reliability and validity issues. The browser tracking method offers a promising means to passively and precisely capture the number and source of online political ads. Hence, this study specifically filled the theoretical and methodological gap regarding the persuasive function of political ads exposure online.

Second, this study supports the mere exposure theory, by showing a positive relationship between party preferences and ad exposure. Our findings also indicate the significant role of individual characteristics in online advertising. The effect of online political advertising is more pronounced for less knowledgeable and literate citizens, which is in line with previous studies (Lecheler & de Vreese, 2012; Vraga & Tully, 2021). Therefore, to build a functioning democracy, scholars and policymakers should provide thoughts regarding citizens’ competence in dealing with persuasive political communications.

Third, our findings indicate the importance of distinguishing propensity to vote from vote choice, especially in a multi-party system. Recently, the stability of European multi-party systems has been challenged: establishment of new parties, party splits, and electoral alliances or mergers have happened more frequently than ever. Consequently, it has become harder for voters in such context to develop attachment and loyalty toward a certain party. Voters in multi-party democracies are inclined to have a “general orientation toward political objects” based on their long-term political attitude, interest, and ideological stance (Paparo & Sio, 2017, p. 6), and the actual and final vote choice is developed from, but also independent of such orientation. We argue that it is of great importance to study how the propensity to vote and vote choice are impacted differently in multi-party democracies.

Practical Implications

Our findings provide practical implications for campaigners. The findings indicate that more exposure to political ads from a certain party affects people’s party preferences. Accordingly, sending reminder ads can be effective, which aligns with the findings of previous studies (Haenschen & Jennings, 2019). For regulators of political advertising, it is important to know that also in a multi-party context, political advertising has implications for elections and democracy. Furthermore, as online political advertising is more persuasive for less knowledgeable and literate voters, citizens may improve their competence, regulators may develop ways to protect citizens, and platforms may develop transparency policies that increase the detection of persuasive messages online, so citizens can extract useful information from targeted ads and formulating healthy information consumption during election periods.

Limitation and Future Research

Theoretically, having looked into the effects of ad quantity, the next step in our research agenda would be examining the effects of the content of political ads (e.g., valence, emotional appeals, etc.) by conducting a content analysis on the ads retrieved from social media to get a better understanding of what ad features might be most influential. Furthermore, this study examined the effects of Facebook ads which are to some extent targeted to a specific audience. However, we did not look into the extent to which the message is targeted. To investigate the actual targeting strategies, one should also take the actual congruency of the message into account.

From a methodological point of view, our browser tracking tool was only able to capture online political ads on the desktop version of Facebook. The exposure to political ads from other online platforms was not recorded. Besides, statistics show that 18.2% of Facebook users access Facebook via desktop, while 98.5% of Facebook users access Facebook via a mobile device (Backlinko, 2021). Although methodologically challenging, future research should take the hybrid media landscape into account and capture political ads across diverse media and platforms. For instance, the mobile experience sampling method could be used to collect individuals’ exposure to ads on different platforms, which can be compared to the findings of this study. We chose the latest Dutch General Election to study the persuasiveness of online political ads as investigating a country with a higher turnout rate and a multi-party system is also essential. We also propose that future work can examine online political ads in other countries, or conduct comparative studies across different countries.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are openly available at https://osf.io/e3us9/?view_only=1cdc4b9fe15342179089e7aedbd7a36c, http://doi.org/10.17605/osf.io/e3us9

Additional information

Funding

The work was supported by the The project DATADRIVEN is financially supported by the NORFACE Joint Research Programme on Democratic Governance in a Turbulent Age and co-funded by ESRC, FWF, NWO, and the European Commission through Horizon 2020 under grant agreement No 822166. .

Notes on contributors

Xiaotong Chu

Xiaotong Chu is a PhD student working at the University of Amsterdam and Wageningen University and Research. Her research focuses on the effects of targeting practices in modern electoral campaigns on individual-level responses. She has experience in quantitative research methods such as (automatic) content analysis, experiments, surveys, mobile experience sampling method, etc.

Rens Vliegenthart

Rens Vliegenthart is a professor in Strategic Communication, Wageningen University and Research. His research focuses on (social) media content and effects on citizens as well as on political decision making processes.

Lukas Otto

Lukas P. Otto (PhD, U of Koblenz-Landau) is a senior researcher at GESIS – Leibniz Institute for the Social Sciences, Computational Social Science Department. His research interests include dynamics of communication effects, political communication and emotion, and mobile and computational approaches to communication research.

Sophie Lecheler

Sophie Lecheler is Professor of Political Communication at the University of Vienna, Austria. Her research focuses on political journalism, digitalisation and politics, emotions, and experimental research methods.

Claes de Vreese

Claes de Vreese is University Professor of AI and Society with a special emphasis on Media and Democracy, University of Amsterdam. He is the Director of the Digital Democracy Center (SDU) and co-directs the AIMD lab and the national AlgoSoc program on Public Values in the Algorithmic Society.

Sanne Kruikemeier

Sanne Kruikemeier is Professor in Digital Media and Society at the Strategic Communication Group of Wageningen University & Research (WUR). Her research focuses on the consequences and implications of online communication for individuals and society.

Notes

1. Branches of parties were categorized as the parent party (e.g. “SP” and “SP Amsterdam” were both labeled as political party SP).

2. One respondent in our sample received 1,102 political ad impressions within the four waves, which is much higher than the average number of received political ad impressions of each respondent (M = 17.23, SD = 62.45). After conducting regression models with and without the outlier, we noticed that this outlier strongly affects the results, therefore we removed this outlier’s data.

Appendix A.

Phases of data collection and pre-processing

Figure A1. Phases of data collection and pre-processing.

Appendix B.

List of Dutch political parties

Table B1. List of Dutch political parties and number of recieved political ad impressions in each wave.

Parties	Founded in	Seats in 2017	Wave 1	Wave 2	Wave 3	Wave 4
VVD*	01/1948	33	8	115	108	607
PVV*	02/2006	20	0	0	0	0
CDA*	10/1980	19	7	259	185	261
D66*	10/1966	19	1	232	360	406
GroenLinks*	11/1990	14	4	217	287	504
SP*	10/1972	14	3	215	299	470
PvdA*	02/1946	9	7	553	429	508
ChristenUnie*	11/2000	5	1	27	16	106
Partij voor de Dieren*	10/2002	5	0	91	99	296
50 Plus*	08/2009	4	1	80	46	34
SGP*	04/1918	3	0	4	6	2
DENK*	02/2015	3	1	7	13	78
Forum voor Democratie*	09/2016	2	5	168	137	541
JA21*	12/2020	0	2	104	30	0
BIJ1*	12/2016	0	0	35	32	142
Code Oranje*	10/2018	0	2	53	11	28
Lijst Henk Krol*	11/2020	0	0	0	0	0
Volt*	06/2018	0	0	3	163	383
Piratenpartij**	03/2010	0	1	0	0	0
Splinter**	12/2020	0	0	0	0	0
Trots op Nederland**	10/2007	0	0	0	0	0
BoerBurgerBeweging**	10/2019	0	0	0	0	0

* indicates parties that were listed when measuring vote propensity and vote choice. Please note that Volt was only included in the third wave of the panel survey when measuring vote propensity.

**indicates parties that were not listed when measuring vote propensity and vote choice, but was manually filled in by respondents when measuring vote choice.

Appendix C.

Measures of variables

Table C1. Measures of dependent variables, moderators, and control variables.

Variable	Items	M	SD
Vote propensity	How likely it is that you will vote for … (party name)?	2.26	3.08
Vote choice	Which party would you vote for if there were elections tomorrow?	0.04	0.20
Political knowledge	Which of the following is the current Finance Minister?	0.91	0.29
	By whom are the members of the Senate elected?	0.82	0.38
	Which is the largest opposition party in the House of Representatives?	0.70	0.46
Online literacy	I know how to operate Facebook.	5.92	1.12
	I understand how my Facebook news feed is personalized for me.	5.29	1.53
	I have control over the privacy settings of the apps and websites I regularly use.	3.68	1.66
	I can make an informed decision about what information to disclose online and which not to.	4.07	1.51
Online media use	If you think about the past week, on how many of the 7 days did you visit … (website names)?	2.28	0.94
Newspaper use	If you think about the past week, on how many of the 7 days did you read … (newspaper names)?	1.63	0.73
TV use	If you think about the past week, on how many of the 7 days did you watch … (TV program names)?	1.95	0.88

Appendix D.

Bivariate correlations

To get an overview of the measured variables, we report the bivariate correlations (see Table D1). The correlation matrix shows significant correlations between the number of received political ads and dependent variables: vote propensity and vote choice. Dependent variables were positively correlated. The party-level moderator new party was negatively correlated with political ad count, as well as with vote propensity and vote choice. The individual-level moderators political knowledge and online privacy literacy were positively correlated with the number of received online political ads. The reported correlations between variables provide the first indication of the magnitudes and directions of effects among variables.

Table D1. Pearson’s correlations among variables.

Variables	1	2	3	4	5	6
1. Ad count	—
2. Propensity to vote	.105***	—
3. Vote choice	.090***	.541***	—
4. New party	−.055***	−.270***	−.117***	—
5. Political knowledge	.028***	−.002	.006	.000	—
6. Online literacy	.029***	−.016**	.001	.000	−.039***	—

*p <.1; **p < .05; ***p < .01.

Appendix E.

Bivariate correlations

Figure E1. Proportion of political ad impressions.

Figure E2. Change in vote choice of the absolute number of respondents from wave 1 to wave 4.

Appendix F.

Robustness check I

As a robustness check, multilevel regression analyses were conducted for all five of the models to predict the propensity to vote and vote choice. Identical to the regression models, we had vote propensity and vote choice from the previous wave as lagged dependent variables. Online media use, newspaper use, and TV use were included as predictors, so as respondents’ demographic variables (i.e., age, gender, educational level, and income). We found similar results compared to results from the previous regression analyses.

To be more specific, the main effect of the number of received political ads was supported for both the propensity to vote (B = 0.046, p = .001) and vote choice (OR = 1.081, p < .001). The moderating effect of new parties were not supported for both the vote propensity (B = 0.075, p = .099) and vote choice (OR = 1.009, p = .821). The moderating effects of political knowledge were marginally supported for the effect on vote choice (OR = 0.964, p = .055), but not for the effect on the propensity to vote (B = -0.021, p = .084). The moderating effects of online privacy literacy were marginally supported for the effect on vote choice (OR = 0.964, p = .055), but not for the effect on the propensity to vote (B = -0.018, p = .076).

Appendix G.

Robustness check II

We also conducted several fixed effects models to test whether effects are only based on between-individual variation or whether it indeed also captures variation within individuals over time. More specifically, we replicated Model 1 in Tables G1 and G2 using least squared dummy variables (LSDV) fixed effects for respondents, parties, and both. As fixed effects for individuals removed all the between-individual variation, we did not include individual-level characteristics. Results showed that the positive effect of ad exposure prevailed in all models, demonstrating that our analyses also captured within-individual (and within-party) variation.

Table G1. Fixed effects models predicting the propensity to vote.

	Model 1	Model 2	Model 3
Constant	.317*** (.055)	.216*** (.058)	.289*** (.066)
Ad count	.049*** (.015)	.033*** (.012)	.037*** (.013)
Previous propensity	.862*** (.011)	.823*** (.013)	.814*** (.015)
Wave 3	.036 (.061)	.048 (.057)	.036 (.060)
Fixed effects	respondent	party	respondent, party
N observations	15,130	15,130	15,130
N respondents	471	471	471
R squared	.767	.757	.773

Unstandardized coefficients (B) are reported; ***p < .001.

Table G2. Fixed effects models predicting vote choice.

	Model 1	Model 2	Model 3
Constant	.009*** (.001)	.003*** (.001)	.003*** (.001)
Ad count	1.189*** (.025)	1.053*** (.012)	1.083*** (.020)
Previous vote choice	396.364*** (62.878)	234.479*** (37.315)	243.197*** (40.669)
Wave 3	1.060 (.061)	1.079 (.061)	1.081 (.063)
Wave 4	1.121** (.056)	1.187*** (.060)	1.165** (.062)
Fixed effects	respondent	party	respondent, party
N observations	27,940	27,279	26,670
N respondents	454	471	454
R squared	.577	.597	.601

Odd Ratio’s are reported; **p < .05; ***p < .01.