Does the prospect of further sovereignty loss fuel Euroscepticism? A population-based survey experiment

ABSTRACT

Whether further European integration is desirable is an ongoing question in public opinion research: both extant research and the outcome of various EU-related referendums show that many citizens hold negative views on proposals that lead to further loss of national sovereignty. An important issue thus arises: does the prospect of such additional sovereignty loss increase negativity towards the EU? This study attempts to answer this question using a pre-registered original survey experiment conducted among members of a nationally-representative high-quality Dutch panel. Our focus is on how exposure to proposed abolishment of EU member states’ right of veto affects EU attitudes. In addition, we analyse whether the exposure effect is shaped by 1) citizens’ prior populist attitudes and 2) the Eurosceptic character of the medium. Concerning the former, informed by recent in-depth qualitative research, we hypothesise that populist attitudes aggravate the extent to which exposure to potential loss of national sovereignty leads to more negative EU attitudes. Concerning the latter, we hypothesise that exposure via a Eurosceptic medium might either aggravate or abate the extent to which the newspaper message leads to more negative EU attitudes. We discuss our findings and provide suggestions for further research.

KEYWORDS:

• Euroscepticism
• European integration
• population-based survey experiment
• right of veto
• populist attitudes

1. Introduction

While the existence of the European Union (EU) evidently entails a degree of European integration and hence sovereignty loss of individual member states, the issue of whether further integration is desirable is fiercely debated and a key issue in public opinion research. Various studies have scrutinised citizens’ attitudes on this matter (e.g. De Vries and Steenbergen 2013; Ejrnaes and Jensen 2019; Janssen 1991; Toshkov and Krouwel 2022; Vössing 2015), which is of paramount importance as ‘[p]ublic attitudes, through mass political behaviour, shape and constrain the process of European integration’ (Gabel 1998: 333). The surprising results of the referendums on the EU constitution in 2005 (Baden and De Vreese 2008; Hobolt 2007; Hobolt and Brouard 2011; Lubbers 2008; Schuck and De Vreese 2008) and the association treaty with Ukraine in 2016 (Abts et al. 2023; Van der Brug et al. 2018), indeed made it abundantly clear that citizens’ stances influence the process of European integration. Moreover, those referendums suggest that many citizens hold negative views on proposals that lead to further loss of national sovereignty.

Against this background of popular aversion to further European integration and accompanied additional loss of national sovereignty, we ask: does the prospect of further sovereignty loss fuel Euroscepticism? This study aims to address this question by employing a pre-registered survey experiment conducted among members of a high-quality panel representative of the Dutch population to test empirically whether exposure to plans for further EU integration strengthens negative attitudes to the institution. To this end, we experimentally exposed respondents to the potential abolition of the right of veto of EU member states. We strategically selected that issue for our experimental condition, as it is supported by some countries in the EU (BNR 2021) and, if implemented, would clearly lead to an additional loss of national sovereignty.

Using the EU member states’ right of veto as a case adds to the external validity of our experiment, and it does not tap into substantive political topics discussed within the EU, like migration or measures to mitigate climate change. Moreover, by situating the study in the Netherlands, which is far less influential in the EU than, e.g. France or Germany, we are increasing the likelihood that the respondents will regard the loss of the right of veto as a threat to national sovereignty. Our experiment is used to first answer the following research question: does the proposed abolishment of the EU member states’ right of veto increase negative EU attitudes?

It is doubtful that the impact will be universal: the same information is likely to be interpreted differently, depending on its source and the individual viewing it. After all, ‘every opinion is a marriage of information and predisposition: information to form a mental picture of a given issue, and predisposition to motivate some conclusion about it’ (Zaller 1992: 6). That the same information can be interpreted differently by different segments of the population depending on a person’s understanding of a subject has also been found by various empirical studies (De Koster et al. 2016; De Koster and Achterberg 2015; Kahan 2010; Van Rijn et al. 2019). With regards to EU attitudes, the degree of populist attitudes has been found to be crucial for how the EU is understood (Van den Hoogen et al. 2022a), thus we will scrutinise whether populist attitudes amongst citizens moderate the effect of exposure to plans to abolish the right of veto.

As information does not exist in a vacuum, how a proposal to remove the right of veto is communicated probably affects its impact. Research has found that the media in general play a ‘double role in both fuelling and reducing Euroscepticism’, depending, amongst other things, on the source (De Vreese 2007: 42). It also shows that specific media can be affiliated with a specific stance on the EU – ranging from Europositive to Euro-ambivalent to Eurosceptic (Leruth et al. 2017). Consequently, we also test whether the medium (Europositive versus Eurosceptic) in which the message is communicated shapes the attitudinal consequences of the message. Therefore, the second question to be answered with our experiment is: does the effect of the proposed abolishment of the EU member states’ right of veto on EU attitudes differ depending on the level of populist attitudes of the receiver or the Eurosceptic character of the medium in which the message is communicated?

2. The effect of a proposed loss of national sovereignty on EU attitudes

People’s attitudes on whether the EU should integrate further have been included in many different kinds of studies. For example, some studies have used citizens’ stances on further integration as an indicator of support for the institution more generally (e.g. Anderson and Hecht 2018; Baute et al. 2018; Carey 2002; Hooghe and Marks 2004; Jayet 2020). At the same time, various studies have linked negative attitudes towards the European Union to concerns about loss of national sovereignty (e.g. Borriello and Brack 2019; Carrieri and Vittori 2021; Pirro and Van Kessel 2017; Yordanova et al. 2020). Moreover, recent qualitative and quantitative studies set in the Netherlands (Van den Hoogen 2022a; 2022b) have demonstrated that further sovereignty loss is an issue of concern for Eurosceptics and Euro-enthusiasts alike. More specifically, these earlier studies showed that being explicitly pro-EU, i.e. being in favour of their country’s membership to the EU, does not necessarily go hand in hand with being in favour of further European integration. It thus seems likely that proposals that would result in further loss of national sovereignty, of which abolishment of the right of veto of member states is a key example, would negatively impact attitudes to the EU across the board.

To assess whether public opinion on the EU is affected by a proposal that signals national sovereignty loss, we carefully selected five distinct EU attitudes. Informed by prior survey research (Boomgaarden et al. 2011; De Vreese et al. 2019) and in-depth interviews among Dutch citizens (Van den Hoogen et al. 2022a), this study distinguishes four attitudes tapping into the rational aspects of EU attitudes: on The Netherlands’ membership to the EU; on a Dutch exit out of the EU (‘Nexit’); on Dutch sovereignty within the EU; and on EU enlargement. Informed by recent studies highlighting the importance of emotions in EU attitudinal research (e.g. De Vreese et al. 2019; Marquart et al. 2019; Nielsen 2018; Verbalyte and Von Scheve 2018), we also selected an attitude focusing on a negative emotive response to the EU. This attitude, measuring disgust with the EU, is based on earlier research and frequently used in EU attitudinal research (e.g. Bakker and De Vreese 2016; Boomgaarden et al. 2011; De Vreese et al. 2019). We therefore hypothesise (H1a-H1e): A proposal for abolishment of EU member states’ right of veto causes citizens to be a) more negative towards EU membership; b) more negative towards EU expansion; c) more supportive of a ‘Nexit’; d) more likely to believe that the EU endangers Dutch sovereignty; e) more disgusted with the EU.

Of course, not everyone is likely to interpret such cues in the same way: how citizens understand and respond to certain prompts depends on their ‘principles of selection, emphasis and presentation composed of little tacit theories about what exists, what happens, and what matters’ (Gitlin 1980: 6; cf. Hall 1980; Zaller 1992). In other words, it is likely that the meaning an individual ascribes to a social phenomenon influences how cues about it are perceived. Indeed, extant research on a wide range of issues has shown that worldviews moderate how informational or neighbourhood cues affect political attitudes and behaviour, e.g. concerning opinions on hydrogen technologies (de Koster and Achterberg 2015), Turkey becoming an EU member (Azrout et al. 2011), suspended prison sentences (de Koster et al. 2016), or vote choices (Van Noord et al. 2018).

This is likely also the case when the focus is on cues regarding an EU-induced additional loss of national sovereignty. As there are various indications that people have different understandings of the EU (Van den Hoogen et al. 2022a; 2022b), this likely shapes their reactions to EU policy proposals. Populist attitudes are crucial in this respect: a recent interview study among Dutch citizens showed that those whose views are more anti-establishment tend to perceive the EU as a malicious instrument of the elites, which is only used to further their interests at the detriment of the common people. Those whose views are more populist were found to perceive the EU as being led by a group of elites that make up the top of a power hierarchy, from which they ‘exploit’ all member states out of their own self-interest (Van den Hoogen et al. 2022a). Respondents from this study even described the EU as a ‘tactical coup’ (Van den Hoogen et al. 2022a: 11) and named the Dutch prime minister ‘the bellboy of the EU’ (p. 11).

Having a populist understanding of the EU, then, is likely to underlie a more negative response to a proposal that would result in additional loss of national sovereignty. After all, from that perspective, such a proposal would give more power to a ‘malignant’ EU at the cost of the sovereignty of the nation states themselves: it would be even better positioned to ‘exploit’ the Netherlands than it currently does. The notion of ‘motivated reasoning’ predicts something similar: citizens’ prior populist attitudes could act as a ‘perceptual screen’, motivating ‘effortful processing of relevant information’ (Bolsen et al. 2014: 241; cf. Taber and Lodge 2012 [2006]; Taber and Lodge 2012). Therefore, the policy proposal in our experiment will likely reinforce negative opinions about the EU for people with a higher level of populist attitudes. We therefore anticipate that the hypothesised effect of the proposal for abolishment of EU member states’ right of veto on negative EU attitudes is stronger for citizens with a higher level of populist attitudes (H2a-H2e):

There is a positive interaction between exposure to a proposal for abolishment of EU member states’ right of veto and having more populist attitudes on a) a negative view on EU membership; b) a negative view on EU enlargement; c) support for a ‘Nexit’; d) the belief that the EU endangers Dutch sovereignty; e) disgust with the EU.

Besides the potentially moderating effect of citizens’ characteristics, various studies have also highlighted the role played by news media in forming public opinion (e.g. de Vreese and Boomgaarden 2006c; Kepplinger 2012; Zaller 1992). More specifically, the effect of media on citizens’ EU attitudes and knowledge has been thoroughly studied. Some studies focus on the effect of media on EU attitudes on an aggregate level (e.g. Schuck et al. 2013; Vliegenthart et al. 2008), other studies focus on the effect of media on individual EU attitudes (e.g. Brosius et al. 2019; de Vreese 2007; de Vreese and Boomgaarden 2006b; Michailidou 2015; van Spanje and de Vreese 2014; Wojcieszak et al. 2018), and again others focus on the role of that exposure on attitudes via knowledge gains on the EU (e.g. Azrout et al. 2012; Marquart et al. 2018). In the context of the present study, it is likely that the degree of negativity towards the EU as a result of a proposal that further reduces national sovereignty will depend on the medium via which it is communicated. After all, whether the media fuel or quell Euroscepticism depends on the source (de Vreese 2007). Extant research has shown that media messages that are negative about the EU are associated with support for Eurosceptic parties (Galpin and Trenz 2017). Vliegenthart and colleagues (2008) demonstrated that a negative representation in such messages leads to the EU being regarded less favourably, while Van Spanje and De Vreese (2014) noted that ‘the more negative the evaluations of the EU a voter is exposed to [in the media], the more likely she or he is to cast a vote for a Eurosceptic party’ (p. 341).

The way newspapers report on EU issues typically reflects their general stance towards the EU, which is often consistent and ranges from Europositive via Euro-ambivalent to Eurosceptic (Leruth et al. 2017: 101). We consider it likely that such a stance will influence how reports on EU-related matters are interpreted by readers. Inspired by research on the distinctions between tabloid and broadsheet newspapers (e.g. de Vreese and Boomgaarden 2006a; Dutceac Segesten and Bossetta 2019; Hameleers 2019; Leruth et al. 2017), our focus in this study is more specifically on the differences between a tabloid known to be consistently Eurosceptic and a broadsheet widely acknowledged as representing the opposite position. We hypothesise that the medium’s stance could have two possible effects. First, exposure to a report contained in a newspaper which generally has a negative position concerning EU matters, and thus a Eurosceptic connotation, could lead to more negative interpretations of our proposed treatment (H3a-H3e):

There is a positive interaction between exposure to a proposal for abolishment of EU member states’ right of veto and the message being published in a Eurosceptic newspaper on a) a negative view on EU membership; b) a negative view on EU enlargement; c) support for a ‘Nexit’; d) the belief that the EU endangers Dutch sovereignty; e) disgust with the EU.

Second, the opposite could also be true. If a newspaper that is normally positive about the EU, and thus has more of a Europhile connotation, emphasises a potential loss of national sovereignty due to policy changes by the institution, this issue and its impact might be taken more seriously. Consequently, negative attitudes towards the EU may increase even more strongly if information about that potential loss is reported in a newspaper that is not usually Eurosceptic (H4a-H4e):

There is a negative interaction between exposure to a proposal for abolishment of EU member states’ right of veto and the message being published in a Eurosceptic newspaper on a) a negative view on EU membership; b) a negative view on EU enlargement; c) support for a ‘Nexit’; d) the belief that the EU endangers Dutch sovereignty; e) disgust with the EU.

3. Data, treatment & measures

Our study was pre-registered on the Open Science Framework (OSF)¹ prior to the start of data collection and received ethical approval from the Research Ethics Review Committee of the Department of Public Administration and Sociology at Erasmus University Rotterdam.² The causal relationship between a proposal resulting in a loss of national sovereignty and negative attitudes towards the EU was assessed via a randomised, population-based, survey experiment using a 2×2 factorial between-subjects design (n = 1,587). Respondents were therefore randomly assigned to one of four conditions, ensuring that any characteristics potentially relevant for processing those conditions were randomly distributed. We included pre-measures that were identical to the outcome measures from a previous survey completed by the same respondents as controls (De Koster et al. 2020). This made the identification of the treatment’s effect more precise (Clifford et al. 2021). In addition to these pre-measures, we also measured populist attitudes in the same earlier survey (Akkerman et al. 2014). These pre-measured EU and populist attitudes were linked to the data newly collected for the present study.

We used the high-quality LISS (Longitudinal Internet Studies for the Social Sciences) panel, and only included participants aged 18 or older. The LISS panel was established via true probability sampling of households in the Netherlands using the official Dutch population register (Scherpenzeel, 2009). Centerdata, an independent scientific research institute affiliated with Tilburg University, manages the panel and collected the data on our behalf.

After exposure to one of the four treatment conditions detailed below, the respondents answered questions on their attitudes towards the EU. This enabled us to estimate both the causal effects of a proposal concerning the abolition of the EU member states’ right of veto on EU attitudes, and whether these effects are moderated by prior populist attitudes and the Eurosceptic character of the medium. The respondents also answered two manipulation check questions, one on the topic of the text and the other on the medium in which it was, supposedly, published. However, since the literature indicates that removing respondents from a sample in this way could bias results (Aronow et al. 2019; Montgomery et al. 2018), we only excluded those who had failed the check included in the sensitivity analyses (see below), and not when testing our pre-registered hypotheses.

3.1. Treatment

The treatment was an image of an online newspaper article reporting on EU member states’ right of veto. The text either just explained this right (control) or, in addition, addressed plans for its potential abolition (treatment). Furthermore, the report was shown as an image that mimicked either an online page of a Eurosceptic tabloid or a non-Eurosceptic broadsheet. We chose De Telegraaf and de Volkskrant for this purpose, as an expert survey has determined the former to be largely Eurosceptic and the latter to be generally Europositive or Euroambivalent (Leruth et al. 2017: 101). The text of the report was based on actual coverage of EU member states’ right of veto and its potential abolition (BNR 2021), thus increasing the experiment’s external validity. This resulted in four experimental conditions differing both in message (control versus treatment) and source (Eurosceptic versus non-Eurosceptic). The experimental conditions, including the text and the layout, can be found in the online supplementary material (Figure S1a – Figure S1d).

3.2. Measures

Informed by survey research (Boomgaarden et al. 2011; de Vreese et al. 2019) and recent in-depth interviews (Van den Hoogen et al. 2022a), we selected five items to measure EU attitudes: 1) ‘Dutch membership of the European Union is a good thing’; 2) ‘The European Union should be enlarged with other countries’; 3) ‘The Netherlands should leave the European Union’; 4) ‘The European Union endangers the independence of the Netherlands’; and 5) ‘I am disgusted by the European Union’. The respondents were asked to respond to each statement on a seven-point Likert scale, with the possible answers ranging from ‘completely disagree’ to ‘completely agree’. The first two items were recoded such that a higher score reflects more negative attitudes on the EU. Reflecting our hypotheses, the variables measuring EU attitudes are not combined into a single scale.

Populist attitudes were measured using Akkerman et al.’s (2014) widely used scale, for which we combined the following items: 1) ‘The people, and not politicians, should make our most important policy decisions’; 2) ‘The political differences between the elite and the people are larger than the differences among the people’; 3) ‘I would rather be represented by a citizen than by a specialised politician’; 4) ‘Elected officials talk too much and take too little action’; and 5) ‘What people call ‘compromise’ in politics is really just selling out on one’s principles’. Respondents answered each of these statements on a seven-point Likert scale, ranging from ‘completely disagree’ to ‘completely agree’. We calculated the mean score for respondents with valid scores on all five items. A reliability analysis demonstrated that the scale was highly reliable, with α = 0.83 (M: 4.26; SD: 1.13).

4. Results

4.1. Main analyses

We estimated the treatment effects using ordinary least squares (OLS) regression models (Table 1). These concerned the causal effects of the proposed abolition of the right of veto on negative attitudes towards the EU (Model 1), as well as the interaction effects on those attitudes between the proposal and populist attitudes (Model 2) and the type of newspaper containing the report (Model 3). As we used five distinct attitudes towards the EU, we ran each of the three models five times – once per EU attitude. Each model also included the corresponding pre-measured attitude to increase the precision of the estimates (Clifford et al. 2021).

Table 1. OLS regression estimates for negative EU attitudes.

Variables	Negative view on EU membership			Negative view on EU enlargement			Support for Nexit			Belief that the EU endangers Dutch sovereignty			Disgust with the EU
	Model 1	Model 2	Model 3	Model 1	Model 2	Model 3	Model 1	Model 2	Model 3	Model 1	Model 2	Model 3	Model 1	Model 2	Model 3
Treatment: proposal for abolishment veto	−0.060 (0.053)	−0.249 (0.205)	−0.009 (0.075)	−0.006 (0.076)	−0.547 (0.292)	−0.065 (0.108)	−0.036 (0.056)	−0.379 (0.216)	0.048 (0.079)	0.086 (0.070)	−0.226 (0.268)	0.112 (0.099)	0.039 (0.061)	−0.298 (0.235)	0.015 (0.087)
Treatment: Eurosceptic newspaper			0.217** (0.075)			0.006 (0.108)			0.256** (0.079)			0.128 (0.099)			0.117 (0.087)
Populist attitudes		0.105** (0.035)			0.128** (0.047)			0.116** (0.037)			0.193*** (0.046)			0.135** (0.041)
Proposal for abolishment veto * populist attitudes		0.045 (0.046)			0.127 0.066			0.081 (0.049)			0.073 (0.061)			0.080 (0.053)
Proposal for abolishment veto * Eurosceptic newspaper			−0.100 (0.106)			0.121 (0.153)			−0.170 (0.112)			−0.052 (0.141)			0.049 (0.123)
Premeasure dependent variable	0.671*** (0.017)	0.629*** (0.019)	0.673*** (0.017)	0.507*** (0.026)	0.483*** (0.026)	0.509*** (0.026)	0.640*** (0.017)	0.590*** (0.019)	0.642*** (0.017)	0.634*** (0.022)	0.554*** (0.025)	0.634*** (0.022)	0.696*** (0.019)	0.632*** (0.022)	0.697*** (0.019)
Constant	0.611*** (0.062)	0.284 (0.146)	0.498*** (0.073)	1.162*** (0.146)	0.736** (0.237)	1.149*** (0.156)	0.752*** 0.060	0.388* (0.154)	0.618*** (0.073)	0.780*** (0.098)	0.260 (0.192)	0.716*** (0.109)	0.413*** (0.070)	0.021 (0.168)	0.350*** (0.084)
R^2	0.498	0.506	0.501	0.349	0.367	0.350	0.461	0.473	0.463	0.349	0.367	0.350	0.461	0.473	0.463

*** p<0.001, ** p<0.01, * p<0.05. Unstandardised coefficients, with standard errors in parentheses. n = 1,587

As shown in Table 1, we did not identify any significant direct effects of the treatment on attitudes to the EU. H1a-H1e are therefore not supported. Moreover, its interaction effects with populist attitudes (models 2) or with the medium in which the message is reported (models 3), are also not significant. We plotted the interaction effects as to aid their interpretation (Brambor et al. 2006).

Figure 1 shows the conditional marginal effects of a proposal for the abolishment of right of veto on EU attitudes by populist attitudes. Although H2a-H2e need to be rejected, the consistent pattern suggests that the EU attitudes of more populist citizens may become more negative after reading that proposal while that is not the case among the least populist citizens. We will discuss this further in the conclusion.

Figure 1. Conditional marginal effects of abolishment of EU’s right of veto on negative EU attitudes, by prior populist attitudes (95% confidence interval), including histograms of the distribution of prior populist attitudes.

In contrast to Figure 1, Figure 2 shows no clear pattern. The proposal of abolishment of right of veto hardly interacts with the type of newspaper in which it is published, and the weak interaction effects also differ in direction across EU attitudes. Clearly the kind of newspaper in which the message is reported does not shape how exposure to a proposal of abolishment of right of veto influences those attitudes, and H3a-H3e and H4a-H4e need to be rejected.

Figure 2. Predicted negative EU attitudes by treatment condition and the medium in which the message is communicated.

4.2. Four exploratory analyses

We conducted four types of exploratory analyses. First, we looked at the direct effects on EU attitudes of the type of newspaper in which the treatment proposal was published. As indicated in Table 2, there was a positive, significant, direct effect for three of the five attitudes measured: publishing the text in a Eurosceptic newspaper led to more negative views on the EU; more support for a Nexit; and higher levels of disgust with the EU. As the message reported in either newspaper was the same for both, this indicates that the Eurosceptic connotation of a certain newspaper affects EU attitudes, even though it does not moderate the impact of our treatment. We reflect on this further in the conclusion.

Table 2. OLS regression estimates for negative EU attitudes.

Variables	Negative view on EU membership	Negative view on EU enlargement	Support for Nexit	Belief that the EU endangers Dutch sovereignty	Disgust with the EU
Treatment: Eurosceptic newspaper	0.167** (0.053)	0.066 (0.076)	0.171** (0.056)	0.102 (0.070)	0.142* (0.061)
Premeasure dependent variable	0.672*** (0.017)	0.508*** (0.026)	0.642*** (0.017)	0.634*** (0.022)	0.697*** (0.019)
Constant	0.495*** (0.062)	1.121*** (0.146)	0.644*** (0.060)	0.772*** (0.098)	0.357*** (0.071)
R^2	0.500	0.192	0.488	0.349	0.463

*** p<0.001, ** p<0.01, * p<0.05. Unstandardised coefficients, with standard errors in parentheses (n = 1,587)

Second, we included measures on the credibility (‘The information in the newspaper article is credible’) and comprehensibility (‘The newspaper article is hard to understand’) of the text. The participants reacted to these statements after responding to the measures of EU attitudes. The responses to these items demonstrated that most of them regarded the (control and treatment) texts as credible, with only 5.9% indicating otherwise. Moreover, only 8.1% of the respondents said that the text they had read was hard to understand. ANOVAs were used to explore whether the treatments differed in terms of their credibility (F(6, 1579) = 0.69, p = 0.656) and/or incomprehensibility (F(6, 1579) = 0.95, p = 0.456), but no significant differences were identified. This finding underscores the validity of our experiment and results.

Third, to further explore the indication that more-populist citizens respond differently to the message than less-populist citizens, we calculated the association of populist attitudes with the response to questions about the credibility and incomprehensibility of the treatments: r = −0.204 (p < .001) and r = 0.140 (p < .001), respectively. Respondents with more populist attitudes regarded the treatments as less credible and harder to comprehend than people with a lower level of populist attitudes. The former finding is comparable with extant research showing that an anti-establishment perspective on the EU goes together with low trust in the EU (Van den Hoogen et al. 2022b, see also Van den Hoogen et al. 2022a), while the finding that populist respondents regard the experimental material harder to comprehend relates to the idea that populist attitudes are associated with possessing limited political knowledge (e.g. Van Kessel et al. 2021).

Fourth, as national identity related concerns are also often used as an explanation for opposing further European integration (e.g. Carey 2002; Ejrnaes and Jensen 2019; Luedtke 2005), we also explored whether nationalist attitudes moderate the effect of exposure to a proposal for the abolishment of EU member states’ right of veto on EU attitudes. The panel used to field our survey experiment included two items that could be used to that end in a previous survey (De Koster et al. 2020): ‘The world would be a better place if people from other countries were more like the Dutch’ and ‘Generally speaking, The Netherlands is a better country than most other countries’, both based on Davidov (2011). Both were measured in 2020, and we used their mean score as a moderator (α = 0.64; M: 4.31; SD: 1.19). Similar to our findings for populist attitudes, OLS regression analyses did not show a significant interaction effect of nationalist attitudes with our treatment for any of the five distinct EU attitudes (see table S3 in the Online Supplementary Material).

4.3. Sensitivity analyses

Informed by the suggestion that removing respondents based on manipulation checks may bias results (Aronow et al. 2019; Montgomery et al. 2018), we did not exclude those who failed the checks in our main analyses. Yet, we applied a sensitivity analysis by excluding them, especially because of their remarkably high number: 122 respondents failed to identify the right textual treatment and 788 to recognise the correct medium (incorrect answers and ‘don’t knows’ combined). Reflecting on why roughly half of the respondents were unable to identify the right newspaper, we noted the following: 1) the subtlety of the manipulation, i.e. just the layout, font and heading, may have made this task difficult; and 2) the four answer categories of the manipulation check, i.e. two tabloids and two broadsheet newspapers, might also have caused this question to be harder to answer and, as a result, responsible for the high number of respondents who chose the ‘don’t know’ option (n = 389).

The results of the sensitivity analyses after excluding the high number of respondents who failed the manipulation checks were, nonetheless, very similar to those of our main analysis. In other words, no significant direct and interaction effects were identified (table S1 and table S2 in the online supplementary material). Moreover, excluding these respondents did not affect the results reported in Table 2 of the main analysis, i.e. those exposed to an article in a newspaper known for its Eurosceptic stance had a more negative attitude towards EU membership, greater support for a Nexit and higher levels of disgust with the institution (Table S2 in the online supplementary material). The finding that the respondents who were unable to identify the newspaper correctly were affected equally by the article as those who were able to do so suggests that the Euroscepticism of the medium was, in general, processed unconsciously. We will elaborate on this further in the conclusion.

5. Conclusion

Informed by the literature, we tested whether exposing Dutch citizens to a report of plans for further EU integration that would result in additional loss of national sovereignty increased negative EU attitudes. Our pre-registered survey experiment, conducted among members of a high quality, nationally representative panel, demonstrated that those exposed to a proposal to abolish member states’ right of veto did not report more negative attitudes to the EU than those who were just informed about what this right of veto entailed.

There may be three reasons for this unexpected result. First, as we modelled our experimental treatment after a real-world news message, it might be that some respondents were already familiar with its contents. If that is indeed the case, it could be that an effect is only present among those not yet familiar with the message’s content, which could render the overall effect too small to detect. However, this seems unlikely, as the issue of abolishment of the right of veto was not extensively and structurally covered by Dutch media and neither hotly debated in the Dutch public sphere. In addition, even if the general content of the message was already familiar to some respondents, being exposed to the treatment would still have highlighted the salience of the issue relative to the respondents in the control condition.

Second, the issue chosen for exposing respondents to potential sovereignty loss might be too complex for inspiring feelings of infringement on national sovereignty. We did ensure that our treatment condition made explicit mention of concerns about the loss of sovereignty following the proposed abolition of the right of veto. Nevertheless, the procedures and processes relating to this right mostly occur ‘behind the scenes’, and the direct impact on citizens’ day-to-day lives may be quite subtle. It should, however, be noted that very few respondents said the text in the article they read was too difficult to understand. Moreover, we did not identify any differences in relation to comprehensibility between the control and treatment conditions. Future research could thus investigate whether other examples of national sovereignty loss have an impact on the public’s attitudes. This could be achieved in at least two ways, for example, by conducting: 1) survey experiments that adopt a similar approach, but use different examples, such as announcing an EU constitution, potential increases in its budget or the implementation of EU taxes; and 2) natural experiments that involve actual further EU integration in the future, possibly employing difference-in-difference analyses of the effects on public opinion.

Third, the influence of the Netherlands, as a net contributor to and founding member of the EU, might be overestimated by Dutch citizens. They may thus regard the prospect of sovereignty loss as less problematic, at least in comparison to citizens of member states that joined the EU later and are net beneficiaries, such as Greece, Portugal or Spain (Hobolt 2014), or those whose countries are experiencing an ‘illiberal turn’ that is the subject of strong criticism from EU officials, Poland and Hungary for instance (Krastev and Holmes 2018). The attitudes towards the EU of citizens in those member states might, consequently, become more negative if they are confronted with a potential loss of national sovereignty than is the case in the Netherlands. Future studies could investigate this issue by fielding a similar kind of experiment in those types of countries.

Next, in relation to the conditional treatment effects, we found that the interaction terms used to estimate whether the treatment especially leads to more negative EU attitudes among citizens with a high level of populist attitudes did not reach pre-registered levels of significance. Consequently, the associated hypotheses were not supported. It should, however, be noted that those interaction terms did affect all five dependent variables in the hypothesised directions, and that exposure to the proposal seemed to have a negative impact on the attitudes of the most populist citizens. This suggests that, in different circumstances, the prospect of sovereignty loss might strengthen the negative opinions on the EU of citizens who hold populist views. As populism in some countries is more intertwined with opposition to further European integration than in other ones (e.g. Kneuer 2018, see also Taggart and Pirro 2021), it is not unlikely that a similar experiment may yield stronger, or weaker, effects when fielded elsewhere. Future research that scrutinises this suggestion would thus be worthwhile.

In contrast to the case with populist attitudes, the type of newspaper did not shape in any way how the exposure to a proposed abolition of the right of veto affected attitudes towards the EU. It is, however, notable that our exploratory analyses did identify that the respondents exposed to the treatment in a Eurosceptic newspaper were more negative than those exposed to it in one that is not. We can draw two conclusions from this direct effect on attitudes to the EU of the type of newspaper employed: 1) a lack of internal validity is not a likely cause of the non-significant interaction with exposure to the proposal; and 2) a very subtle change in the layout of the messaging can have very noticeable effects, even when the content does not change. In terms of the latter, it is interesting to note that while roughly half of the respondents incorrectly identified the medium they were exposed to, this medium nevertheless affected their attitudes to the EU in a similar way to those who made the correct identification. In other words, the part played by the type of medium in shaping negative views on the EU seems to run deeper than the literature typically reports. Indeed, previous studies have reported that exposure to negative news on EU-related matters adversely affects opinions towards it (Galpin and Trenz 2017; van Spanje and de Vreese 2014; Vliegenthart et al. 2008). Our analysis, however, demonstrates that a non-negative article on these issues may even have such an impact in cases where it is published in a newspaper renowned for its Euroscepticism, irrespective of whether people are able to identify the medium correctly, but are exposed to it via subtle cues like its layout. Note that higher aversion to the EU after exposure to such subtle cues include reporting more negative emotions, which is measured as disgust. In other words: the most subtle hint at a Eurosceptic messenger can already increase an outspoken negative emotion towards the EU. Given the role played by the tabloid press in the Brexit vote in 2016 (e.g. Simpson and Startin 2022) and Euroscepticism more generally (e.g. Foos and Bischof 2021; Leruth et al. 2017), this finding makes an important contribution to the field. After all, it highlights the potential influence a Eurosceptic connotation of a medium could have on general attitudes towards the EU.

Generally, one could argue that the weak to non-existent effects found in this study reflect that many attitudes, including those on the EU (e.g. Kunst et al. 2022), hardly change due to events or policy changes. Especially because two aspects of our experiment made it likely to detect any policy-change induced attitude change if it actually occurred. First, it provides a realistic powerful manipulation (Kinder and Palfrey 1993: 27), by exposing respondents to a policy change with vast consequences for national sovereignty. Second, our experiment has high external validity (Barabas and Jerit 2010: 227), as the proposed policy change was presented in existing newspaper articles’ layout, using content based on real-life material. A future natural experiment utilising the actual abolishment of veto rights in case this would occur would be the litmus test for uncovering whether attitudinal change really does not occur due to such a profound policy change.

In summary, exposure to a proposal to abolish the right of veto of EU member states did not have a negative effect on the attitudes towards the institution of Dutch citizens, and nor was this outcome conditional on their populist attitudes or the Eurosceptic nature of the newspaper in which the proposal was published. Nevertheless, our findings also suggest that potential sovereignty loss might especially upset the most populist citizens. They, moreover, suggest that a subtle and unconscious association with the Eurosceptic character of a newspaper might already make its readers more Eurosceptic. Future research could uncover whether our findings depend on the specific issue used to expose respondents to potential sovereignty loss, and how far they travel beyond the Dutch case.

Ethics approval statement

Our study conforms with all relevant ethical regulations. In addition to Centerdata’s ethical regulations, our study was reviewed by and received ethical approval from the Ethics Review Committee of the Department of Public Administration and Sociology, Erasmus School of Social and Behavioural Sciences, Erasmus University Rotterdam, the Netherlands.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Research data were collected from Centerdata’s LISS panel and are freely available for academic research purposes from the LISS Data Archive repository, which has the international CoreTrustSeal certification. Given reasons of privacy and ensuring a secure storage and responsible usage of the data, the data can only be accessed through the LISS Data Archive. A replication package that contains links to the data, as well as the code that enables reproducing our results, can be found here: https://surfdrive.surf.nl/files/index.php/s/TpyBJStqk1rBh24.

Additional information

Funding

This work was supported by a Vidi grant awarded to Willem de Koster by the Dutch Research Council (NWO) [016.Vidi.185.207].

Notes

1 Link to the OSF registration: https://osf.io/2mv7a?view_only=ffca4f6b189f4111bda0961fde991688

2 Link to the data and code that can be used to replicate our results: https://surfdrive.surf.nl/files/index.php/s/TpyBJStqk1rBh24