Virtuous people and evil elites? The role of moralizing frames and normative distinctions in identifying populist discourse

Figures & data

Table 1. Morality and antagonistic groups as markers of populist discourse.

Table 2. Keywords for references to the people and the elite.

People		Elite
citizen	population’s	apparatchik*	europhil*
citizen’s	resident	aristocrat*	financial sharks
citizens	resident’s	bad apples*	financier*
citizens’	residents	banks	mafia
europeans	residents’	bosses	mafia’s
europeans’	taxpayer	bosses’	old elites
families	taxpayer’s	Brussels mafia	oligarch*
family	taxpayers	bureaucrat	political class*
man on the street*	taxpayers’	circles*	politician*
ordinary person*	the public	cronies	power monger
the public’s	people	ruling circles	elit*
people’s	population	establishment	ruling class
	establishment’s	sycophancy	eurocra*

Table 3. Selection of samples for gold standard.

Sample	Sampling	No of speeches	% Populist (hand-coded)
	False positives
1	Instances of people-centrism following my approach	100	65 per cent
2	Instances of anti-elitism following my approach	100	69 per cent
	False negatives
3	Based on Bonikowski and Gidron (2018) and Rooduijn and Pauwels (2011)	100	25 per cent
	False positives and false negatives
4	Random, over-sampling of populist parties (50:50)	100	6 per cent

Table 4. Classification accuracy.

	My approach	Bonikowski & Gidron	Rooduijn & Pauwels	References to groups
Sensitivity	0.66	0.49	0.49	0.51
Specificity	0.84	0.61	0.72	0.94
Balanced accuracy	0.75	0.55	0.61	0.72
Overall accuracy	0.74	0.56	0.55	0.58

Table 5. Classification of speeches for different dictionaries.

Populist party		Own approach	Bonikowski & Girdon	Rooduijn & Pauwels
No	Count	3623	2003	53,282
No	Proportion	0.025	0.014	0.371
Yes	Count	1148	880	13,715
Yes	Proportion	0.032	0.025	0.389

Table 6. Cross tabulation of anti-elitism and people-centrism being present in the same speech.

Anti-elitism	People-centrism
		Yes	No
	Yes	61	594
	No	4055	174,195

Figure 1. Scatter plot for percentage of people-centrism and anti-elitism in MEPs’ speeches.

Figure 2. Box plots for parties by left-right dimension.

Table A1. Keywords for positive and negative framing of the people and the elite.

negative frame					positive frame
abuse*	deprav*	hungry for power	political class	sins	betray*	noble
alienate	desecrat*	ignore*	power grabbing	spurn	blameless	ordinary man
annihilate	deserted	immoral*	power hungry	stain*	brave	ordinary men
apostasy	deserter*	impair	power monger	stomp	bravely	ordinary person
apostate	deserting	imperialist*	power-grabbing	sycophany	cleanliness	ordinary
apparatchik*	destroy	impiety	power-hungry	taint*	decency	praiseworthy
arrogan*	detriment*	impious	privilege	tarnish*	decent	principled
bad	discrimnat*	imposter	privileges	traitor*	defen*	reasonable
bad apples	disgust*	indecen*	profan*	transgress*	digni*	refined
betray*	dishones	inequitable	profligate	trashy	ethic*	the good
bias*	disloyal*	inhuman*	ravage	treacher*	expense of	upright
bigot*	disobe*	instable	refuse	treason*	fair	upstanding
blemish	disproportion	insubordinat*	refuse*	unacceptable*	forget	valor
Brussels mafia	disrespect*	insurgent	renegade	uncaring	forgetting	valour*
contagio*	dissociate	intemperate	repuls*	unchaste	hard earned	virtuous
corrupt*	dysfunctional*	jilt*	ruin*	unclean*	hard working
coward*	endanger*	lawless*	ruling circles	undemocratic*	hard-earned
cronies	enem*	lobby*	ruling circles	unelected	hard-working
crooked*	eurocrat*	miscreant	sediti*	unequal*	hardworking
cynical*	europhil*	mutinous	self-interested	unfaithful	honest*
damag*	evil	nonconformist	self-satisfied	unhealthy	honor*
debase*	favouritism	not in touch	self-serving	unresponsive	honour*
debauche*	filth*	obscen*	sequester	unscrupulos	ignor*
deceiv*	fraud	obstruct	sick	useless	interest
defector	greed	offend*	sin	venal	loyal
defian*	gross	old elites	sinful*	wantona	loyalt*
defile*	harmful*	oppose	sinned	wealthy	man on the street*
defy*	heretic*	out of touch	sinner*	wicked*	modesty
denounce	hot air	pervert	sinning	wretched*	moral
				wrongdo*	morally

Table A2. Frequency of keywords for people and elite dictionaries.

People	Count	Elite	Count
Total	183,980	total	13,967
people	74,047	banks	5725
citizens	57,482	politicians	4049
the public	9988	establishment	729
population	8570	elite	681
family	8336	politician	583
taxpayers	5653	circles	345
europeans	5140	mafia	345
families	4763	bosses	239
peoples	4004	eurocrats	236
citizen	3227	political class	153
taxpayer	1050	elites	109
residents	902	europhile	104
resident	643	europhiles	91
populations	124	eurocracy	83
ordinary person	43	financiers	65
man on the street	8	elitist	59
		apparatchiks	44
		oligarchy	37
		ruling class	34
		oligarchs	26
		eurocrat	24
		eurocratic	24
		political classes	24
		cronies	22
		financier	22
		oligarchic	20

Note: For the elite dictionary, only the 25 most-occurring keyword are depicted.

Table A3. Example for snippets.

Pre	Keyword	Post
European Parliament resolution vilifying Poland, the Polish	people	and Radio Maryja, the much-loved Catholic radio
of protection enjoy the same rights as long-term	residents	of that country within the scope of Directive
is an important signal to the banks and	citizens	of Europe . The financial crisis was largely
of very small businesses – often run by	families	- working in the sector of seaside establishments
our people have always cherished the most vulnerable	citizens	? Finally, Taoiseach, why have you
. A budget freeze is the minimum our	citizens	expect . I should like to see EU
the learning of foreign languages will bring European	citizens	closer together by giving them the means to
plays a part in democratic politics, as	the public	, if they are dissatisfied with the services
performing tasks that should be done by two	people	. Of European women, five per cent
we are going to explain this to the	citizens	. We are going to have Treaties 1
able to defend the interests of	hard-working	Dutch taxpayers yesterday .
. You are elected by the people to	defend	democracy . What do the selected

Table A4. Classification accuracy.

Bonikowski, B., and N. Gidron. 2018. “Populism in Legislative Discourse: Evidence from the European Parliament, 1999-2004.” Working Paper.

 Google Scholar

Rooduijn, M., and T. Pauwels. 2011. “Measuring Populism: Comparing two Methods of Content Analysis.” West European Politics 34 (6): 1272–1283. doi:10.1080/01402382.2011.616665

 Web of Science ®Google Scholar

Data availability statement

The data that support the findings of this study are openly available in [Harvard Dataverse] at https://doi.org/10.7910/DVN/9UGJVM

 Google Scholar

, reference number [9UGJVM].

<will be created (including replication material) upon publication>

The raw data from the EU Parliamentary Speeches can be fund via https://github.com/Connected-Politics-Lab/EUDD/tree/main/Datasets/EP_Corpus