Measuring Populism: A Quantitative Text Analysis of Party Literature in Belgium

Abstract

Despite the increased use of the term populism in vernacular and scholarly language, the measurement of the concept has long been neglected. The label is often attached to a certain party without any justification. Minimal definitions are alternatives but lack rigor. Classical content analyses provide more systematic measurements of populism but are extremely resource hungry. This article proposes an alternative, quantitative text analysis to measure the degree of populism among Belgian parties, drawing on both internally and externally oriented party literature. The results confirm that usual suspects such as the Vlaams Belang or Lijst Dedecker are the most populist of all parties under study. Populism turns out not to be an “either–or” concept, however, since we also identify a moderately populist party. It is furthermore demonstrated how populism can be attached to other ideologies, such as the radical right and (neo)liberalism. A cross‐validation of the proposed method with independent voter survey data confirms its validity. This article concludes that a quantitative text analysis might be a promising method to measure populism over time and space without the huge costs of hand coding.

Acknowledgments

I would like to thank Kenneth Benoit and Will Lowe for their course on quantitative text analysis at the ECPR summer school in methods and techniques. I am also grateful to Régis Dandoy and the M2P research group for providing me with the party manifestos. Finally I would like to thank Matthijs Rooduijn, Jean‐Benoit Pilet, Dave Sinardet, Pascal Delwit and the anonymous referees for their helpful comments on earlier versions of this article. The research for this article was made possible by the generous support of the Belgian science policy (Interuniversity Attraction Pole on Participation and Representation, ⟨http://www.partirep.eu⟩).

Notes

1. Theoretically it could be argued that a quantitative text analysis is 100% reliable since a computer will generate the same results no matter how many times one runs the analysis. At the same time there is often discussion about which texts should be analyzed. Since the results could differ depending on which sources are being used, it is flawed to consider quantitative text analysis perfectly reliable.

2. The line between internal and external sources is becoming more difficult to draw. For instance, many of the internally orientated sources can be found on the websites of political parties.

3. For a good introduction on Belgian politics, see Deschouwer (2009).

4. The level of political trust for the different electorates was the following: SP.a = 5.47; Groen! = 5.36; Open VLD = 5.47; CD&V = 5.6; N‐VA = 5.18; VB = 4.14; LDD = 4.91. For the degree of populism we used the average scores, being: SP.a = 2.2%; Groen! = 3.5%; Open VLD = 4.1%; CD&V = 5.7%; N‐VA = 10.5%; VB = 11.9%; LDD =15.5%.